Stack scoping with custom resource suffixes

[JensRoland]

Use Case

There are many cases where it can be useful to run multiple instances of the same application on a single cloud account:

• live development environments
• integration testing
• running blue-green environments during complex deployments or major data migrations
• for test and staging environments in single-account organizations
• and my personal favorite: for automatically generating test stacks for individual PRs

As a concrete example, let's assume that you have an App UserService in one repo which depends on an App PlatformInfra from a different repo. These are in different repos because they are owned by different teams and have very different risk and security models. You really want the ability to deploy multiple PlatformInfra instances at once; you might have your stable main-branch PlatformInfra stack running and also an alternative next-version-candidate PlatformInfra stack, which is still not quite done.
On top of this, the UserService team would have their stable main-branch UserService running on top of the stable PlatformInfra, but they would also like to have a PR branch of the UserService running on top of the next-version-candidate PlatformInfra for testing with the new infra.

To achieve this kind of collision-free "scoped deployment", all deployed resources must have unique ARNs / resource IDs (and sometimes names too, as in the case of domain names and S3 buckets which have global namespaces).

Some IaC tools make this simpler than others, usually by incorporating a randomly-generated 'StackId' into all resource identifiers. However, this mechanism of randomly generated resource IDs suffers from a few downsides:

• Resource IDs can not be predictably generated from outside the stack/app itself, complicating cross-stack resource imports
• Random resource IDs are not human-friendly, making it sometimes very frustrating to navigate a cloud account trying to identify which randomly-named Lambda function is the one you are looking for
• Random names are not useful for production resources that are part of a public 'interface'; if e.g. a bucket is automatically generated as "company-public-data-StackZ4vBno91zH" that is not a bucket name you want to share with external partners who are consuming your data, since 1) it is indistinguishable from a spoofed bucket by a nefarious third party, and 2) if you ever need to redeploy that stack for any reason, the bucket name could change, breaking the contract with your external partners.
• Cross stack (non-monorepo) dependencies become incredibly complex for multi-region deployments, wherein the IaC tool generates different random StackIds for each region and you need to ensure that your multi-region depending stack depends on the correct stack for its region.

Proposed Solution

At a previous company I built a CLI tool which handled this in a flexible and general way, and I would recommend taking a similar approach with Wing:

Instead of randomness, incorporate a custom stack suffix provided on compile (or ideally deploy) time. The wing compile command would take this suffix as an argument:

> cd ~/git/PlatformInfra
> wing compile --suffix next-candidate

# and

> cd ~/git/UserService
> wing compile --suffix pr-126

Cross-App dependencies would be specified similarly:

> wing compile --suffix pr-126 --dependencies PlatformInfra:next-candidate,MasterData:main,SharedEventBus:main

and the compile/deploy command will simply add the suffix to the IDs for all resources imported from that particular external stack.

Implementation Notes

The natural pattern for teams using this feature is to have a main suffix which corresponds to their stable main branch and have the tool always default to main for unspecified dependencies.

A convenient extension of this pattern is to combine the target environment and custom suffix, so you actually end up with suffixes like -pr-126-test and -main-prod. This is necessary so your test resources don't collide with your prod resources. These combined suffixes can be done by hand (the teams simply format the suffixes like that by convention) or enforced (Wing adding an --env parameter and combining the strings behind the scenes). In the tool I wrote, we went for adding an --env parameter, since this made our CI pipelines trivial to build.

This works incredibly well in practice, allows devs to work seamlessly with simple or multi-region dependencies and ad-hoc deployments for dev, testing or PRs without collisions or interference.

The complexity (from the perspective of implementing this into Wing) is in ensuring that suffixes are added to all the right IDs, that those IDs don't exceed character limits or use illegal chars, and in finding ways to implement this across different IaC toolchains. It's not trivial, but it makes for an incredible developer experience.

Component

Compiler, CLI

Community Notes

• Please vote by adding a 👍 reaction to the issue to help us prioritize.
• If you are interested to work on this issue, please leave a comment.
• If this issue is labeled needs-discussion, it means the spec has not been finalized yet. Please reach out on the #dev channel in the Wing Discord.