Add full support for stages

[jamesls]

Right now, the stages feature needs more work to be feature complete. The proposed idea is:

• chalice deploy prod - Will create a new API gateway stage if necessary
• The lambda function code is zipped up and sent to lambda. A new function version is created
• A lambda alias for the the stage name points to the new function version updated.

I also need to update the API gateway integration request to forward to the appropriate lambda function based on the stage name. I'll need to add a context variable with the stage name.

I'd also like to add an optimization to just update lambda aliases when you go from dev->prod where the lambda code has not changed. In that case we can just flip the alias.

[garnaat]

What is the stage in API Gateway buying us in this scenario? This isn't really a Chalice-specific question. This is a general question about Lambda and API Gateway although I think it's important that the approach we build into Chalice make sense.

Stages seem to be an attempt to provide some sort of resource isolation but I think accounts are really a much better way to accomplish isolation because they can be applied to all resource types, not just API Gateway endpoints.

So, if we are using accounts for isolation it seems like we would just deploy new versions of functions within the appropriate account, move aliases around as necessary and stages would be superfluous.

Thoughts?

[timff]

Yes, I agree accounts are much better way for resource isolation.

In my opinion, stage only required if you have multiple environments in one account, for example, DEV and TEST in one account, while PREPROD and PROD in another. But this is really API Gateway specific.

[JamieCressey]

In my mind, stages are a must for dev, test, prod environments. Creating additional accounts will not be an option for everyone.

[garnaat]

@JamieCressey I'm curious how you would isolate other resources in that scenario. For example, if you had a DynamoDB table that the API was fronting.

Also curious why additional accounts would not be an option. What would be the barrier?

[JamieCressey]

@garnaat assuming we are talking about AWS accounts here, there's a multitude of reasons why a corporation would have problems here; cost consolidation being one.

Depending on what isolation you mean, in our current setup (using Serverless) we have a custom role per Lambda function (and therefore API endpoint) which restricts access to the bare minimum per function.

[garnaat]

I am talking about AWS accounts.

In my experience most people rely on AWS accounts for resource isolation between prod accounts and dev/test accounts. I'm not saying there aren't other ways to get reasonable isolation but accounts are pretty bulletproof.

So I'm curious how people accomplish sufficient isolation for non-API Gateway resources using stages.

[mannytoledo]

@JamieCressey On a billing level, AWS is more than willing to aggregate all costs into one account. This way you can have 1 place to go for billing but can still have a logical divide of multiple accounts.

AWS and most large corporations suggest multiple accounts for security reasons. It reduces the risk and complexity of isolating access to parts of infrastructure.

I've seen the model where there are 3 api endpoints/lambas for each tier, and it works well. It would really be nice to also have a multi-account model. Maybe a place where credentials can be configured in per tier or even using AWS profiles?

[sdole]

Could be just a matter of operational preference. We don't prefer multiple accounts. We do deal with ongoing IAM operations for safely isolating resources. Cost allocation is done via tags. Admittedly, we are on year 2 of our AWS migration, it might change after we are all in.

[jamesls]

There's a PR with a first attempt at this approach: #264
Hopefully this is a reasonable compromise of the suggested approaches so far:

• API Gateway "stages" are no longer used.
• A "chalice stage" is a completely separate set of AWS resources (APIG, Lambda function, IAM role, etc).
• You can still use separate accounts for isolation (in this case you'd specify a --profile to a different account if you were deploying locally)
• You can have multiple chalice stages in the same account if you want. Also allows for nice integration with CodePipelines.

So for example, running:

$ chalice deploy dev
...
$ chalice deploy prod

Will result in 2 APIG APIs, 2 lambda functions, 2 roles, etc. I'm also adding lambda env var integration so if you had other resources such as DDB tables, you could configure a separate table per stage and specify the table names vie environment variables that are passed to the lambda function.

[jamesls]

Closing, #264 has been merged.