sam config - Improving the Configuration File in the AWS SAM CLI

[sriram-mv]

Status: Proposed

Type: AWS SAM CLI Standards

Created: 1/19/2023

Abstract

The AWS SAM CLI is a powerful tool for developing and deploying Serverless applications, but the current samconfig.toml configuration file has several limitations that make it difficult to use for certain use cases. This proposal outlines several improvements that could be made to the samconfig.toml file to enhance its functionality and usability.

Changes

The proposed changes include (not any particular order):

• A new sam config command that will allow to view and change the contents of the configuration file based on interactive prompts instead of having to go through the guided deploy flow or through a text editor of choice.
  • Allow to setup different environments
  • Validation while setting up the configuration file such that it’s not malformed as well just validation overall.
• Creation of a json-schema for samconfig.toml such that it has extensive IDE format while attempting to make changes in the IDE.
• Possibility of supporting an additional configuration format such as yaml in addition to TOML specifically due to user feedback at: TOML Alternative #2379
• Better documentation and examples: The current documentation for the samconfig.toml file is limited, making it difficult for new users to understand how to properly configure applications. Adding more detailed documentation and examples will make it easier for developers to use the AWS SAM CLI.
  • Example: sam config docs → Brings up a home page explaining all the tips and tricks with the configuration file and how to use it.
• Cascading configuration files that are present at home directory as well as at application level. This way certain properties such as cached and parallel builds can always be turned on (with a whole host of other things).
  • One example of a cascade: ~/.aws-sam/samconfig.toml → /Users/sam-user/apps/my-app/samconfig.toml
• A way to upload user’s configuration file over to AWS Cloud Service (TBD based on feedback) so that users can pair their configuration file with their application.

Conclusion

The proposed changes to the samconfig.toml file in the AWS SAM CLI would greatly enhance its functionality and usability.
We encourage the community to provide feedback on these proposals and contribute to the development of the AWS SAM CLI.

Acknowledgements

Sriram Madapusi Vasudevan would like to thank the community members in advance for providing feedback and suggestions for improving the AWS SAM CLI configuration file.

[singledigit]

I like the idea of updating the functionality of the file. I like the proposed ideas with some caveats. I have a few points of feedback.

1. I think the cascading feedback is redundant to the profile creation in the template. In the example shown, we are encouraging developers to make configuration changes in the .aws-sam folder. This folder should never hold persistent data, as we often overwrite it.
2. If we are going to introduce a new format, I would encourage that we do it to deprecate TOML.
3. Agree with the lack of documentation. However, I filled the gap a bit with this if you need it.

[sriram-mv]

1. This directory houses files like metadata.json too where users can turn off telemetry for example. Might be interesting to see if users would like to say --skip-pull-image to be always turned on in their dev environment.
2. Interesting! Might need to think of strategy on how to notify users of that, since its integral to the workflow.
3. That is a good data-point, perhaps it could be a temporary landing page for sam config docs. 🤔

[buggy]

In order:

1. Change the format to YAML or JSON.
2. Add a sam config which allows getting/setting individual options
3. Improve the docs

There's a really weak use case for cascading config files where you want to check a copy into version control as defaults. Currently we handle this with setup instructions in the README file and I'm perfectly happy with that solution

[sriram-mv]

Thanks! @buggy On the improving of docs, are there any specific scenarios you would want to highlight?

[jasonterando]

• +10 on YAML. Please give us some deprecation warnings for a while if you are going to phase out TOML. I'd spend the energy on optimizing the YAML implementation as opposed to spending time on json-schema for TOML
• A cascading default configuration in .aws-sam seems problematic, but one in ~/.aws (or Windows equivalent) may prove useful useful. Developers could have their own local definitions (references to private AWS account resources) and CI/CD build images could have enterprise defaults baked in. Seems like a nice way of dealing configuration for stuff like VPC subnets, endpoints, certificate ARNs, etc.
• Nice to have - Validation of the parameter overrides against what's defined in template.yaml (names and types), maybe implement with optional parameter file setting on sam validate?

[sriram-mv]

Thanks! I like the idea of validating an application's config too. How often do you go back and edit things in your configuration file? Where do you think we could improve our UX there?

[jasonterando]

In general we work with config files:

1. During development (adding new stuff to the application and SAM template)
2. Post-development: Adding configurations for post-dev environments (QA, staging, production)
3. Post-deployment: Added new features that require new AWS resources

[sriram-mv]

Interesting @jasonterando could you elaborate on point 3? What kind of features does samconfig.toml expose today that helps with new resources that you like?

[jasonterando]

Just if we add a new application feature that requires access to an AWS resource, we typically would add a SAM template parameter, and then update our SAM config files with the value for that parameter appropriate to our deployment targets.

[igorlg]

• Support setting up cross-environment parameters

At the moment, it is possible to set parameters common to all sections of any given environment in the [<environment>.global.parameters] section, e.g.:

[dev.global.parameters]
stack_name = "my-app-dev"
resolve_s3 = true

[dev.build.parameters]
use_container = true

[dev.deploy.parameters]
parameter_overrides = "Environment=\"dev\""

However, when defining multiple environments in samconfig.toml, there are cases where the same config key/value needs to be set across all environments. Today, this requires duplication and is potentially error-prone.

It would be great if one could set common parameters across environments only once. These could be applied to specific SAM subcommands (e.g. deploy, build), or all subcommands, like the current behaviour of global:

[ALL.build.parameters]
use_container = true
resolve_s3 = true

[dev.global.parameters]
stack_name = "my-app-dev"

[dev.deploy.parameters]
# [...]

[prod.global.parameters]
stack_name = "my-app-prod"

[prod.deploy.parameters]
# [...]

[sigJoe]

I would absolutely love an overhaul of parameter_overrides. It would just be so handy and elegant if sam cli natively merged key-value pairs so we could do something like this in one command without any custom scripting:
sam deploy ... --parameter-overrides file://global.env --parameter-overrides file://dev.env --parameter-overrides file://team.env --parameter-overrides deploy_version=${BRANCH}-${COMMIT}

[rhbecker]

I recently created issue #5492, before realizing this discussion existed. My issue doesn't add a ton to what's already been stated here, other than a nice-to-have: supporting regex or glob-style pattern matching in the environment references.

[cscetbon]

It seems crazy that it's not supported yet ... Same with YAML anchors, something like this could be easily done if only the parser of SAM CLI supported it

version: 0.1

common:
  deploy:
    parameters: &common_parameters
      s3_bucket: my-bucket
      region: my-region
      ...
      tags:
        - tag1=val1
        - tag2=val2
        ...

stg:
  deploy:
    parameters:
      <<: *common_parameters
      stack_name: my-stack-stg
      parameter_overrides: StageName=stg

uat:
  deploy:
    parameters:
      <<: *common_parameters
      stack_name: my-stack-uat
      parameter_overrides: StageName=uat

prod:
  deploy:
    parameters:
      <<: *common_parameters
      stack_name: my-stack-prod
      parameter_overrides: StageName=prod

[lucashuy]

Hey @cscetbon, we've released support for YAML formatted samconfig files in SAM CLI v1.91.0. Using anchors like you've outlined in the code snippet should work in this version. If you end up trying it out and it does not work, feel free to let us know and we can investigate further.

[cscetbon]

Thanks @lucashuy I'm going to give it a try, last time I checked I was indeed on an older version.

[SimonCMoore]

Could a mechanism be added to make sharing configurations easier e.g. point it at your companies GitHub repo or include default settings in projects that are then overridden by local ones or interactively worked through.

[igorlg]

Yes!! Perhaps a global ~/.config/sam/samconfig.toml, where configs take precedence over defaults, but are overritten by any others!

[ajantrania]

I generally like the ideas proposed here but I wonder if there is something more that we could do to make it so that users don't have to care/know about samconfig by auto-populating (some? all?) sam config as users use the CLI.

Using the example in sam-config.md - If the user want's to do the following workflow:
sam build --use-container --template ... --parameter-overrides=... --skip-pull-image --manifest ...

sam package --s3-bucket ... --template-file ... --output-template-file ... --s3-prefix ... --kms-key-id ...

sam deploy --template-file ... --stack-name ... --capabilities ... --tags ... --parameter-overrides ... --kms-key-id ...

Currently, the suggestion is that the user should pre-populate the config. But that's maybe hard to find/realize it exists and not really top of mind for the user as they are doing their work.

Could SAM instead auto-save the users parameters such as the following:

sam build --use-container --template ... --parameter-overrides=... --skip-pull-image --manifest ...

Then, for sam package, the user wouldn't need to provide --template:
sam package --s3-bucket ... --output-template-file ... --s3-prefix ... --kms-key-id ...

And for sam deploy the user wouldn't need to provide template, parameter-overrides, kms-key-id as those were provided previously
sam deploy --stack-name ... --capabilities ... --tags ...

Where this really shines is the next time the user does this flow, they get the full benefits of having pre-populated their samconfig without the work, as they could just run:
sam build
sam package
sam deploy

Implied in this flow is that the parameters are not saved at the command scope, but higher. Not sure what's right though - environment, global, or something else.

This would remove a lot of friction for the user without them actually needing to learn about samconfig in the first place.

There's open questions in my head about if this falls apart for certain sam commands or applies less for some parameters that you all would know far better than me.

[igorlg]

Love this idea! This could be as simple as adding a global argument (e.g. --save) to persist all CLI arguments - or perhaps all resolved parameters - to samconfig.toml!

[Amir-Ahmad]

A few more requests:

1. Ability to reference cloudformation stacks output. So you could create a cloudformation stack separately for things like the deployment bucket, and then reference it. e.g.

[default.deploy.parameters]
s3_bucket = "${cf:serverless-infra:BucketName}" 

Here serverless-infra is a cloudformation stack deployed to all accounts, and BucketName is a cloudformation output in it. SAM would do a lookup to cloudformation and resolve this prior to deployment. In the future this could be extended to support other lookups like environment variables, Parameter store, etc.

2. Ability to append to parameters from cli instead of overriding

Right now if you specify 5 parameters in samconfig.toml, and a 6th one is sensitive and won't be committed, if you do sam deploy --parameter-overrides you'll need to provide the values of all 6 parameters and not just the sensitive one.

[igorlg]

And while we're at it, doing the same as CloudFormation's {{resolve:ssm:...}} would be really cool as well!

[kohidave]

Howdy ya'll! This sounds really nice - I'm excited for evolving the SAM config 🎉

I'm curious what ya'll are considering for the scope of the new config? Will it continue to focus around needed build & local configuration like it is today - or are you thinking it might be more application-specific configuration?

Either way - exciting stuff!

---

I spent a while working on the Copilot CLI manifests (the nearest equivalent to SAM configs) and thought I'd share some of our tenets / decisions. Obviously these won't all apply for ya'll - but food for thought! 👩‍✈️

Tenet 1: Make manifest files portable 💌

I don't know if this applies so much to your use-case, but we wanted folks to basically be able to fork a github repository with a copilot application and be able to run copilot init and get the same app up and running in their account.

This meant that generally, we'd try to keep things parameterized rather than hardcoded. Things like the S3 bucket name would be stored in SSM instead of directly in the manifest, and there'd be a naming convention for the SSM parameters /{copilot app}/{copilot workload}/{environment}/artifact-bucket-name.

We couldn't always do this with more specific usecases like DNS - but I'm pretty happy with it!

But it's important to us to make sure we give our customers a super easy way to handle paramaters like secrets, etc - so using this naming convention scheme in SSM makes it really easy. example of secrets usage

Tenet 2: Defaults & Inline documentation 📓

Copilot generates a default manifest.yaml file for you - and it's filled in with the basics of your app. The generated config also includes "commented out" common knobs for folks - so that if they want to enable something [say, autscaling of fargate tasks] they'd just need to uncomment out the line. Looking up documentation is basically never fun - so we found this helped most folks!

Tenet 3: You can overwrite anything per environment 🪆

Most configuration is universal per environment [or can be easily parameterized] - but for environment specific configuration, we wanted to make it super easy to just overwrite the top level values with custom values per environment.

name: api
type: Backend Service
# default for all environments
cpu: 256
memory: 512
count: 2
exec: true

# environment specific values
environments:
  prod:
    count: 10

[sriram-mv]

All these are great points! Thanks for this!

[allanchua101]

I'd love to have environment specific variables too but instead of having all in the same file, I was thinking of having them on a separate file just like how it works on react, vue and other popular development frameworks.

samconfig.toml # Global file

# Environment specific files
samconfig.dev.toml
samconfig.prd.toml
samconfig.uat.toml

# If we think about it, we can even support developer-specific environments
samconfig.tom.toml
samconfig.john.toml
samconfig.bill.toml

If we think about it, this approach offers the following benefits:

• Its easier to scale number of configuration files by copy-pasting and swapping out values on the copied file
• We can exclude production configuration files from the source code and just pull it on CI/CD which helps reduce the risk of misconfiguration from development activities
• Each engineer can define his own configuration file targeted at his AWS account if a team uses one account per developer. This would significantly prevent conflicting ARNs for references of secrets manager type of imports in SAM files.

[sriram-mv]

This works today! you can pass in custom files as you see fit. Could you go deeper? do you want environments to be config file specific?

[sriram-mv]

Hey All! Thanks a ton for the comments! Keep 'em flowing. At this point, we will take the feedback that we have and start building out the design out in the open in the coming weeks. Please continue the discussion ❤️

[jasonterando]

Hi @sriram-mv , just curious which of these ideas have traction with your team and are undergoing development? One of the things that has come up on our end is the following scenario:

1. We have a number of Serverless APIs (API Gateway + Lambda) that will be deployed to multiple locations
2. We are using a template.yaml + configuration TOML to hold information
3. Having to store things like Lambda subnets, Route 53 domains, certificates, etc. for each domain in every API seems painful
4. It would be nice to have an "out of the box" way where one could be able to inject, maybe from Parameter Store, centrally stored values for these types of things, using syntax maybe something like this:

      VpcConfig:
        SecurityGroupIds:
          - !ParamStore !Sub /configs/${DeployEnvironment}/DefaultVpcSecurityGroupID
        SubnetIds:
          - !ParamStore !Sub /configs/${DeployEnvironment}/DefaultVpcSubnet1ID
          - !ParamStore !Sub /configs/${DeployEnvironment}/DefaultVpcSubnet2ID

[lucashuy]

Hi, we recently release YAML file support for writing samconfig files in SAM CLI v1.91.0. We are currently working on JSON schema file generation, you can track the current progress in it's branch to see how things are going so far.

As for no.4, Cloudformation provides a way to dynamically reference values (from locations such as Parameter Store) into a template file. Does this solve the particular use case you've outlined?