[RFC] SAM Accelerate (Cloud based development)

[praneetap]

We are working on adding a new set of features of the SAM CLI (Serverless Application Model Command Line Interface) that allows you to test against your application in AWS cloud in a production like environment, which you can quickly and iteratively build . Using the sam sync --watch capability lets SAM CLI monitor for any changes to AWS resources defined in your application and automatically sync them in your developer account in AWS cloud so you don't have to run manual deployments. We are also adding the ability to tail aggregated AWS CloudWatch logs and AWS X-Ray traces in real time.

To summarize, we are working on the following features:

1. The ability to quickly synchronize changes between the local development environment and AWS cloud environment
2. The ability to “tail” aggregated logs for all resources in an application
3. The ability to pull X-Ray traces related to your latest test executions

Want to try out these features in Beta? Instructions are here.

As an enhancement we are considering implementing development environments in a SAM application, tied to an AWS profile and AWS CloudFormation stack.

We would love your feedback/thoughts on this feature! Please feel free to add comments in this issue.

1. What are your thoughts on the sam sync --watch command?
2. What are the biggest pain points in moving to AWS cloud based development in your work environment?
3. What other features would you like to see added here?

Features in development

Project synchronization

The sync command will first check that the CloudFormation Stack exists and then run UpdateStack. After this one-off, initial update of the infrastructure, the SAM CLI will start monitoring local files. Specifically, the SAM CLI will monitor the infrastructure definition files such as CloudFormation template; and the source code for API definitions, State Machine templates, Lambda functions and Layers. Whenever the SAM CLI detects a change, it automatically synchronizes the changes with the AWS cloud using the service APIs and notifies the developer through a message in the console.

$ sam sync --watch --stack-name {stack-name}
  Looking for resources needed for deployment: Found!

        Managed S3 bucket: aws-sam-cli-managed-default-samclisourcebucket-abcde
        A different default S3 bucket can be set in samconfig.toml
        Or by specifying --s3-bucket explicitly.
Queued infra sync. Wating for in progress code syncs to complete...
Starting infra sync. 

.
.
.
Stack update succeeded. Sync infra completed.

{'StackId': 'arn:aws:cloudformation:us-east-1:123456789:stack/sam-app/abcde-12345', 'ResponseMetadata': {'RequestId': 'guid', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': 'guid', 'content-type': 'text/xml', 'content-length': '377', 'date': 'Thu, 09 Sep 2021 20:41:14 GMT'}, 'RetryAttempts': 0}}
Infra sync completed.

Fetching logs

We are improving sam logs to fetch aggregated logs across multiple AWS services for a CloudFormation stack, and --tail option to fetch them in real time.

$ sam logs --stack-name {stack-name} --tail
2021/09/09/[$LATEST]f9a86d8186be41f580de502740063f78 2021-09-09T21:13:04.750000 START RequestId: e524a367-3194-44f0-bf22-5c688129daec Version: $LATEST
2021/09/09/[$LATEST]f9a86d8186be41f580de502740063f78 2021-09-09T21:13:04.753000 END RequestId: e524a367-3194-44f0-bf22-5c688129daec
2021/09/09/[$LATEST]f9a86d8186be41f580de502740063f78 2021-09-09T21:13:04.754000 REPORT RequestId: e524a367-3194-44f0-bf22-5c688129daec 
Duration: 1.46 ms Billed Duration: 2 msMemory Size: 128 MB Max Memory Used: 48 MB Init Duration: 114.21 ms

Traces

Use SAM CLI to pull X-ray traces using sam traces. You can optionally include a trace-id and use the --tail option to get traces in real time. There is also a --include-traces option with sam logs command to aggregate traces with logs.

$ sam traces --trace-id {trace-id} --tail
XRay Event at (1970-01-19T21:07:02.583000) with id ({trace-id}) and duration (0.005s)
 - 0.005s - build2-HelloWorldFunction-V6h40pCqh7Ma [HTTP: 200]
 - 0.001s - build2-HelloWorldFunction-V6h40pCqh7Ma
   - 0.001s - Invocation
   - 0.000s - Overhead

Features in design

Environment configuration

This is a proposed design for sam env which will manage environment configuration on SAM CLI.
Developers can define a “dev” environment for the SAM CLI to operate against. A development environment can be in any account that the developer’s local AWS credentials have access to, however SAM CLI will have guardrails against using “prod” environment. The SAM CLI will first attempt to use the AWS credentials and CloudFormation stack active for the project; if no AWS credentials or CloudFormation stack are configured the SAM CLI asks developers to select an AWS profile, and a CloudFormation stack or create a new one.

// Creating first environment
$ sam env create —name dev
Creating new environment...

New environment: dev successfully created. (#ms)

Your environment: dev is now associated with profile: default and project: Test-app-1. To change the associated AWS profile or project, run the command:
sam env edit —name dev
X-Ray and CloudWatch logs have been enabled on the event sources in your project.

//Trying to create a local environment
$ sam env create —name local
Creating a local environment means it will not be associated with any cloud resources. You can still conduct local testing with select resources.
Do you want to create environment: local anyway? yes/No [y/n]:

//Guardrails against creating a prod environment
$ sam env create —name prod
SAM environments are iterative test resources. It is not recommended to create a production environment.

Do you want to create environment: prod anyway? yes/No [y/n]: n

The SAM CLI saves the development environment information in its configuration file in the local project folder:

…
[environment-dev]
AWSProfile = DevProfile
StackName = devDataProcessingStack 
ProjectType = CloudFormation

[praneetap]

If you would like to try these features out in Beta, here are the instructions to get it running! Please give it a spin and give us feedback on how we can improve this feature.

[lazyDude71982]

@praneetap I have seen the sam accelerate demo in serverless office hours and I see it will great addition to sam cli. But I have a doubt, If team members who are working on same lambda function they cannot use sam sync, right?
If no is there any way to make it possible?

[praneetap]

@praneetap I have seen the sam accelerate demo in serverless office hours and I see it will great addition to sam cli. But I have a doubt, If team members who are working on same lambda function they cannot use sam sync, right? If no is there any way to make it possible?

That is the case today, sam sync is tightly bound with the account and stack name. Typically one would use sam sync before git push changes onto the main branch where collaboration is happening. If there arent any shared resources, each dev can use a different stack name to test out their changes.

[ben-elsen]

@praneetap I am a huge fan of AWS SAM (cli) and I am exited to test this feature! I followed the instructions but I am stuck now on the following error: botocore.exceptions.NoCredentialsError: Unable to locate credentials

The command samdev sync --watch --stack-name eventbroker-app-stack-dev -t sam.yml --profile assumeAdminINMSEventbridgeDev went well and I got the output:
Stack update succeeded. Sync infra completed.

{'StackId': 'arn:aws:cloudformation:eu-west-1:732279076585:stack/eventbroker-app-stack-dev/9b6359f0-2378-11ec-8628-0acbb15b3ae5', 'ResponseMetadata': {'RequestId': 'ba23507c-a9cf-42f9-a412-2cba397f59e0', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': 'ba23507c-a9cf-42f9-a412-2cba397f59e0', 'content-type': 'text/xml', 'content-length': '395', 'date': 'Mon, 11 Oct 2021 14:13:28 GMT'}, 'RetryAttempts': 0}}

but then I immediately got the error'
output terminal.txt

[praneetap]

@praneetap I am a huge fan of AWS SAM (cli) and I am exited to test this feature! I followed the instructions but I am stuck now on the following error: botocore.exceptions.NoCredentialsError: Unable to locate credentials

The command samdev sync --watch --stack-name eventbroker-app-stack-dev -t sam.yml --profile assumeAdminINMSEventbridgeDev went well and I got the output: Stack update succeeded. Sync infra completed.

{'StackId': 'arn:aws:cloudformation:eu-west-1:732279076585:stack/eventbroker-app-stack-dev/9b6359f0-2378-11ec-8628-0acbb15b3ae5', 'ResponseMetadata': {'RequestId': 'ba23507c-a9cf-42f9-a412-2cba397f59e0', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': 'ba23507c-a9cf-42f9-a412-2cba397f59e0', 'content-type': 'text/xml', 'content-length': '395', 'date': 'Mon, 11 Oct 2021 14:13:28 GMT'}, 'RetryAttempts': 0}}

but then I immediately got the error' output terminal.txt

Hi @ben-elsen! Thanks for your feedback. While we are taking a look at this, could you please confirm the profile exists locally and is valid?

[ben-elsen]

@praneetap thanks for the fast response! I have just checked the profile and it exists and is valid.
The sam deploy command works perfectly.

[ben-elsen]

Hi @praneetap , I just tried sync command with a clean app and the same problem arises. I created a new sam app with sam-init and I chose the Hello World one. I haven't changed anything, sam deploy goes well but with the samdev sync command, the same error shows.

[mndeveci]

Hi @ben-elsen , thanks for reporting this issue. We find out that --profile option is not taken into account in some parts of sam sync flow. We will be working on to fix this problem. For a workaround, you can update your default profile with the one that you are trying to use, which should resolve the issue.

[ben-elsen]

Thanks @mndeveci ok thanks for the update! Do you have an estimate about when the fix will be ready? Changing the default problem will cause me a bit too much work I think and I am just in the testing phase with sam sync..

[mndeveci]

@ben-elsen The PR for the fix is already open, can you give it a try? #3368

[ben-elsen]

Hi @mndeveci , I am sorry that I wasn't able to respond earlier but illness was my part..
I am trying the fix but now he always syncs the infra, any idea why? I just updated the code of my lambda that is linked to the rest api.
Syncing Lambda Function Api/IngestionFunction... Building codeuri: /Users/benelsen/Engineering/Air/eventbroker-App-Stack/api/ingestion runtime: python3.9 metadata: {} architecture: x86_64 functions: ['Api/IngestionFunction'] requirements.txt file not found. Continuing the build without dependencies. Running PythonPipBuilder:CopySource Missing physical resource. Infra sync will be started. Queued infra sync. Wating for in progress code syncs to complete... Starting infra sync.

[mndeveci]

@ben-elsen no worries at all. Is this happening when you run samdev sync --watch? If you can give us a reproducible example we can investigate what is the issue here.

[ben-elsen]

@mndeveci Yes it is, I will add the output. I cannot add the template as it is not supported. It is a basic stack with a nested api stack. The api stack is a AWS::Serverless::Api resource with one resource that is connected to a simple lambda via an open api spec. The lambda only outputs a json response. That response is what I chance to test the samdev sync command.
Output iterm.txt

[cayter]

Hi @mndeveci , I am sorry that I wasn't able to respond earlier but illness was my part.. I am trying the fix but now he always syncs the infra, any idea why? I just updated the code of my lambda that is linked to the rest api. Syncing Lambda Function Api/IngestionFunction... Building codeuri: /Users/benelsen/Engineering/Air/eventbroker-App-Stack/api/ingestion runtime: python3.9 metadata: {} architecture: x86_64 functions: ['Api/IngestionFunction'] requirements.txt file not found. Continuing the build without dependencies. Running PythonPipBuilder:CopySource Missing physical resource. Infra sync will be started. Queued infra sync. Wating for in progress code syncs to complete... Starting infra sync.

Facing the same issue, I checked the source code, it seems the watcher/observer isn't ignoring the .aws-sam folder that leads to infinite re-compile & re-sync. Is there a way for us to specify the folders to ignore for the watcher/observer?

PS: Currently on MacOS Big Sur amd64 with SAM 1.34.1.

[mndeveci]

Sorry for the inconvenience @cayter, this is a known issue that we will be prioritizing soon.

For a workaround, I would recommend to have a separate folder for function code, and update your resource's CodeUri value with that folder.

@ben-elsen, sorry for the late response, but we released Accelerate features withing regular SAM CLI. Can you install current version and give it a try? And please let us know if the use-case is different than what @cayter described above.

[cayter]

@mndeveci No problem, thanks for confirming.

[ben-elsen]

@mndeveci, I can understand that totally! I just updated to version 1.35 but the problem remains. I don't know if I understand your workaround because I already have my lambda code in a handler.py file in a subfolder 'ingestion' but still, whenever I update code in the handler.py, the whole infra stack starts syncing.

[mndeveci]

@ben-elsen can you provide us steps to re-produce so we can take a closer look at it? You may consider to create a new GH issue so that we can follow-up the conversation separately there.

[ben-elsen]

@mndeveci I opened #3454
I tried to be complete, happy to hear if I must provide more info!

[heitorlessa]

I loved this feature! The only thing I missed was a way to run builds within a container as I have some C-deps and SAM Accelerate doesn't work as it tries to build it locally for speed -- Workaround was to build a layer first then run sam sync

[jfuss]

Accelerate is GA as of v1.53.0.

Closing