feat: Add SAM Metadata resources to enable the integration with SAM CLI tool #325

moelasmar · 2022-07-08T05:21:10Z

Description

This change is to add new resources (sam_metadata_*) of type (null_resource) that will maintain some information about the lambda resources exist in the terraform application to enable the integration with SAM CLI. SAM CLI tool is looking for the lambda resources source code, and the resources that contain the building logic of these resources. SAM CLI can use this information to extend the SAM CLI testing capabilities to the terraform applications.

Motivation and Context

This change will enable the customers who use Serverless TF module to define Lambda resources to integrate with SAM CLI tool, and so the customers can test their applications either locally or on cloud.

Breaking Changes

no breaking changes.

How Has This Been Tested?

I have updated at least one of the examples/* to demonstrate and validate my change(s)
I have tested and validated these changes using one or more of the provided examples/* projects
As the new resource is just to collect metadata for SAM CLI tool, so it does not require any changes from the modules users side, and so no change in the examples. So I tested that I can run terraform apply for the available examples without any issues.
I have executed pre-commit run -a on my pull request

…LI tool.

moelasmar · 2022-07-08T05:22:20Z

@bryantbiggs .. Could you please check this PR.

antonbabenko

Looks pretty good. I am excited to see an additional integration with SAM, thank you! It would be great to have at least a paragraph saying about this feature and commands to use this with SAM in readme or in example. (If there is no notion about something in terraform-aws-modules - feature doesn't exist)

antonbabenko · 2022-07-08T10:25:44Z

main.tf

+    resource_type = "ZIP_LAMBDA_FUNCTION",
+
+    # The Lambda function source code.
+    original_source_code = jsonencode(var.source_path),


source_path can be rather complicated type (see examples or readme), I am not sure this will be well supported by SAM. Could you please verify this?

In Sam Cli, we expect the value of this property to be either String or JSON object. In case if it is a JSON object as we have here in serverless TF module, SAM CLI will expect another property source_code_property to let SAM CLI know, where it can find the Lambda resource code path. In Sam Cli , we do not support multiple paths, so in case if the value of this attribute is a list (of strings, or objects), SAM CLI will process only the first item in the list.

moelasmar · 2022-07-08T12:00:00Z

Thanks @antonbabenko for your quick response. Regarding the Readme update, is it ok to add a new section regarding SAM CLI integration before the FAQ section in this Readme file

bryantbiggs · 2022-07-19T11:49:44Z

@moelasmar I would put it under the How to deploy and manage Lambda functions

moelasmar · 2022-07-28T21:58:20Z

@moelasmar I would put it under the How to deploy and manage Lambda functions

I founded that it is better to have a separate section for SAM CLI integration, as we plan to support testing only for now, and no plan to support package and deploy in the current project.

@antonbabenko, @bryantbiggs could you please have a look, and please let me know if you have any comments.

…olution

antonbabenko

I looked at it in details after I had read the specs. In general, this PR looks pretty good, but I was not able to test it for real without building SAM with your changes.

There is one hard part I want to highlight:

Instead of doing the whole cycle for AWS resource extraction (terraform init, plan into a file, show as jіon, jq, python, etc), I recommend that this module optionally creates a local file in the known location (if the user specifies like create_sam_metadata = true), SAM just reads the captured JSON object and Prepare Hook will generate CFN Template (or even Terraform). This will be much faster, more flexible, and transparent for the user.

Maybe I don't know the whole picture, so please correct me (you have my email, too). I will be glad to cooperate with you on this (I am back from vacation, finally)!

antonbabenko · 2022-08-19T06:48:16Z

README.md

+applications. Currently, SAM CLI tool only supports CFN applications, but SAM CLI team is working on a feature to extend the testing capabilities to support terraform applications (check this [Github issue](https://github.com/aws/aws-sam-cli/issues/3154) 
+to be updated about the incoming releases, and features included in each release for the Terraform support feature).
+
+SAM CLI provides two ways of testing, local testing, and testing on-cloud (Accelerate).


Suggested change

SAM CLI provides two ways of testing, local testing, and testing on-cloud (Accelerate).

SAM CLI provides two ways of testing: local testing and testing on-cloud (Accelerate).

antonbabenko · 2022-08-19T13:02:19Z

main.tf

+  triggers = {
+    # This is a way to let SAM CLI correlates between the Lambda function resource, and this metadata
+    # resource
+    resource_name = "aws_lambda_function.this[0]",


Please remove comas after each value on the map.

moelasmar · 2022-08-19T20:48:38Z

Thanks @antonbabenko for your comment. Let me think more about it, also we can discuss it more in the meeting we have next week.

antonbabenko · 2022-08-24T08:46:30Z

Hi @moelasmar !

Thanks for your time and yesterday's discussion about this PR and SAM/TF integration.

If you don't mind, I'd like to keep this PR open until I can run SAM CLI locally myself and verify that it works as expected. It also doesn't look good for the users if we include SAM CLI commands in the docs, but users can't run them because SAM CLI doesn't have them available (not even in sam-nightly).

…olution

…module

…integration_null_resource_solution' into master_sam_cli_integration_null_resource_solution

…olution

antonbabenko

LGTM

antonbabenko · 2022-10-31T15:05:14Z

modules/docker-build/main.tf

+# This resource contains the extra information required by SAM CLI to provide the testing capabilities
+# to the TF application. This resource will maintain the metadata information about the image type lambda
+# functions. It will contain the information required to build the docker image locally.
+resource "null_resource" "sam_metadata_docker_registry_image_this" {


Can we drop _this from the name in resources?

…olution

antonbabenko

Looks good!

antonbabenko · 2022-11-03T10:45:19Z

Great work, @moelasmar !

## [4.6.0](v4.5.0...v4.6.0) (2022-11-03) ### Features * Add SAM Metadata resources to enable the integration with SAM CLI tool ([#325](#325)) ([bfcd34c](bfcd34c))

antonbabenko · 2022-11-03T10:45:31Z

This PR is included in version 4.6.0 🎉

moelasmar · 2022-11-03T13:20:19Z

Thanks @antonbabenko for your support

github-actions · 2022-12-04T02:17:42Z

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

moelasmar added 3 commits July 7, 2022 21:44

feat: Add SAM Metadata resources to enable the integration with SAM C…

eff1b03

…LI tool.

fix the formatting issues

e73faa3

updating the readme docs

39e45b6

antonbabenko requested changes Jul 8, 2022

View reviewed changes

Adding some documentation about using SAM CLI for testing.

8acaa09

moelasmar requested a review from antonbabenko July 28, 2022 21:56

Merge branch 'master' into master_sam_cli_integration_null_resource_s…

d227d67

…olution

antonbabenko reviewed Aug 19, 2022

View reviewed changes

resolve pr comments

f681324

moelasmar requested a review from antonbabenko August 23, 2022 08:07

antonbabenko and others added 7 commits September 22, 2022 12:35

Merge branch 'master' into master_sam_cli_integration_null_resource_s…

f4b0756

…olution

Merge branch 'master' into master_sam_cli_integration_null_resource_s…

e3312fc

…olution

add built_image_uri to the sam metadata resource in the build docker …

c9fd1a2

…module

Merge remote-tracking branch 'moelasmar-serverless-tf/master_sam_cli_…

dc2e8d7

…integration_null_resource_solution' into master_sam_cli_integration_null_resource_solution

Merge branch 'master' into master_sam_cli_integration_null_resource_s…

088ee24

…olution

Merge branch 'master' into master_sam_cli_integration_null_resource_s…

89f9eb5

…olution

Fixed style a bit

3acd612

antonbabenko changed the title ~~feat: Add SAM Metadata resources to enable the integration with SAM CLI tool.~~ feat: Add SAM Metadata resources to enable the integration with SAM CLI tool Oct 31, 2022

antonbabenko reviewed Oct 31, 2022

View reviewed changes

antonbabenko added 2 commits November 1, 2022 19:44

Merge branch 'master' into master_sam_cli_integration_null_resource_s…

dcbb8f0

…olution

Removed _this from the resource names

8c1f7e3

antonbabenko approved these changes Nov 3, 2022

View reviewed changes

antonbabenko merged commit bfcd34c into terraform-aws-modules:master Nov 3, 2022

antonbabenko pushed a commit that referenced this pull request Nov 3, 2022

chore(release): version 4.6.0 [skip ci]

be47945

## [4.6.0](v4.5.0...v4.6.0) (2022-11-03) ### Features * Add SAM Metadata resources to enable the integration with SAM CLI tool ([#325](#325)) ([bfcd34c](bfcd34c))

lorengordon mentioned this pull request Nov 15, 2022

Native support for Terraform in SAM CLI (now in Beta) aws/aws-sam-cli#3154

Closed

github-actions bot locked as resolved and limited conversation to collaborators Dec 4, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Add SAM Metadata resources to enable the integration with SAM CLI tool #325

feat: Add SAM Metadata resources to enable the integration with SAM CLI tool #325

moelasmar commented Jul 8, 2022

moelasmar commented Jul 8, 2022

antonbabenko left a comment

antonbabenko Jul 8, 2022

moelasmar Jul 8, 2022

moelasmar commented Jul 8, 2022

bryantbiggs commented Jul 19, 2022 •

edited

Loading

moelasmar commented Jul 28, 2022 •

edited

Loading

antonbabenko left a comment

antonbabenko Aug 19, 2022

moelasmar Aug 23, 2022

antonbabenko Aug 19, 2022

moelasmar Aug 23, 2022

moelasmar commented Aug 19, 2022

antonbabenko commented Aug 24, 2022

antonbabenko left a comment

antonbabenko Oct 31, 2022

antonbabenko left a comment

antonbabenko commented Nov 3, 2022

antonbabenko commented Nov 3, 2022

moelasmar commented Nov 3, 2022

github-actions bot commented Dec 4, 2022

	SAM CLI provides two ways of testing, local testing, and testing on-cloud (Accelerate).
	SAM CLI provides two ways of testing: local testing and testing on-cloud (Accelerate).

feat: Add SAM Metadata resources to enable the integration with SAM CLI tool #325

feat: Add SAM Metadata resources to enable the integration with SAM CLI tool #325

Conversation

moelasmar commented Jul 8, 2022

Description

Motivation and Context

Breaking Changes

How Has This Been Tested?

moelasmar commented Jul 8, 2022

antonbabenko left a comment

Choose a reason for hiding this comment

antonbabenko Jul 8, 2022

Choose a reason for hiding this comment

moelasmar Jul 8, 2022

Choose a reason for hiding this comment

moelasmar commented Jul 8, 2022

bryantbiggs commented Jul 19, 2022 • edited Loading

moelasmar commented Jul 28, 2022 • edited Loading

antonbabenko left a comment

Choose a reason for hiding this comment

antonbabenko Aug 19, 2022

Choose a reason for hiding this comment

moelasmar Aug 23, 2022

Choose a reason for hiding this comment

antonbabenko Aug 19, 2022

Choose a reason for hiding this comment

moelasmar Aug 23, 2022

Choose a reason for hiding this comment

moelasmar commented Aug 19, 2022

antonbabenko commented Aug 24, 2022

antonbabenko left a comment

Choose a reason for hiding this comment

antonbabenko Oct 31, 2022

Choose a reason for hiding this comment

antonbabenko left a comment

Choose a reason for hiding this comment

antonbabenko commented Nov 3, 2022

antonbabenko commented Nov 3, 2022

moelasmar commented Nov 3, 2022

github-actions bot commented Dec 4, 2022

bryantbiggs commented Jul 19, 2022 •

edited

Loading

moelasmar commented Jul 28, 2022 •

edited

Loading