Bug: Is there a way to supply overrides for non-specified environment variables when using with terraform

[toddtarsi]

Description:

We use the module pattern in terraform, which leads to most resources having a sort of wrapper. As a result, our environment variables are a bit abstracted like this:

resource "aws_lambda_function" "this" {
  ...
  environment {
    variables = var.environment.variables
  }
  ...
}

The command sam build expectedly won't pick up these variables, so they don't get into the generated template.json file, and I think thats fine. The next problem is that sam local invoke xyz won't allow me to inject these dynamic environment variables into the runtime. At least, --env-vars doesn't seem to be able to actually inject environment variables. Is there something I'm missing?

Steps to reproduce:

1. Have a lambda within a module with environment variables supplied as a map(string) type variable to the module.
2. sam build; sam local invoke xyz --env-vars env.json
3. The environment variables needed won't work here.

Observed result:

None of the env vars under Parameters are injected.

Expected result:

I would expect the env-vars under Parameters to be injected as environment variables.

Additional environment details (Ex: Windows, Mac, Amazon Linux etc)

{
  "version": "1.95.0",
  "system": {
    "python": "3.8.13",
    "os": "macOS-13.4.1-arm64-arm-64bit"
  },
  "additional_dependencies": {
    "docker_engine": "24.0.2",
    "aws_cdk": "Not available",
    "terraform": "1.4.6"
  },
  "available_beta_feature_env_vars": [
    "SAM_CLI_BETA_FEATURES",
    "SAM_CLI_BETA_BUILD_PERFORMANCE",
    "SAM_CLI_BETA_TERRAFORM_SUPPORT",
    "SAM_CLI_BETA_RUST_CARGO_LAMBDA"
  ]
}

[moelasmar]

Hello @toddtarsi,
Thanks for raising this issue.
This issue is similar to these issues #4439, #4440
Currently, we do not support passing Terraform flags or parameters through SAM CLI. You still can pass these variables through using terraform environment variables TF_VAR_<name> https://developer.hashicorp.com/terraform/cli/config/environment-variables#tf_var_name

[moelasmar]

one question, How do you originally get the value of these variables before you passed them to the module?
Are they passed to the TF Project as variables, if yes, so my previous answer should work for you.
If not, and these variables are based on other resources that will be created by the TF project, so you may have another issue.

[toddtarsi]

@moelasmar - Thank you for the quick response! (sorry for delay, had to pick up kiddo from preschool) So, I actually build the zip files in another codebase, and then I am using sam build without metadata to generate the template config (I just override filename to point at my other repos dist files, all of that works fine). My template looks like a lot of this:

    "ModuleXYZLambdaAwsLambdaFunctionThisB5F50C77": {
      "Type": "AWS::Lambda::Function",
      "Properties": {
        "FunctionName": "local-xyz",
        "Code": "/Users/me/other-repo/lambda.zip",
        "Handler": "index.handler",
        "PackageType": "Zip",
        "Runtime": "nodejs16.x",
        "Layers": [],
        "Timeout": 600,
        "MemorySize": 512
      },
      "Metadata": {
        "SamResourceId": "module.xyz.aws_lambda_function.this",
        "SkipBuild": true
      }
    },

However, these zip files rely on some specific environment variables during their runtime / execution (PG_HOSTNAME, etc), but because there is some abstraction at the module boundary, these get missed when scraping ENVIRONMENT.
So, I'm trying to do this via my samconfig.toml:

version=0.1

[default]
[default.local.invoke.parameters]
debug = true
hook_name = "terraform"
beta_features = true
env_vars = "./samconfig.env.json"
container_env_vars = "./samconfig.container.env.json"

Example samconfig.env.json:

{
  "Parameters": {
    "PG_HOSTNAME": "localhost",
  }
}

Example samconfig.container.env.json:

{
  "PG_HOSTNAME": "localhost",
}

Log output when I console.log the env on the running process:

Mounting /private/var/folders/lr/nmc29g010md2f9hhkc2qwf4m0000gn/T/tmpueyx9aka as /var/task:ro,delegated, inside runtime container                                 
START RequestId: f2d39f8e-3536-427e-b946-5493d695c944 Version: $LATEST
2023-08-22T18:46:43.276Z	f2d39f8e-3536-427e-b946-5493d695c944	INFO	PG_HOSTNAME undefined
{"level":"ERROR","message":"Database call failed. PrismaClientInitializationError: The provided database string is invalid. Error parsing connection string: empty host in database URL. Please refer to the documentation in https://www.prisma.io/docs/reference/database-reference/connection-urls for constructing a correct connection string. In some cases, certain characters must be escaped. Please check the string for any illegal characters."}
END RequestId: f2d39f8e-3536-427e-b946-5493d695c944
REPORT RequestId: f2d39f8e-3536-427e-b946-5493d695c944	Init Duration: 0.93 ms	Duration: 7699.07 ms	Billed Duration: 7700 ms	Memory Size: 512 MB	Max Memory Used: 512 MB	
null%                         

[toddtarsi]

@moelasmar - Sorry for second message, should have included this in the first. Basically, I'm trying to supply unspecified environment variables to the running lambda local invocation but I just can't figure out how to get them in there. In a lot of ways, this might not even involve terraform at this stage and I'm probably conflating like 10 different things. Thank you for the help, and sorry for the confusion.

[toddtarsi]

After looking around, I believe I'm hitting this issue basically:

#3795