Terraform cloud provider - Variables not sent

[maximerenou50]

Hello,

My company is currently using Terragrunt with Terraform 0.12, using AWS S3 and DynamoDB as a backend. It works like a charm :)
I was looking into Terraform cloud as a new backend, it offers some interesting features.

Currently, I'm having issues when trying to use that backend, variables are not being sent to Terraform cloud, so I'm having that kind of errors:

Error: Unassigned variable

The input variable "aws_iam_policy_name" has not been assigned a value. This
is a bug in Terraform; please report it in a GitHub issue.

[...]

I do see the remote plan in Terraform cloud, everything else seems to be working as expected, only the variables are not sent. Here are my files:

terragrunt.hcl

terraform {
  source = "github.com/scalair/terraform-aws-iam-s3?ref=v1.0.0"
}
inputs = {
  iam_user_pgp_key = "keybase:test"
  iam_user_name = "my-user"
  aws_iam_policy_name = "my-policy" 
  bucket_name = "my-bucket"
}

main_providers.tf

terraform {
  backend "remote" {
    organization = "my-organization"

    workspaces {
      name = "my-awesome-workspace"
    }
  }
}

I'm using the exact same module with the exact same variables with an S3/DynamoDB backend and it's working fine (variables are being sent to the module).
Maybe I'm missing something obvious, or Terragrunt doesn't support Terraform cloud at the moment.

FYI, without Terragrunt, the following files are working:

main.tf

module "terraform-aws-iam-s3" {
  source = "github.com/scalair/terraform-aws-iam-s3?ref=v1.0.0"

  iam_user_pgp_key = "keybase:test"
  iam_user_name = "my-user"
  aws_iam_policy_name = "my-policy" 
  bucket_name = "my-bucket"
}

provider "aws" {
  region = "eu-west-1"
}

backend.tf

terraform {
  backend "remote" {
    organization = "my-organization"

    workspaces {
      name = "my-awesome-workspace"
    }
  }
}

Thanks!

[brikis98]

Terragrunt passes the variables in inputs = { ... } as environment variables when calling Terraform. Do remote plans/applies not support variables passed in via environment variables?

Try an experiment where you use terraform apply directly, but only set the variables via TF_VAR_foo=bar.

[maximerenou50]

Hi @brikis98 ,
I tried passing variables using the TF_VAR_ syntax and it doesn't work indeed. I'm new to Terraform Cloud so not 100% sure of the expected behaviour.
Inside a workspace in Terraform Cloud, you can define variables, but they are not passed to the module, so I'm still trying to figure it out how it's supposed to work.

I need to read more about Terraform cloud and see what's possible or not. I will update here with what I found.

[brikis98]

Thx for investigating @maximerenou50. I'm surprised env vars don't work, so perhaps file a bug with the HashiCorp team and see what they say?

[jwenz723]

Environment variables won't work if you have the terraform cloud workspace configured to use 'remote' execution mode, meaning that the terraform cli will actually be invoked on a VM hosted by terraform cloud. You can read about it here and here. To pass vars when using 'remote' execution mode you have to place them into a *.auto.tfvars file.

The best solution I came up with to deal with this is to generate an *.auto.tfvars file with the necessary inputs. So these inputs:

inputs {
  cluster_name = "${local.env}"
  subnets = dependency.vpc.outputs.private_subnets
  vpc_id = dependency.vpc.outputs.vpc_id
}

Would be replaced with this:

generate "autovars" {
  path = "terraform.auto.tfvars"
  if_exists = "overwrite_terragrunt"
  contents = <<EOF
cluster_name = "${local.env}"
subnets = dependency.vpc.outputs.private_subnets
vpc_id = dependency.vpc.outputs.vpc_id
EOF
}

I'm not sure how to accomplish placing outputs from a dependent module into the generated file though. It seems that dependency.vpc.outputs.private_subnets and dependency.vpc.outputs.vpc_id are unable to be resolved in my generate block above.

[bwhaley]

According to the docs on configuration parsing order, I think the dependency outputs should be available. Have you tried wrapping them in ${}?

generate "autovars" {
  path = "terraform.auto.tfvars"
  if_exists = "overwrite_terragrunt"
  contents = <<EOF
cluster_name = "${local.env}"
subnets = ${dependency.vpc.outputs.private_subnets}
vpc_id = ${dependency.vpc.outputs.vpc_id}
EOF
}

[jwenz723]

According to the docs on configuration parsing order, I think the dependency outputs should be available. Have you tried wrapping them in ${}?

generate "autovars" {
  path = "terraform.auto.tfvars"
  if_exists = "overwrite_terragrunt"
  contents = <<EOF
cluster_name = "${local.env}"
subnets = ${dependency.vpc.outputs.private_subnets}
vpc_id = ${dependency.vpc.outputs.vpc_id}
EOF
}

I swear I had tried that and it didn't work. However, I just tried again and it did work with one modification to your suggestion. I had to place quotes around the values since the variables are string variables, like this:

generate "autovars" {
  path = "terraform.auto.tfvars"
  if_exists = "overwrite_terragrunt"
  contents = <<EOF
cluster_name = "${local.env}"
subnets = "${dependency.vpc.outputs.private_subnets}"
vpc_id = "${dependency.vpc.outputs.vpc_id}"
EOF
}

[michaelssingh]

I don't believe the work around above is an object/map. Has anyone gotten around this?
I run into the following error:

Invalid template interpolation value; Cannot include the given value in a string template: string required.

generate "tfvars" {
  path      = "terragrunt.auto.tfvars"
  if_exists = "overwrite"
  disable_signature = true
  contents = <<-EOF
account_settings = ${dependency.account_info.outputs.config} // this is an object
EOF
}

I have tried wrapping it with " " and using the tostring() method, no dice.

[jwenz723]

I had my scenario working at one point with a map variable. I believe I had to do it using a go templating for loop to iterate through each key/value and print each key/value into the generate block 1 at a time.

Terragrunt ultimately didn't fit my use-case, so I unfortunately no longer have an example of exactly how this is done. Hopefully you can post back here if you figure it out @michaelssingh

[brikis98]

I don't believe the work around above is an object/map. Has anyone gotten around this?
I run into the following error:

Invalid template interpolation value; Cannot include the given value in a string template: string required.

generate "tfvars" {
  path      = "terragrunt.auto.tfvars"
  if_exists = "overwrite"
  disable_signature = true
  contents = <<-EOF
account_settings = ${dependency.account_info.outputs.config} // this is an object
EOF
}

I have tried wrapping it with " " and using the tostring() method, no dice.

Are you saying account_settings should be set to a string? Or an object? And what type is dependency.account_info.outputs.config returning?

[krisdevopsbot]

Similar issue "Invalid template interpolation value; Cannot include the given value in a string template: string required."

local.region_vars.locals.azs is a list(string)

${local.region_vars.locals.azs} = invalid template interpolation when used in a generate block heredoc

Using join/split works but is not ideal

[rsmets]

@krisdevopsbot mind showing an example of how you would it work using join/split?

or

@michaelssingh did you ever sort out how to get around Invalid template interpolation value; Cannot include the given value in a string template: string required.?

[yorinasub17]

I believe all the interpolation issues around complex types can be worked around by using json for the tfvars instead of HCL. The following should work:

generate "tfvars" {
  path      = "terragrunt.auto.tfvars.json"
  if_exists = "overwrite"
  disable_signature = true
  contents = jsonencode({
    account_settings = ${dependency.account_info.outputs.config} // this is an object
  })
}

[nkaravias]

The above (generating auto.tfvars) solves the problem of missing variable inputs with terraform cloud, however terragrunt keeps initializing on every terragrunt plan / apply.

The prompt for apply/destroy is also missing and changes are going through as if --terragrunt-non-interactive is specified.

Is anyone else experiencing the same? (v0.35.16)