Support for remote_state backend 'remote'

[DenisBY]

Since HashiCorp has offered a free tier for Terraform Enteprise it would be nice to have a support for

remote_state {
  backend = "remote"
  ....
}

Or I've missed something and it's already there?

I have the following config:

remote_state {
  backend = "remote"
  config = {
    hostname = "app.terraform.io"
    organization = "my-company"

    workspaces = {
      name = "da-dev-${path_relative_to_include()}"
    }
  }
}

but it gives an error:

[terragrunt] [/home/denis/work/git/da/terraform/dev/ntv] 2019/07/09 17:44:11 Running command: /usr/local/bin/terraform-12 init -backend-config=hostname=app.terraform.io -backend-config=organization=my-company -backend-config=workspaces=map[name:da-dev-ntv]

Initializing the backend...

Error: Invalid backend configuration argument

The backend configuration argument "workspaces" given on the command line is
not expected for the selected backend type.

[terragrunt] 2019/07/09 17:44:11 Hit multiple errors:
exit status 1

[brikis98]

-backend-config=workspaces=map[name:da-dev-ntv]

Ah, Terragrunt is not converted nested values (e.g., nested maps, such as workspaces = { ... }) to proper HCL syntax when passing them as args on the command line. Would you be up for a quick PR to fix this? I'm pretty sure we have a function that does the conversion properly (for passing variables in inputs = { ... }) and just need to use it in one more place!

[tlvenn]

Facing the same issue, wish i could help but my go fu is not strong.

[LinguineCode]

👍 me three

[lsc]

I'm trying to write a patch for this, do you have any pointers @brikis98 as to in what file this is done?
I'm looking at the config.go file and the remote_state.go file, but it feels like I might be complicating things more than needed.

I'm a novice Go developer at best which isn't really to my advantage :)

[brikis98]

@lsc https://github.com/gruntwork-io/terragrunt/blob/master/remote/remote_state.go#L151

[lae]

Even if parsing is fixed, hashicorp/terraform#21830 still needs to be resolved for this to work with tf 0.12.

[joshuabalduff]

This would be awesome currently need this.

[aditsachde]

How does Terragrunt actually run the shell commands?

Bash can do process substitution in order to bypass the need to create a temporary file, this works correctly when using terraform init on the command line using the following command

terraform init -backend-config=hostname=app.terraform.io -backend-config=organization=org -backend-config=<(echo 'workspaces { name = "manual" }')

However, it seems like terraform is being run in such a way that doesn't allow this as the following does not seem to work. In what shell are terraform commands run in?

remote_state {
  backend = "remote" 

  config = {
    hostname = "app.terraform.io"

    organization = "org"

  }
}

terraform {
  extra_arguments "init_workspace" {
    commands = ["init"]

    arguments = [
      "-backend-config=<(echo \"workspaces { name = ${path_relative_to_include()} }\" )"
    ]

  }
}

Terragrunt says that it is running the command terraform init -backend-config=hostname=app.terraform.io -backend-config=organization=sachde -backend-config=<(echo "workspaces { name = manual }" ) but results in the warning

Error: Invalid backend configuration argument

The backend configuration argument "<(echo \"workspaces { name " given on the
command line is not expected for the selected backend type.

This is not the most elegant way to do this, but it seems like one of the few possible workarounds that could be used until this and hashicorp/terraform#21830 get fixed, other than adding a backend.hcl file that contains the desired workspace, and using "-backend-config=backend.hcl"

[aditsachde]

Actually, a better workaround is having a before_hook which runs a python script that writes the relative path to a file called backend.hcl

terraform {
    before_hook "init" {
    commands     = ["init"]
    execute      = ["${get_parent_terragrunt_dir()}/../scripts/terragrunt-env.py", "${path_relative_to_include()}"]
    run_on_error = false
  }

  
  extra_arguments "init_workspace" {
    commands = ["init"]

    arguments = [
      "-backend-config=backend.hcl"
    ]

  }
}

#!/usr/bin/python3

import sys

workspace = "workspaces { name = \""+' '.join(sys.argv[1:])+"\" }\n"
f = open("backend.hcl","w+")
f.write(workspace)
f.close

[yorinasub17]

We just released a new feature, generate blocks which allows you to generate tf code directly into the working directory. Check out the updated docs to learn more.

Since this is generating the raw TF file instead of using the CLI args, it won't run into all the parsing issues where you couldn't set blocks or complex maps on the CLI with --backend-config.

With this, you can generate a backend.tf file with the remote backend configuration:

generate "backend" {
  path = "backend.tf"
  if_exists = "overwrite_terragrunt"
  contents = <<EOF
terraform {
  backend "remote" {
    hostname = "app.terraform.io"
    organization = "my-company"
    workspaces {
      name = "da-dev-${path_relative_to_include()}"
    }
  }
}
EOF
}

[anagarjunr]

This issue still exists.
Yet to try the generate block solution. But hoping that this issue will be resolved and we can use remote backend without generate block

[GammaBlaze]

The generate block looks promising, however it doesn't like name = "da-dev-${path_relative_to_include()}". Getting the following error atm (very similar to 1389

$ terragrunt plan
[INFO] Getting version from tgenv-version-name
[INFO] TGENV_VERSION is 0.32.4

Initializing the backend...

Successfully configured the backend "remote"! Terraform will automatically
use this backend unless the backend configuration changes.
Error loading state: Error creating workspace my-worspace-prefix-here.: invalid attribute

Name must only contain letters, numbers, dashes, and underscores
ERRO[0004] 1 error occurred:
        * exit status 1

Edit: Figured it out, running terragrunt plan from the parent rather than child directory! Workspace generates with the right name now.

[rdettai]

Generate block works fine, but this incompatibility should be pointed out in the documentation.