Absolute paths in Statefile

[so0k]

Output from template and local providers store absolute path in statefile.

This causes issues when using remote state and working in a team

Perhaps this is a known issue, please point me to related tickets or suggested workarounds as I was unable to find any so far.

Terraform Version

terraform -v
Terraform v0.10.2

Affected Resource(s)

Please list the resources as a list, for example:

• template_dir

Terraform Configuration Files

# in the bootstrap module:
resource "template_dir" "configs" {
  source_dir      = "${path.module}/resources/configs"
  destination_dir = "./generated/bootstrap/configs"

  vars {
    # template variables here
  }
}

# in the root project - modules.tf
module "my_module" {
  source         = "./modules/bootstrap"
  # module config here
}

Debug Output

module.my_module.template_dir.configs: 1 error(s) occurred:

* module.my_module.template_dir.configs: template_dir.configs: could not generate output checksum: lstat /Users/path/to/root/project/.terraform/modules/0a870bfa99885a552018aa68a1aac335/resources/configs: no such file or directory

Expected Behavior

Relative paths should be stored in statefile instead of Absolute paths

Actual Behavior

Absolute paths are stored in statefile and cause lstat errors on terraform refresh

Steps to Reproduce

Please list the steps required to reproduce the issue, for example:

1. keep terraform plan on different paths between laptop 1 and laptop 2
2. use remote state and terraform init both laptops
3. on laptop 1 run terraform apply
4. on laptop 2 run terraform refresh

[so0k]

is this related? External provider stores full path to command in state file causing thrash when executing on different machines - that issue was closed as a data resource state is replaced on every refresh, but this issue relates to resource for which refresh does a diff and errors out

[apparentlymart]

Hi @so0k! Sorry for this limitation.

This is indeed a general problem, caused by the fact that providers currently don't have access to a base directory to relative-ize stored paths against. The current working directory isn't suitable, because Terraform supports running against configurations elsewhere.

We do have plans to fix it, and that issue you referred to is one record of that, though I think there isn't currently a single issue capturing the broader problem. (It's on the team's radar, nonetheless.)

With that said, I think there is a more localized bug here that we may be able to fix more easily to mitigate this issue in the short term: the expected behavior of a resource during refresh is for it to detect when the resource has been deleted outside (or in this case, has never existed on this machine in the first place) and flag that in the state so the resource can be re-created on the next run.

Interestingly there is already code trying to handle this:

	// If the output doesn't exist, mark the resource for creation.
	if _, err := os.Stat(destinationDir); os.IsNotExist(err) {
		d.SetId("")
		return nil
	}

It seems that for some reason this initial os.Stat call is returning an error that isn't classified as a "does not exist" error. This is curious since the later error is pretty clearly a "no such file or directory" error resulting from a stat operation. It seems like there's a strange interaction to figure out here, to see why the above isn't working. If we can get to the bottom of that, you should be able to get past this error and have Terraform plan to create the directory from scratch as expected when running on a new machine.

---

This particular issue aside, it's worth noting that in general local disk resources (template_dir and local_file in particular) have some additional caveats compared to most providers unless the target is a shared filesystem, since although the state is remote, and the config is presumably in version control, these generated files would not exist on local disk for other users even if the paths were relative. For many use-cases this is okay, since just re-generating files for each new system is straightforward, but it does nonetheless lead to some state churn as the resource in question will get marked as deleted each time it's refreshed on a machine that doesn't have the files present.

Terraform's architecture is primarily design around working with remote resources that shared for all users, with these local-disk resources provided for reasons of pragmatism. We may be able to improve the situation in the future -- for example, by using relative paths as we're discussing in this issue -- but it may be the case that resources that exist only on local disk are fundamentally incompatible with a shared remote state strategy and so some of these caveats may be very difficult to totally resolve within Terraform's current assumptions.

We've heard reports of people using these resources in environments where Terraform is run in a controlled, remote environment... either on a shared remote system (which many people log into to run Terraform, or which runs Terraform via some sort of job automation system) or on systems that have a shared network filesystem mounted at a common path. This strategy is of course not workable for everyone, but within the current constraints it's the most robust way to make use of these resources.

[rowleyaj]

Hi folks,

I don't think that this is an issue with the output directory not being found, it seems to me to be a problem with the input directory not being found due to the modules being in a different location. The error is just ambiguous due to the message in generateDirHash, I think it's the source directory due to the lstat being within the .terraform/modules directory.

https://github.com/terraform-providers/terraform-provider-template/blob/5ac67904f65462fbf32c25831afb18a9b85b2793/template/resource_template_dir.go#L67

https://github.com/terraform-providers/terraform-provider-template/blob/5ac67904f65462fbf32c25831afb18a9b85b2793/template/resource_template_dir.go#L166-L177

https://github.com/terraform-providers/terraform-provider-template/blob/5ac67904f65462fbf32c25831afb18a9b85b2793/template/resource_template_dir.go#L179-L187

I've hit the same problem but through a different method, mine was due to using an updated version of the source module where the the modules within had been moved around. This meant that the hashed folder names in .terraform/modules were different due to them being in different places in the graph. So in my case terraform should mark them for deletion from the state (although I'd likely try and use state mv on them to put them in the right place instead).

I'm not sure the best way to resolve this, but happy to try and raise a PR with some guidance. Should we check the existence of the source directory before attempting to hash, and mark for deletion from state if it doesn't exist? Is blanking the id enough for that, or does it need to be done differently? It's essentially an orphaned module in some ways, how is this handled elsewhere in other providers? Perhaps even tarDir should exit early and return an empty buffer if the directory doesn't exist?

[adnoh]

No update on this one for so a long time? It is really a show stopper working with Terraform in a team. If anyone would tell me a workaround I would highly appreciate that. At the moment I'm removing the relevant parts in the state and generate new ones

[rowleyaj]

@apparentlymart any thoughts on my assessment of the above?

[stuartsan]

A potential workaround: instead of

source_dir      = "${path.module}/resources/configs"

You may be able to do something like this, dropping the absolute path:

source_dir      = "my-great-module/resources/configs"

This is not super portable because the path is now relative to where terraform is executed, but it worked out ok for my use case.

[egorchabala]

Any updates on this issue? It currently blocks us from executing terraform from multiple sources

[shopskyi]

The "local_file" resource also stores absolute path in a state file but works fine. Any updates with "template_dir" resource?

[the-nicolas]

... I wasted 10+ hours on that.

The problem was so nested and I searched everywhere for the wrong paths, nut no match. Sure... remote state... just there during execution.