Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

No way to use data "template_file" on a file generated on the fly by local-exec as it fails during planning #8406

Closed
bloo opened this issue Aug 23, 2016 · 4 comments

Comments

@bloo
Copy link

bloo commented Aug 23, 2016

Terraform Version

Terraform v0.7.1

Affected Resource(s)

  • null_resource
  • data "template_file"

Terraform Configuration Files

variable etcd_cluster_size {
  default = 3
}

resource "null_resource" "discovery_url_get" {
  provisioner "local-exec" {
    command = "curl -s 'https://discovery.etcd.io/new?size=${var.etcd_cluster_size}' > /tmp/discovery.url"
  }
}

data "template_file" "discovery_url" {
  template = "${file("/tmp/discovery.url")}"
  depends_on = ["null_resource.discovery_url_get"]
}

Debug Output

Errors:

  * file: open /tmp/discovery.url: no such file or directory in:

${file("/tmp/discovery.url")}

Expected Behavior

I (perhaps incorrectly) assumed that terraform plan wouldn't require that a file referenced by file() needed to exist during the plan stage, but would fail on it being missing during terraform apply.

I suppose this could be considered intended behavior, and that a better solution would be to have something like tmp_file() as a file() alternative, or a core way to capture a local-exec output into a variable instead of a temp file?

Actual Behavior

terraform plan fails on not finding a file that had yet to be generated by a dependent null_resource.

Steps to Reproduce

  1. terraform plan
@bloo bloo changed the title No way to use data "template_file" on a file generated on the fly local-exec as it fails during plan No way to use data "template_file" on a file generated on the fly by local-exec as it fails during planning Aug 23, 2016
@mitchellh
Copy link
Contributor

This is kind of working as intended but there are definite buggy behavior in here.

Data sources cannot depend on resources (we should've given you an error, we didn't, that's a bug) since data sources must be able to run prior to any resources. This is the point of data source: to give you data that is external to Terraform that isn't dependent on the ordering of Terraform's execution.

At least, that's how they work in their current form.

I think the primary issue here is that we didn't give you a config validation error. There are other issues like #8144 (referenced) that aim to solve your use case.

@apparentlymart
Copy link
Contributor

@mitchellh what you described there isn't entirely correct: a data source can depend on a resource, but that causes its resolution to be deferred until the apply phase. In this case they appear in the plan output as "read" actions, with the <= icon.

I think the issue here is more that interpolation functions are dealt with during the validation pass as well as other passes, but during validation the file will of course not exist yet.

Using the filename argument to template_file was the traditional workaround for this problem, though of course we've deprecated it. 😀

I think a more robust solution to this class of problem (I need to get some data or do some operation that Terraform doesn't support but that I can script) is something like what I proposed in #8144. Under that proposal, it becomes possible to execute external programs as data sources, rather than only as provisioners, and thus they can interact more reasonably with other data sources and resources.

@bloo
Copy link
Author

bloo commented Aug 31, 2016

@mitchellh thanks - I also noticed that file names in quotes are treated like files that must exist, whereas variable names that represent filenames go unquoted and are not validated when running terraform plan. I think the $file() docs could reflect that difference if indeed that's expected behavior. I stumbled upon it though and got it working.

@ghost
Copy link

ghost commented Apr 22, 2020

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@ghost ghost locked and limited conversation to collaborators Apr 22, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

3 participants