List files uploaded to remote run

[Sytten]

Current Terraform Version

Terraform v0.14.5

Use-cases

When using the remote run feature, it is possible to define a custom .terraformignore file using glob syntax similar to a .gitignore. But there seems to be no way to list the files either uploaded or ignored to make sure everything is ok. This can lead to cases where sensitive files are uploaded by accident or the upload time is very long.

Attempted Solutions

You can create a copy of you repo, init a new git, use the .terraformignore as a .gitignore and run git ls-files --ignore --exclude-from=./gitignore. But it is not practical nor the real pattern matching that happens by terraform.

Proposal

I would propose a new command similar to git ls-files for terraform to list uploaded or ignored files.

References

[pselle]

It might help you out in the meantime to use the TF_IGNORE environment variable (warning: it's case sensitive, there's an open bug to make it more forgiving) https://www.terraform.io/docs/cli/config/environment-variables.html#tf_ignore

If TF_IGNORE is set to "trace", Terraform will output debug messages to display ignored files and folders. This is useful when debugging large repositories with .terraformignore files.

[Sytten]

Ho thanks for that, I never got that from my google search. Maybe it would be nice to add that information in the section that talks about the .terraformignore.

[j0hnsmith]

I ran a plan with TF_LOG=TRACE and expected to see which files were being uploaded, why a separate var?

[StummeJ]

TF_IGNORE=trace is great, but it shows what files aren't uploaded. It would be a lot more useful to show what files ARE uploaded.

[nwalters512]

It looks like support for TF_IGNORE was removed in hashicorp/go-slug#42. Are there any alternatives we can use?

As for why I want this feature: I'm running terraform plan with Terraform Cloud. I'm running that command in a directory with a main.tf file, but the remote run claims that no Terraform files were found. I'd like to be able to see which files are in fact being uploaded.

[apparentlymart]

Since the files to upload are decided by the upstream library go-slug, which is in turn wrapped by another library go-tfe, Terraform CLI itself is not actually directly involved in deciding what goes into these "slug" archive files that get uploaded.

However, since that behavior lives in a library I was able to wrap it in a simple CLI tool here: https://github.com/apparentlymart/go-slug-cli

If you build that using a Go toolchain then you can run a command line like go-slug-cli pack to run the same packing logic that Terraform would indirectly be using, and then print out all of the filenames that would have been included.

I made this quickly in some idle time while I was waiting for a build to complete, in the hope that it would be helpful to the participants in this issue, but it's over in a repository I own to indicate that it's not a real HashiCorp project and so there's no official support for it. I'm going to leave this issue open to represent that there is not yet any official way to list which files would be included if a directory were uploaded to Terraform Cloud by Terraform CLI.

I hope this is helpful in the meantime, at least.

[StummeJ]

THANK YOU!

[mltsy]

Even if terraform CLI isn't responsible for generating the slug archive, surely there must be a simple way to expose the filenames contained in that slug archive somewhere (either via the CLI or in the TF Cloud UI) before or after it's uploaded? It's just a tar file right? (Even just stashing the file in the .terraform directory for inspection would be great if the location were documented...)