Module source files deleted on Windows (possibly due to hard links)

[yonatanp]

I am developing locally (v0.6.16, Windows 10, cygwin shell) and had my whole module folder wiped clean.
What I did:

1. created folder /c/work/module and wrote a bunch of source files there (e.g. xxxxx.tf), and created a files subfolder containing a bunch of data files.
2. created folder /c/work/setup and wrote a mysetup.tf file which instantiates the module with some parameters, referencing the module using source = ../module
3. ran terraform get in the setup folder
4. worked a little bit with my setup until I decided it was not needed anymore
5. went to /c/work and did rm -rf setup
6. all contents of the module file have been wiped clean, including sources and data files subfolder

Repeating this exercise shows complete reproducibility. Each time the module sources are wiped clean. Naturally I would expect them to remain unharmed.

Looking through Windows eyes, I see that the "link" to the module source is created as a Junction:

C:\>dir c:\work\setup\.terraform\modules
Directory of C:\work\setup\.terraform\modules
07/19/2016 02:38 PM <DIR> .
07/19/2016 02:38 PM <DIR> ..
07/19/2016 02:38 PM <JUNCTION> dad20d2bf082af4bcc75117f33d47285 [c:\work\module]
0 File(s) 0 bytes
3 Dir(s) 26,666,666,468 bytes free

This may be related. A <JUNCTION> in Windows is "harder" than a <SYMLINK>, and it could be that rm -rf deletes its contents recursively.

I suspect that #7484 might be related, but cannot be sure.

[yonatanp]

The actual issue is in the "go-getter" repo.
This seems to have been introduced by hashicorp/go-getter#16 as a measure to overcome the Administrative rights required on some machines for creating symlinks. In the PR, the call to os.Symlink(u.Path, dst) was replaced by exec.Command("cmd", "/c", "mklink", "/J", dst, sourcePath).

It could be that a middle way would be calling mklink /d instead.
@jen20 - if you're around - have you tried this alternative?

I'll attempt to experiment a bit and will report whether it shows the same resilience to permissions. My go is slightly rusty but I hope it will be useful.

[csabatini]

Did you find any workarounds for this? module source files are getting deleted on windows for me when I remove the .terraform directory

[csabatini]

I ended up removing the junction first before deleting .terraform, not clean but seems to avoid the module sources being removed

fsutil reparsepoint delete .terraform/modules/3f3713839625f8b7f8c2a7ef06c40b82
rm -rf .terraform

[davyt10]

+1 on this , a fix would be great. This is critical really as how can we ever launch many environments from the same bunch of modules and never have the ability to delete the old environments directories that are no longer in use. To delete would and does(I have tested) blast away all the module source code. Every other active environment running from these modules is then completely broken.

[apparentlymart]

Hi all! Sorry for the long silence here.

In the master branch now we have a change which will stop using symlinks for this purpose altogether, and will instead just refer directly to the relative directory path given in the module source. This should avoid the problem described, since we'll no longer be creating symlinks (or, on Windows, junction points) in the modules directory at all.

Since this change is already merged, I'm going to close this out. The fix will be included in the forthcoming v0.12.0 release.

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.