With symlinked home directory, init fails: providers could not be installed: lstat var: no such file or directory #25367
Comments
|
Actually I could progress a step further by looking at the TF_LOG=TRACE output. The correct path seems to be something else and the guide is wrong: -terraform.example.com/awesomecorp/happycloud/v1.0.0/linux_amd64/terraform-provider-happycloud_v1.0.0
+terraform.example.com/awesomecorp/happycloud/1.0.0/linux_amd64/terraform-provider-happycloud_v1.0.0I put my provider at |
|
stracing terraform I found But I have absolutely no idea why it is looking up for this non existent file |
|
The source of the error seems to be: terraform/internal/getproviders/hash.go Line 94 in 49e2e00 as if called from: called from: terraform/internal/providercache/installer.go Lines 398 to 400 in f86fb61 as if |
|
terraform/internal/providercache/installer.go Line 389 in f86fb61 cached is generated here: terraform/internal/providercache/dir.go Lines 92 to 109 in 0d62001 which is generated using the fillMeta function: terraform/internal/providercache/dir.go Lines 127 to 202 in 0d62001 corresponding to the following TF_LOG=TRACE logs: |
|
terraform/internal/providercache/dir.go Lines 176 to 182 in 0d62001 is where the meta cache is filled, corresponding to the following trace:
packageDir := filepath.Clean(string(meta.Location.(getproviders.PackageLocalDir)))none of which are looking like they would return |
|
Something to add, it's running on a Fedora Silverblue machine as root user (for local provisioning), and |
|
strace logs: Seems like there is something here, and setting |
|
As if With package main
import (
"path/filepath"
"fmt"
)
func main(){
fmt.Println(filepath.ToSlash(filepath.Clean("/root/.terraform.d/plugins/localhost/local/sys/1.0.0/linux_amd64")))
}Running on the same system, I do not get anything strange: |
|
With the following test program: package main
import (
"path/filepath"
"fmt"
)
func main(){
fmt.Println(filepath.ToSlash(filepath.Clean("/root/.terraform.d/plugins/localhost/local/sys/1.0.0/linux_amd64")))
fmt.Println(filepath.EvalSymlinks(filepath.ToSlash(filepath.Clean("/root/.terraform.d/plugins/localhost/local/sys/1.0.0/linux_amd64"))))
}I see this in strace: Which indicates that it works in this case. I also see a similar pattern as terraform strace (newfstatat + readlinkat + newfstatat) The symlink to |
|
using latest filepath library from go 294edb272d5d145665bdf8b4254609eae0363a8d, I get no error |
|
Might be related: golang/go#30520 |
|
terraform v0.13.0-beta2 built with my toolchain does have the error, not a problem of the toolchain. |
|
No, this is indeed a problem with the runtime. I added traces to terraform until I found that EvalSymlinks is called with and I could reproduce it with a trivial example: package main
import (
"path/filepath"
"fmt"
)
func main(){
fmt.Println("filepath.EvalSymlinks(.terraform/plugins/localhost/local/sys/1.0.0/linux_amd64)")
fmt.Println(filepath.EvalSymlinks(".terraform/plugins/localhost/local/sys/1.0.0/linux_amd64"))
} |
|
Can be reproduced out of context with: package main
import (
"path/filepath"
"fmt"
"syscall"
"os"
)
func main(){
os.Mkdir("origin", 0777)
os.Mkdir("origin/var", 0777)
os.Mkdir("origin/var/home", 0777)
os.Mkdir("origin/var/home/foo", 0777)
os.Symlink("var/home", "origin/home")
os.Symlink("/home/foo", "origin/home/bar")
os.Chdir("origin")
syscall.Chroot(".")
os.Chdir("home")
fmt.Println(filepath.EvalSymlinks("bar"))
} |
|
Golang issue can be worked around by updating the symlink: |
alisdair
changed the title
|
Thank you @mildred for this excellent investigation! As you noted yourself, the initial problem with installing the local provider was a documentation bug (thanks for fixing that). I've retitled this issue and will leave it open to track the upstream symlink evaluation issue, in the hopes that your diagnosis and workaround will help anyone else who encounters it. |
|
I was hinted by the go team that See: golang/go#40180 So, it might be a good idea to try to use something else. Here is the documentation text that will go with the function: // EvalSymlinks returns the path name after the evaluation of any symbolic
// links.
// If path is relative the result will be relative to the current directory,
// unless one of the components is an absolute symbolic link.
// EvalSymlinks calls Clean on the result.
// Use of this function is unsuitable for the vast majority of applications.
// This function always resolves all links on path, but links may have
// been created to solve administrative problems of which
// most applications should remain unaware.
// Most applications should only resolve specific links that they
// require to resolve, use the result immediately, forget the result
// and never show the result to the user.
// On Windows, the result is not stable and must not be cached
// or stored in any way, resolving links may introduce complexities that
// the application must be prepared to deal with and the result
// does not identify the volume that a file or directory resides on. |
After following guide at https://github.com/hashicorp/terraform/blob/guide-v0.13-beta/draft-upgrade-guide.md#in-house-providers I failed to make terraform recognize in-house providers
I created a terraform-provider-sys which is a fork of terraform-provider-null with added features for local provisioning, but I failed to make it work with terraform 0.13-beta2. With terraform-0.12 I installed the provider manually under
~/.terraform.d/plugins/terraform-provider-sysand I could then use it.With terraform 0.13 I modified my module accordingly:
I chose the
terraform.localhostdomain because I obviously control it and the doc explicitely says that I do not need to run a registry at that address:I put my provider at:
~/.terraform.d/plugins/terraform.localhost/local/sys/terraform-provider-sysbut I get the following error:I also tried with
~/.terraform.d/plugins/terraform.localhost/local/sys/v1.0.0/linux_amd64/terraform-provider-sys_v1.0.0with no better results.This should be made to work, or if it works already, the documentation should be clearer with examples. There is the command
terraform providers mirrorwhich provides an example layout but it contains various JSON files, are we expected to create those JSON files?The text was updated successfully, but these errors were encountered: