Update go-getter vendored library

[prologic]

What I did:

$ govendor get ..

And a lot of mucking around with broken dependencies until make dev produces a correct binary

This fixes Nomad's lack of "symlinks" support in Tarball(s) (Artefacts).

[notnoop]

Thanks for the PR. We are stabilizing code for 0.9.0 release, and concentrating on urgent fixes. I'll target merging this PR after we cut 0.9.0.

Also, when symlink issue came up before, some colleagues had security concerns about symlinks - we need to ensure that symlinks in artifacts don't provide an escape hatch to host when running with exec/docker drivers.

[prologic]

Unlesss I am missing something about the way chroot works (but really we're using runC here right in both cases)? This isn't possible unless you're using hard links -- which my patch didn't attempt to solve (we don't need it). See this q&a for example

We would prefer this get in for the 0.9 release as we're still building a patched version of Nomad in our infra which isn't great. Any chance we can put some priority on this?

[schmichael]

@prologic Unless I am missing something about the way chroot works (but really we're using runC here right in both cases)?

Unfortunately artifacts are downloaded and templates are rendered outside the chroot. We test to ensure their path's don't escape the chroot/alloc directory, but allowing artifacts to write symlinks complicates this. An artifact could create a symlink outside the alloc dir, and then another artifact or a template could overwrite arbitrary files on the filesystem by following that symlink.

0.9.0 is way overdue and feature frozen, so I'm afraid this will have to wait until post-release.

If you're able to add tests and any necessary code to prevent alloc dir escaping by symlinks from artifacts, we'd be happy to merge it into the 0.9.1 branch where we're already staging features. Otherwise this is definitely something we need to support, but I'm afraid I can't commit to a roadmap. It's possible our existing checks prevent symlink-from-artifact abuse, but it will take some testing and thinking-like-an-attacker to ensure that's the case.

[prologic]

If you're able to add tests and any necessary code to prevent alloc dir escaping by symlinks from artifacts, we'd be happy to merge it into the 0.9.1 branch where we're already staging features. Otherwise this is definitely something we need to support, but I'm afraid I can't commit to a roadmap. It's possible our existing checks prevent symlink-from-artifact abuse, but it will take some testing and thinking-like-an-attacker to ensure that's the case.

I might have some bandwidth next week. Do you have any requirements for these tests?
Should they go into Nomad's test suite for go-getter's (which I understand is a more general purpose library/tool)?

[schmichael]

I think we'll need a test on both sides (go-getter and nomad) as it's subtle enough logic it seems easy for one side to break without the other noticing.

go-getter

I don't think the symlink support in go-getter won't support Nomad's use case as-is, but I haven't tested. It needs to support a working directory like tar's -C option, and implement tar's default behavior of stripping leading paths (effectively treating absolute paths like paths relative to the working directory) and disallowing files with .. in the name completely. For symlinks it would need to enforce that behavior for both the target and link name.

This should make go-getter's tar behave like gnu tar which I believe has the chroot-esque security properties we need (my kingdom to just unpack artifacts while chroot'd, but I think that's even more complicated).

nomad

Then this test might be a good one to copy, paste, and change. I would create a tarball with a few symlinks: one valid (relative path), one invalid with absolute path (/etc/nomadtest-symlink or something appropriately scary yet not actually harmful in case the test fails on a dev's machine as root 😅 ), and one invalid with dots (../../../etc/nomadtest-symlink2 or similar).

Call ExecCompatible(t) at the beginning to require root and Linux for the test.

The symlink with dots in the path should be dropped (logging like tar) and the absolute symlink target should get resolved relative to the working directory.

Does all of that make sense? I think it should provide the desired feature while preventing security issues, but I could be missing something.

[tgross]

@schmichael and @notnoop it looks like go-getter was updated after this PR in #6215, but it also looks like there's some ongoing discussion here that maybe we should capture in a standalone issue? Or do we think that's all resolved at this point?

[notnoop]

Thank you for the contribution. We've updated go-getter few times since this PR is made. Though, the symlink support got reverted, and the user visible issue is tracked in hashicorp/go-getter#60 and #2292 . Closing for now.

[github-actions]

I'm going to lock this pull request because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active contributions.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.