[Tracking] Support for Hashicorp Vault

[svenkata9]

[There is no issue in Gramine at present; This issue is filed only for tracking purposes]

The issue is with building Hashicorp Vault source as a dynamically linked binary instead of a statically linked library which comes by default.

Vault source code present at https://github.com/hashicorp/vault was previously using gox for building the binary. That did not support dynamically linked binary. Over the second half of last year vault scripts were updated to use go directly, but by default the binary is built as statically linked. As we know, this binary has inline syscalls that degrades performance while running in Gramine.

When I tried to introduce --linkshared option there are three dependent ingredients that fail to link properly. There are issues discussed in GitHub and other forums, but I couldn't find any resolution for those.

I also tried to build a hello binary with --linkshared and that builds and works without issues.

root@gramine-sgx:/home/svenkat9/vault/vault1# make dev
==> Checking that build is using go version >= 1.20...
==> Using go version 1.20.1...
==> Removing old directory...
==> Building...
# github.com/cespare/xxhash/v2
asm: xxhash_amd64.s:120: when dynamic linking, R15 is clobbered by a global variable access and is used here: 00092 (/home/svenkat9/go/pkg/mod/github.com/cespare/xxhash/v2@v2.1.1/xxhash_amd64.s:120)  ADDQ    R15, AX
asm: assembly failed
# github.com/cloudflare/circl/math/fp25519
asm: fp_amd64.s:68: when dynamic linking, R15 is clobbered by a global variable access and is used here: 00215 (/home/svenkat9/go/pkg/mod/github.com/cloudflare/circl@v1.1.0/math/fp25519/fp_amd64.s:68)        MULXQ    24(SI), AX, R15
asm: assembly failed
# github.com/cloudflare/circl/math/fp448
asm: fp_amd64.s:68: when dynamic linking, R15 is clobbered by a global variable access and is used here: 00350 (/home/svenkat9/go/pkg/mod/github.com/cloudflare/circl@v1.1.0/math/fp448/fp_amd64.s:68)  MULXQ   48(SI), AX, R15
asm: assembly failed
Makefile:36: recipe for target 'dev' failed
make: *** [dev] Error 1

[kailun-qin]

There are issues discussed in GitHub and other forums, but I couldn't find any resolution for those.

Pls kindly see my comments below.

github.com/cespare/xxhash/v2

For xxhash, pls consider to update it to v2.1.2+(https://github.com/cespare/xxhash/releases/tag/v2.1.2) as a fix had been introduced since then (see cespare/xxhash#54 for details).

github.com/cloudflare/circl/math/fp25519
github.com/cloudflare/circl/math/fp448

For circl, pls consider to try its latest main branch. A fix addressing the issue has been introduced recently (cloudflare/circl#407, cloudflare/circl#391).

[svenkata9]

Thanks, Kailun. Updating with your suggestions made that build error go away, but landed in another error related to azure go sdk.

# github.com/hashicorp/vault
2023/03/07 13:53:12 duplicated definition of symbol github.com/Azure/azure-sdk-for-go/sdk/azcore/runtime.(*Poller[go.shape.struct { github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/compute/armcompute/v4.VirtualMachineAssessPatchesResult }]).PollUntilDone.func1, from github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/compute/armcompute/v4 and github.com/hashicorp/vault-plugin-auth-azure
Makefile:36: recipe for target 'dev' failed

[dimakuv]

I don't know how to fix the latest build error (maybe some googling will help?).

But it looks like slowly but steadily Golang itself and its popular libraries/modules are embracing the "dynamic linking" world. This is very good news for Gramine :)

Why?: The main reason is that Go compiler uses R15 to access to global variables, so it alters its value (do not restore it).
[ taken from https://github.com/cloudflare/circl/pull/407 ]

Go is full of such ad-hoc tricks...

[svenkata9]

Yeah. I tried updating some modules in azure-sdk-for-go to their latest, but that ends up in other issues.

[kailun-qin]

I tried updating some modules in azure-sdk-for-go to their latest, but that ends up in other issues.

The latest related modules do not resolve the issue. A more straightforward one is to disable azure plugin in building, however it's a built-in vault plugin which IDK how to make it...

[svenkata9]

Yes. I tried that as well, but I could not find a way to disable it.

[svenkata9]

The hashicorp dependencies that I encountered for building a dynamically linked binary are (possibly) resolved now, but there are errors internal to vault source that prevents the shared object build. I raised a GitHub issue with vault yesterday, and they closed saying it won't be supported.

Error building with -linkshared flag · Issue #22228 · hashicorp/vault (github.com)

[dimakuv]

Not sure what to do with this GitHub issue. Does anyone want to re-test support for Hashicorp Vault and report on the results? Maybe it's fully supported by now.

[mkow]

This should be closed, as it doesn't represent any concrete issue with Gramine and there's nothing to do on our side. If you find some specific problems with Vault then please file separate issues for each.