Remote Cache should be safe to share between environments

[schroederc]

Re: #5642

The thread of bazelbuild/rules_android#321 implies that Bazel requires either --action_env or --experimental_remote_platform_override to change between build environments when sharing a remote cache. This seems like not all inputs to the build are being taken into account when writing/reading from the cache which means it's not safe to share a cache. If users sharing a cache have different C++ toolchains, Bazel will incorrectly reuse some artifacts that relate to specific, local toolchain paths.

This also implies other issues. What happens if a single user of the cache installs a newer version of Clang or switches to GCC from Clang? Can Bazel not detect that and invalidate the cache? What happens if users want to share a cache between their machines and they use different C++ toolchains, but everything else is the same for their Java, Go, etc. targets?

The Remote Caching documentation doesn't say anything about these limitations.

[buchgr]

@schroederc

Ultimately, it's up to the rules to be hermetic and declare all tools an action uses as inputs. The C++ rules are special in that they don't do that, because of legacy issues and it was deemed almost impossible and IIUC a fully hermetic C++ toolchain does not exist. I think one can be mostly hermetic (or indistinguishable from it in practice) by checking in the compiler and the toolchain in the repository or specifying it via remote repository and not using the system installed compiler, libraries.

One the other hand, the Go rules for example are fully hermetic and it should be safe to share a remote cache for Go between different users.

Under known issues, the documentation mentions this problem. I agree it could be improved.

Bazel currently does not track tools outside a workspace. This can be a problem if, for example, an action uses a compiler from /usr/bin/. Then, two users with different compilers installed will wrongly share cache hits because the outputs are different but they have the same action hash. Please watch issue #4558 for updates.

[buchgr]

I think this issue can be closed in favor of #4558?