From 3b2e649fbf65fccf531f14070c9e571abd880474 Mon Sep 17 00:00:00 2001 From: Tiago Quelhas Date: Tue, 13 Feb 2024 16:55:00 +0100 Subject: [PATCH] [7.1.0] Clarify the purpose and overall behavior of RemoteActionFileSystem. (#21294) PiperOrigin-RevId: 584679218 Change-Id: I535fa24c3429285f393278e48e13327c8c52b0c3 --- .../lib/remote/RemoteActionFileSystem.java | 31 +++++++++++++++---- 1 file changed, 25 insertions(+), 6 deletions(-) diff --git a/src/main/java/com/google/devtools/build/lib/remote/RemoteActionFileSystem.java b/src/main/java/com/google/devtools/build/lib/remote/RemoteActionFileSystem.java index 03e7f0827dd3dd..b819f58f30e204 100644 --- a/src/main/java/com/google/devtools/build/lib/remote/RemoteActionFileSystem.java +++ b/src/main/java/com/google/devtools/build/lib/remote/RemoteActionFileSystem.java @@ -68,14 +68,33 @@ import javax.annotation.Nullable; /** - * This is a basic implementation and incomplete implementation of an action file system that's been - * tuned to what internal (non-spawn) actions in Bazel currently use. + * An action filesystem suitable for use when building with remote caching or execution. * - *

The implementation mostly delegates to the local file system except for the case where an - * action input is a remotely stored action output. Most notably {@link - * #getInputStream(PathFragment)} and {@link #createSymbolicLink(PathFragment, PathFragment)}. + *

It acts as a union filesystem over three different sources: * - *

This implementation only supports creating local action outputs. + *

+ * + *

Generally speaking, file operations consult the underlying sources in that order and operate + * on the first result found, although some (e.g. readdir) collate information from all sources. The + * contents of remotely stored files are transparently downloaded when an operation requires them. + * + *

Special care must be taken with operations that follow symlinks, as the symlink and its target + * path may reside on different sources, with an arbitrary number of indirections in between. This + * is required because some actions (notably SymlinkAction) may materialize an output as a symlink + * to an input. Most operations call resolveSymbolicLinks upfront (which is able to canonicalize + * paths taking every source into account) and only then delegate to the underlying sources. + * + *

The implementation assumes that an action never modifies its input files, but may otherwise + * modify any path in the output tree, including modifications that undo previous ones (e.g., + * writing to a path and then moving it elsewhere). No effort is made to detect irreconcilable + * differences between sources (e.g., distinct files of the same name in two different sources). */ public class RemoteActionFileSystem extends AbstractFileSystemWithCustomStat { private final PathFragment execRoot;