Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[7.1.0] Clarify the purpose and overall behavior of RemoteActionFileSystem. #21294

Merged
merged 1 commit into from
Feb 13, 2024
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -68,14 +68,33 @@
import javax.annotation.Nullable;

/**
* This is a basic implementation and incomplete implementation of an action file system that's been
* tuned to what internal (non-spawn) actions in Bazel currently use.
* An action filesystem suitable for use when building with remote caching or execution.
*
* <p>The implementation mostly delegates to the local file system except for the case where an
* action input is a remotely stored action output. Most notably {@link
* #getInputStream(PathFragment)} and {@link #createSymbolicLink(PathFragment, PathFragment)}.
* <p>It acts as a union filesystem over three different sources:
*
* <p>This implementation only supports creating local action outputs.
* <ul>
* <li>The action input map, providing read-only in-memory access to the metadata (but not the
* contents) of the action's declared inputs.
* <li>The remote output tree, an in-memory filesystem providing read/write access to the metadata
* (but not the contents) of remotely stored files injected during action execution.
* <li>The local filesystem, providing read/write access to the metadata and contents of files
* residing on disk, including the inputs and outputs of local spawns.
* </ul>
*
* <p>Generally speaking, file operations consult the underlying sources in that order and operate
* on the first result found, although some (e.g. readdir) collate information from all sources. The
* contents of remotely stored files are transparently downloaded when an operation requires them.
*
* <p>Special care must be taken with operations that follow symlinks, as the symlink and its target
* path may reside on different sources, with an arbitrary number of indirections in between. This
* is required because some actions (notably SymlinkAction) may materialize an output as a symlink
* to an input. Most operations call resolveSymbolicLinks upfront (which is able to canonicalize
* paths taking every source into account) and only then delegate to the underlying sources.
*
* <p>The implementation assumes that an action never modifies its input files, but may otherwise
* modify any path in the output tree, including modifications that undo previous ones (e.g.,
* writing to a path and then moving it elsewhere). No effort is made to detect irreconcilable
* differences between sources (e.g., distinct files of the same name in two different sources).
*/
public class RemoteActionFileSystem extends AbstractFileSystemWithCustomStat {
private final PathFragment execRoot;
Expand Down
Loading