Skip to content

Commit

Permalink
Add FileDependencySerializer.
Browse files Browse the repository at this point in the history
Performs serialization of leaf-level FileValue and DirectoryListingValue
dependencies using the schema defined in file_invalidation_data.proto.

The logical flow mirrors that of FileFunction.

This is intended for use during serialization, where there is a
high degree of concurrency and many nodes may share the same file
dependencies.

PiperOrigin-RevId: 683855840
  • Loading branch information
Googler authored and copybara-github committed Oct 9, 2024
1 parent 1f907ad commit f5f6a7a
Show file tree
Hide file tree
Showing 2 changed files with 12 additions and 7 deletions.
1 change: 1 addition & 0 deletions third_party/bazel/src/main/protobuf/BUILD
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ FILES = [
"test_status",
"worker_protocol",
"execution_graph",
"file_invalidation_data",
]

[proto_library(
Expand Down
18 changes: 11 additions & 7 deletions third_party/bazel/src/main/protobuf/file_invalidation_data.proto
Original file line number Diff line number Diff line change
Expand Up @@ -53,9 +53,9 @@ option java_package = "com.google.devtools.build.lib.skyframe.serialization.prot
// as `parent_mtsv`.
//
// In the common case, `FileInvalidationData` only contains `parent_mtsv` and
// that's sufficient information to derive all the paths, assuming that the
// parent lookups are transitively performed. The `FileStateKey.argument`
// matches the path present in the key.
// `type' and that's sufficient information to derive all the paths, assuming
// that the parent lookups are transitively performed. The
// `FileStateKey.argument` matches the path present in the key.
//
// In the presence of symlinks, special logic applies to produce the resolved
// paths. A `symlinks` entry will be present. To unpack the symlink entry, we
Expand All @@ -80,8 +80,9 @@ message FileInvalidationData {
// Resolution means creating a key from this MTSV and the parent directory of
// the key of this entry, looking up the value and interpreting it.
//
// Optional. Only unset when the path has no parent, i.e. contains no '/'
// separators.
// Optional. Unset in the following two cases.
// 1. When the path has no parent, i.e. contains no '/' separators
// 2. It is a directory and has the common sentinel value, -1.
int64 parent_mtsv = 2;

// This field is repeated, with one entry for each symbolic link encountered
Expand Down Expand Up @@ -115,7 +116,7 @@ message FileInvalidationData {

// Invalidation information for a
// `com.google.devtools.build.lib.skyframe.DirectoryListingValue`. Note
// that this is designed for certain repository deltas have no directory
// that this is designed for certain repository deltas having no directory
// awareness. This means when a file within the directory changes, there's no
// change notification to the directory itself. The consumer is responsible for
// invalidating this entry when its children change in ways that would affect
Expand All @@ -136,6 +137,8 @@ message DirectoryListingInvalidationData {
string overflow_key = 1;

// MTSV required to resolve the real directory path.
//
// Optional. Only set if this is not the common, sentinel value, -1.
int64 file_mtsv = 2;
}

Expand All @@ -162,7 +165,8 @@ message Symlink {
//
// This field is optional. It is only set if the resolved path formed by
// combining the `contents` with the parent directory results in a *different*
// parent directory than the one used to resolve the symlink itself.
// parent directory than the one used to resolve the symlink itself and that
// value is not the common sentinel, -1.
//
// Optional.
int64 parent_mtsv = 2;
Expand Down

0 comments on commit f5f6a7a

Please sign in to comment.