Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[6.0.0] Only inject metadata for outputs that cannot be reconstructed by skyf… #16879

Merged
merged 4 commits into from
Nov 29, 2022
Merged

[6.0.0] Only inject metadata for outputs that cannot be reconstructed by skyf… #16879

merged 4 commits into from
Nov 29, 2022

Commits on Nov 29, 2022

  1. Update RemoteActionFileSystem to apply permission changes to local fi… 

    …les even if remote file exists.

Previously, it only applies permission changes to local files when the remote one doesn't exist. It is fine because the call sites only use these method when remote file are missing. However, this isn't true with future changes.

PiperOrigin-RevId: 490872822
Change-Id: I7a19d99cd828294cbafa7b5f3fdc368d64e556ec
    @coeuvre
    coeuvre committed Nov 29, 2022
    Configuration menu
    Copy the full SHA
    e2a11dd View commit details
    Browse the repository at this point in the history

  2. Fix permission operations for the case of only remote directory. 

    PiperOrigin-RevId: 491280334
Change-Id: I30afef9f069eca8aee4d983664f42b3961e95adf
    @coeuvre
    coeuvre committed Nov 29, 2022
    Configuration menu
    Copy the full SHA
    42911ce View commit details
    Browse the repository at this point in the history

  3. Only inject metadata for outputs that cannot be reconstructed by skyf… 

    …rame later

Currently, they are symlinks.

For other outputs, let skyframe read action file system to construct the metadata.

Before this change, we inject metadata of symlink outputs, tree outputs and file outputs inside `RemoteActionFileSystem#flush()` if these outputs are stored inside the in-memory fs. If the outputs are somehow generated in the local fs, skyframe will read the fs later to construct metadata for them.

However, for tree outputs, skyframe always create an empty directory before action execution in the in-memory fs. So inside the `flush`, we always inject metadata for it. It means local tree outputs are ignored. We could update the code to also read local file system when reading tree files, but then the problem is how to construct metadata from file status which is well done by skyframe.

So instead of injecting metadata by traversal the filesystem inside `flush`, we just let skyframe to do the job.

Fixes bazelbuild#16789.

Closes bazelbuild#16812.

PiperOrigin-RevId: 491622005
Change-Id: I10434e6856a1b2a207f39e07122a9b646edf518c
    @coeuvre
    coeuvre committed Nov 29, 2022
    Configuration menu
    Copy the full SHA
    99928cd View commit details
    Browse the repository at this point in the history

  4. Merge branch 'release-6.0.0' into cherry-pick-16789

    @ShreeM01
    ShreeM01 authored Nov 29, 2022
    Configuration menu
    Copy the full SHA
    4cdf137 View commit details
    Browse the repository at this point in the history