Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix corner cases in Bash runfiles library #16945

Closed
wants to merge 1 commit into from

Conversation

fmeum
Copy link
Collaborator

@fmeum fmeum commented Dec 7, 2022

Due to tests not actually verifying the exit code of rlocation calls, a few special cases were not handled correctly. In particular, absolute paths still went through regular rlocation lookup and could fail there even after the absolute path had been printed to stdout.

Also adds a missing newline in the (very rare) case that a manifest entry is found, but doesn't point to an existing file (e.g. if it is an unresolved symlink), and rlocation is not used with command substitution, but some other mechanism that doesn't strip trailing newlines.

The tests now assert the expected exit code (== 0 or != 0).

Fixes #16933

@fmeum fmeum marked this pull request as ready for review December 7, 2022 11:22
@fmeum
Copy link
Collaborator Author

fmeum commented Dec 7, 2022

@Wyverald Could you review?

FYI @meteorcloudy

@fmeum fmeum force-pushed the 16933-bash-runfiles-lib-rbe branch 2 times, most recently from 21c9f95 to e61f458 Compare December 7, 2022 11:36
@sgowroji sgowroji added team-ExternalDeps External dependency handling, remote repositiories, WORKSPACE file. awaiting-review PR is awaiting review from an assigned reviewer labels Dec 7, 2022
@fmeum fmeum force-pushed the 16933-bash-runfiles-lib-rbe branch from e61f458 to 47cd403 Compare December 7, 2022 12:07
@fmeum fmeum marked this pull request as draft December 7, 2022 12:07
@fmeum fmeum force-pushed the 16933-bash-runfiles-lib-rbe branch 3 times, most recently from f7d0158 to 46bdc3a Compare December 7, 2022 13:04
@fmeum
Copy link
Collaborator Author

fmeum commented Dec 7, 2022

I don't understand the CI failure yet. android_tools.tar isn't found, but it seems that is expected.

@gregestren
Copy link
Contributor

Looks okay to me, pending failures. @ahumesky any Android comments?

@ted-xie
Copy link
Contributor

ted-xie commented Dec 8, 2022

android_tools.tar isn't found, but it seems that is expected.

Seems fishy... why shouldn't it be found?

I have a couple questions about this PR:

  • Do we have to keep the set -x and env = {RUNFILES_LIB_DEBUG} lines in the diff? Seems like those are for debug purposes mostly. I am mildly concerned that leaving these in will inflate build and test logs globally for all Bazel users. I could also be misunderstanding the intent here.
  • The boilerplate runfiles incantation is repeated a bunch of times across the bazel repo, and probably in user projects too. Will we have to update these incantations if you update runfiles.bash?

@fmeum
Copy link
Collaborator Author

fmeum commented Dec 8, 2022

android_tools.tar isn't found, but it seems that is expected.

Seems fishy... why shouldn't it be found?

It looks like only a few tests declare android_tools.tar as a data dependency, but many tests call through to a helper function that locates it. Not sure whether that's correct, but I will update the PR to remain compatible with its previous behavior in this situation.

  • Do we have to keep the set -x and env = {RUNFILES_LIB_DEBUG} lines in the diff? Seems like those are for debug purposes mostly. I am mildly concerned that leaving these in will inflate build and test logs globally for all Bazel users. I could also be misunderstanding the intent here.

These are going away, I am just (ab)using CI to debug issues on Windows.

  • The boilerplate runfiles incantation is repeated a bunch of times across the bazel repo, and probably in user projects too. Will we have to update these incantations if you update runfiles.bash?

No, the incantation has remained compatible with the new version 3 of the runfiles library.

@fmeum fmeum force-pushed the 16933-bash-runfiles-lib-rbe branch from 46bdc3a to d013c6c Compare December 8, 2022 16:18
@fmeum fmeum marked this pull request as ready for review December 8, 2022 16:23
@fmeum fmeum requested review from ted-xie and removed request for gregestren and fweikert December 8, 2022 16:33
@meteorcloudy
Copy link
Member

meteorcloudy commented Dec 8, 2022

Downstream test for this change: https://buildkite.com/bazel/bazel-at-head-plus-downstream/builds/2775

@ted-xie
Copy link
Contributor

ted-xie commented Dec 8, 2022

@meteorcloudy is the original author of the runfiles macro so I think he's the most qualified to give the final review here. No concerns from the Android side for me, as long as all the existing tests pass in presubmit.

@ted-xie ted-xie removed their request for review December 8, 2022 17:56
Due to tests not actually verifying the exit code of rlocation calls,
a few special cases were not handled correctly. In particular, absolute
paths still went through regular rlocation lookup and could fail there
even after the absolute path had been printed to stdout.

Also adds a missing newline in the (very rare) case that a manifest
entry is found, but doesn't point to an existing file (e.g. if it is an
unresolved symlink), and `rlocation` is not used with command
substitution, but some other mechanism that doesn't strip trailing
newlines.

The tests now assert the expected exit code (== 0 or != 0).

Fixes bazelbuild#16933
@fmeum fmeum force-pushed the 16933-bash-runfiles-lib-rbe branch from d013c6c to 027ac02 Compare December 8, 2022 18:38
@fmeum
Copy link
Collaborator Author

fmeum commented Dec 8, 2022

@meteorcloudy Looks like presubmit failed due to CI overload. I amended without changes to rerun the pipeline.

@fmeum
Copy link
Collaborator Author

fmeum commented Dec 8, 2022

@bazel-io flag

@bazel-io bazel-io added the potential release blocker Flagged by community members using "@bazel-io flag". Should be added to a release blocker milestone label Dec 8, 2022
@Wyverald
Copy link
Member

Wyverald commented Dec 8, 2022

Downstream test results look good: https://buildkite.com/bazel/bazel-at-head-plus-downstream/builds/2775#_

@Wyverald Wyverald added awaiting-PR-merge PR has been approved by a reviewer and is ready to be merge internally and removed potential release blocker Flagged by community members using "@bazel-io flag". Should be added to a release blocker milestone awaiting-review PR is awaiting review from an assigned reviewer labels Dec 8, 2022
@sgowroji sgowroji removed the awaiting-PR-merge PR has been approved by a reviewer and is ready to be merge internally label Dec 12, 2022
@fmeum fmeum deleted the 16933-bash-runfiles-lib-rbe branch December 12, 2022 08:24
fmeum added a commit to fmeum/bazel that referenced this pull request Dec 12, 2022
Due to tests not actually verifying the exit code of rlocation calls, a few special cases were not handled correctly. In particular, absolute paths still went through regular rlocation lookup and could fail there even after the absolute path had been printed to stdout.

Also adds a missing newline in the (very rare) case that a manifest entry is found, but doesn't point to an existing file (e.g. if it is an unresolved symlink), and `rlocation` is not used with command substitution, but some other mechanism that doesn't strip trailing newlines.

The tests now assert the expected exit code (== 0 or != 0).

Fixes bazelbuild#16933

Closes bazelbuild#16945.

PiperOrigin-RevId: 494609104
Change-Id: I30333219176a61bda51f08c2c6bc927ce653d681
meteorcloudy pushed a commit that referenced this pull request Dec 12, 2022
Due to tests not actually verifying the exit code of rlocation calls, a few special cases were not handled correctly. In particular, absolute paths still went through regular rlocation lookup and could fail there even after the absolute path had been printed to stdout.

Also adds a missing newline in the (very rare) case that a manifest entry is found, but doesn't point to an existing file (e.g. if it is an unresolved symlink), and `rlocation` is not used with command substitution, but some other mechanism that doesn't strip trailing newlines.

The tests now assert the expected exit code (== 0 or != 0).

Fixes #16933

Closes #16945.

PiperOrigin-RevId: 494609104
Change-Id: I30333219176a61bda51f08c2c6bc927ce653d681
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
team-ExternalDeps External dependency handling, remote repositiories, WORKSPACE file.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Bazel 6.0.0rc4 breaks Bazel RBE tests
7 participants