Make Java runfiles library repo mapping aware

[fmeum]

Work towards #16124

[fmeum]

@Wyverald Could you review this? It's useless without #16534 and #16321, but can be tested and reviewed in isolation.

[Wyverald]

looks good to me!

[comius]

Hey, one more question. I noticed a pattern in the repo, that works and is slightly less invasive:

https://cs.opensource.google/bazel/bazel/+/master:src/test/java/com/google/devtools/build/android/dexer/DexFileSplitterTest.java;l=57-60;drc=0a23d46976c3fc999d44fbd1e37732ec2442d485
https://cs.opensource.google/bazel/bazel/+/master:src/test/java/com/google/devtools/build/android/dexer/BUILD;l=63-66;drc=7ce19e4128f7a5db219623aadf448d5690097140

What do you think about that? Could we ask users to use canonical repo names?

The benefit I see is, that we wouldn't need repo_mapping, changes to Runfiles library or an annotation (in pr #16534). The mechanism is neat, but needs some more effort from the user. Perhaps it's possible to improve on it?

The drawback is, that if you go forward with repo_mappings and annotation, you probably cancel out this mechanism.
WDYT?

[fmeum]

@comius This pattern is just great for binaries and tests with small scope, which is exactly why I'm pushing so hard to get #16428 in without a flag: It would already be useful in today's WORKSPACE world and would be required with Bzlmod as the "hardcoded repo name + $(location)" approach breaks down there due to repo mapping.

It doesn't work well for binaries with more data deps and in particular doesn't work well with libraries that bring in data deps since the top-level binary would need to pass in all their paths explicitly, which gets infeasible pretty quickly.

[fmeum]

The drawback is, that if you go forward with repo_mappings and annotation, you probably cancel out this mechanism.
WDYT?

This is not a problem: The $(rlocationpath ...) or repo + $(location ...) strings are still accepted by Rlocation. This is possible since canonical repo names aren't valid apparent repo names and vice versa. If a canonical repo name is passed in, it will simply not be repo mapped and used as is.

[comius]

This is not a problem: The $(rlocationpath ...) or repo + $(location ...) strings are still accepted by Rlocation. This is possible since canonical repo names aren't valid apparent repo names and vice versa. If a canonical repo name is passed in, it will simply not be repo mapped and used as is.

True.

[comius]

It doesn't work well for binaries with more data deps and in particular doesn't work well with libraries that bring in data deps since the top-level binary would need to pass in all their paths explicitly, which gets infeasible pretty quickly.

Ah, thanks for the answer!

[fmeum]

I rebased onto #16534 and #16652 so that I can add an integration test.

[fmeum]

@comius I updated the documentation and made this work in sandboxes by looking up the repository mapping under its root symlink _repo_mapping introduced by #16652. Could you review the last three commits? The commits I rebased onto are all part of other PRs.

[comius]

@comius I updated the documentation and made this work in sandboxes by looking up the repository mapping under its root symlink _repo_mapping introduced by #16652. Could you review the last three commits? The commits I rebased onto are all part of other PRs.

I did a quick review and the last three commits all look reasonable.

One more thing you should consider is making preload() method a lazy initializer / preloaded runfiles a singleton. Singleton pattern is usually avoided at all costs, however it looks at the moment Bazel's design forces you into a single global instance.

Making it a singleton might prevent involuntary duplication by the user. Libraries using runfiles might be coming from a different repository, that could follow a different philosophy and there might be no way of passing the Runfiles object down to those libraries.

[fmeum]

@comius I can make preload() return a singleton but keep preload(Map) around in case users rely on custom runfiles locations (e.g. when using manifests at install time). What do you think of that?

Should I use WeakReference or SoftReference for the preload() singleton? Not being able to free the manifests memory may otherwise be wasteful.

[comius]

@comius I can make preload() return a singleton but keep preload(Map) around in case users rely on custom runfiles locations (e.g. when using manifests at install time). What do you think of that?

Sounds ok.

Should I use WeakReference or SoftReference for the preload() singleton? Not being able to free the manifests memory may otherwise be wasteful.

I think with or without it should be fine. Without it the could would be simpler and with it you can arguably save some memory in some cases. I can't say there's a strong case for one or the other decision.

[fmeum]

@comius I went with a softly cached singleton instance in a new commit.

[fmeum]

@bazel-io flag

[meteorcloudy]

@bazel-io fork 6.0.0

[fmeum]

Rebased on master, but remains stacked on #16534.

[fmeum]

Rebased and no longer stacked.