Skip to content

Commit

Permalink
[5.1.0] cherrypick subpackages support (#14780)
Browse files Browse the repository at this point in the history
* Split ConfiguredAttributeMapper into a new build target.

PiperOrigin-RevId: 411055066

* Part 1 of the Implementation for new 'subpackages()` built-in helper function.

Design proposal: https://docs.google.com/document/d/13UOT0GoQofxDW40ILzH2sWpUOmuYy6QZ7CUmhej9vgk/edit#

This CL modifies the globber infrastructure to support an additional mode of listing sub-directories.

* Add new Globber Operation enum allowing, Globber implementations to
  discriminate between glob, glob w/directories and the future sub-packages
  use-case.

* Modify UnixGlob to replace Predicate and bools with UnixGlobPathDiscriminator interface for:
  a) Determining whether to traverse a sub-directory (previously was lambda)
  b) function for determing what entries to include in the List<Path> produced by UnixGlob.globAsync.

  These allow relatively simple re-use of the same logic for both subpackages and glob

4) Add a few tests for UnixGlob to ensure both cases continue to work as expected.

PiperOrigin-RevId: 421125424

* Part 2 Implementation for new 'subpackages()` built-in helper function.

Design proposal: https://docs.google.com/document/d/13UOT0GoQofxDW40ILzH2sWpUOmuYy6QZ7CUmhej9vgk/edit#

Overview:

Add StarlarkNativeModule 'subpackages' function with parameters that mirror glob()

PiperOrigin-RevId: 422652954

* Fix some typographical errors in the 'subpackages' docs.

PiperOrigin-RevId: 425942284

Co-authored-by: kkress <kkress@google.com>
Co-authored-by: Googler <noreply@google.com>
  • Loading branch information
3 people authored Feb 14, 2022
1 parent a58ddea commit dc41a20
Show file tree
Hide file tree
Showing 41 changed files with 1,674 additions and 471 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ title: Functions
<li><a href="#exports_files">exports_files</a></li>
<li><a href="#glob">glob</a></li>
<li><a href="#select">select</a></li>
<li><a href="#subpackages">subpackages</a></li>
</ul>
</div>
#end
Expand Down Expand Up @@ -636,48 +637,54 @@ sh_binary(
<li><code>select</code> works with most, but not all, attributes. Incompatible
attributes are marked <code>nonconfigurable</code> in their documentation.

</li>
</ul>

By default, Bazel produces the following error when no conditions match:
<pre class="code">
Configurable attribute "foo" doesn't match this configuration (would a default
condition help?).
Conditions checked:
//pkg:conditionA.
//pkg:conditionB.
</pre>
<!-- =================================================================
subpackages()
=================================================================
-->

You can signal more precise errors with <code>no_match_error</code>.
<h2 id="subpackages">subpackages</h2>

<h3 id="select_example">Examples</h3>
<pre>subpackages(include, exclude=[], allow_empty=True)</pre>

<pre class="code">
config_setting(
name = "windows",
values = {
"crosstool_top": "//crosstools/windows",
},
)
<p>
<code>subpackages()</code> is a helper function, similar to <code>glob()</code>
that lists subpackages instead of files and directories. It uses the same
path patterns as <code>glob()</code> and can match any subpackage that is a
direct decendant of the currently loading BUILD file. See <a
href="#glob">glob</a> for a detailed explanation and examples of include and
exclude patterns.
</p>

cc_binary(
name = "multiplatform_app",
...
linkopts = select({
":windows": [
"-Wl,windows_support1.lib",
"-Wl,windows_support2.lib",
],
"//conditions:default": [],
...
)
</pre>
<p>
The resulting list of subpackages returned is in sorted order and contains
paths relative to the current loading package that match the given patterns in
<code>include</code> and not those in <code>exclude</code>.

<p>In the above example, <code>multiplatform_app</code> links with additional
options when invoked with <code>bazel build //pkg:multiplatform_app
--crosstool_top=//crosstools/windows </code>.
<h3 id=subpackages_example">Example</h3>

<p>
The following example lists all the direct subpackages for the package <code>foo/BUILD</code>

<pre class="code">
# The following BUILD files exist:
# foo/BUILD
# foo/bar/baz/BUILD
# foo/sub/BUILD
# foo/sub/deeper/BUILD
#
# In foo/BUILD a call to
subs = subpackages(include = ["**"])

# results in subs == ["sub", "bar/baz"]
#
# 'sub/deeper' is not included because it is a subpackage of 'foo/sub' not of
# 'foo'
</pre>

<p>
In general it is preferred that instead of calling this function directly
that users use the 'subpackages' module of
<a href="https://github.com/bazelbuild/bazel-skylib">skylib</a>.

#if (!$singlePage)
#parse("com/google/devtools/build/docgen/templates/be/footer.vm")
Expand Down
1 change: 1 addition & 0 deletions src/main/java/com/google/devtools/build/lib/analysis/BUILD
Original file line number Diff line number Diff line change
Expand Up @@ -400,6 +400,7 @@ java_library(
"//src/main/java/com/google/devtools/build/lib/exec:test_xml_output_parser_exception",
"//src/main/java/com/google/devtools/build/lib/graph",
"//src/main/java/com/google/devtools/build/lib/packages",
"//src/main/java/com/google/devtools/build/lib/packages:configured_attribute_mapper",
"//src/main/java/com/google/devtools/build/lib/packages:exec_group",
"//src/main/java/com/google/devtools/build/lib/packages/semantics",
"//src/main/java/com/google/devtools/build/lib/profiler",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ java_library(
"//src/main/java/com/google/devtools/build/lib/exec:module_action_context_registry",
"//src/main/java/com/google/devtools/build/lib/exec:spawn_strategy_resolver",
"//src/main/java/com/google/devtools/build/lib/packages",
"//src/main/java/com/google/devtools/build/lib/packages:globber",
"//src/main/java/com/google/devtools/build/lib/profiler",
"//src/main/java/com/google/devtools/build/lib/rules/cpp",
"//src/main/java/com/google/devtools/build/lib/skyframe:containing_package_lookup_value",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@
import com.google.devtools.build.lib.concurrent.BlazeInterners;
import com.google.devtools.build.lib.events.Event;
import com.google.devtools.build.lib.includescanning.IncludeParser.Inclusion.Kind;
import com.google.devtools.build.lib.packages.Globber;
import com.google.devtools.build.lib.packages.NoSuchPackageException;
import com.google.devtools.build.lib.profiler.Profiler;
import com.google.devtools.build.lib.profiler.ProfilerTask;
Expand Down Expand Up @@ -347,7 +348,7 @@ ImmutableSet<Artifact> getPathLevelHintedInclusions(
containingPackageLookupValue.getContainingPackageName(),
containingPackageLookupValue.getContainingPackageRoot(),
pattern,
/*excludeDirs=*/ true,
Globber.Operation.FILES,
relativePath.relativeTo(packageFragment)));
} catch (InvalidGlobPatternException e) {
env.getListener()
Expand Down
32 changes: 31 additions & 1 deletion src/main/java/com/google/devtools/build/lib/packages/BUILD
Original file line number Diff line number Diff line change
Expand Up @@ -10,21 +10,51 @@ filegroup(
visibility = ["//src:__subpackages__"],
)

java_library(
name = "configured_attribute_mapper",
srcs = ["ConfiguredAttributeMapper.java"],
deps = [
":packages",
"//src/main/java/com/google/devtools/build/lib/analysis:config/config_matching_provider",
"//src/main/java/com/google/devtools/build/lib/cmdline",
"//third_party:guava",
"//third_party:jsr305",
],
)

java_library(
name = "globber",
srcs = ["Globber.java"],
)

java_library(
name = "globber_utils",
srcs = ["GlobberUtils.java"],
deps = [
":globber",
"//third_party:error_prone_annotations",
],
)

java_library(
name = "packages",
srcs = glob(
["*.java"],
exclude = [
"BuilderFactoryForTesting.java", # see builder_factory_for_testing
"Globber.java",
"GlobberUtils.java",
"ExecGroup.java",
"ConfiguredAttributeMapper.java",
],
),
deps = [
":exec_group",
":globber",
":globber_utils",
"//src/main/java/com/google/devtools/build/docgen/annot",
"//src/main/java/com/google/devtools/build/lib/actions:execution_requirements",
"//src/main/java/com/google/devtools/build/lib/actions:thread_state_receiver",
"//src/main/java/com/google/devtools/build/lib/analysis:config/config_matching_provider",
"//src/main/java/com/google/devtools/build/lib/analysis:config/fragment",
"//src/main/java/com/google/devtools/build/lib/analysis:config/fragment_class_set",
"//src/main/java/com/google/devtools/build/lib/analysis:config/transitions/configuration_transition",
Expand Down
Loading

0 comments on commit dc41a20

Please sign in to comment.