Skip to content

Commit

Permalink
[7.4.0] external dependency doc updates (bazelbuild#23741)
Browse files Browse the repository at this point in the history
original commits:
* 13ecdf5
* 114c0c6
* 970120f
* 1ab2c82
* c04a561

---------

Co-authored-by: Grzegorz Lukasik <32371993+hauserx@users.noreply.github.com>
Co-authored-by: Jamison Lahman <lahmanja@gmail.com>
  • Loading branch information
3 people authored Sep 25, 2024
1 parent 3c71860 commit e253020
Show file tree
Hide file tree
Showing 20 changed files with 258 additions and 284 deletions.
2 changes: 1 addition & 1 deletion repositories.bzl
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
"""

load("//tools/build_defs/repo:http.bzl", "http_archive", "http_file")
load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive", "http_file")
load("//src/tools/bzlmod:utils.bzl", "get_canonical_repo_name")

##################################################################################
Expand Down
54 changes: 22 additions & 32 deletions site/en/concepts/build-ref.md
Original file line number Diff line number Diff line change
@@ -1,47 +1,37 @@
Project: /_project.yaml
Book: /_book.yaml

# Workspaces, packages, and targets
# Repositories, workspaces, packages, and targets

{% include "_buttons.html" %}

Bazel builds software from source code organized in a directory tree called a
workspace. Source files in the workspace are organized in a nested hierarchy of
packages, where each package is a directory that contains a set of related
source files and one `BUILD` file. The `BUILD` file specifies what software
outputs can be built from the source.

## Workspace {:#workspace}

A _workspace_ is a directory tree on your filesystem that contains the source
files for the software you want to build. Each workspace has a text file named
`WORKSPACE` which may be empty, or may contain references to [external
dependencies](/docs/external) required to build the outputs.

Directories containing a file called `WORKSPACE` are considered the root of a
workspace. Therefore, Bazel ignores any directory trees in a workspace rooted at
a subdirectory containing a `WORKSPACE` file, as they form another workspace.

Bazel also supports `WORKSPACE.bazel` file as an alias of `WORKSPACE` file. If
both files exist, `WORKSPACE.bazel` is used.
Bazel builds software from source code organized in directory trees called
repositories. A defined set of repositories comprises the workspace. Source
files in repositories are organized in a nested hierarchy of packages, where
each package is a directory that contains a set of related source files and one
`BUILD` file. The `BUILD` file specifies what software outputs can be built from
the source.

### Repositories {:#repositories}

Code is organized in _repositories_. The directory containing the `WORKSPACE`
file is the root of the main repository, also called `@`. Other, (external)
repositories are defined in the `WORKSPACE` file using workspace rules, or
generated from modules and extensions in the Bzlmod system. See [external
Source files used in a Bazel build are organized in _repositories_ (often
shortened to _repos_). A repo is a directory tree with a boundary marker file at
its root; such a boundary marker file could be `MODULE.bazel`, `REPO.bazel`, or
in legacy contexts, `WORKSPACE` or `WORKSPACE.bazel`.

The repo in which the current Bazel command is being run is called the _main
repo_. Other, (external) repos are defined by _repo rules_; see [external
dependencies overview](/external/overview) for more information.

The workspace rules bundled with Bazel are documented in the [Workspace
Rules](/reference/be/workspace) section in the [Build
Encyclopedia](/reference/be/overview) and the documentation on [embedded
Starlark repository rules](/rules/lib/repo/index).
## Workspace {:#workspace}

A _workspace_ is the environment shared by all Bazel commands run from the same
main repo. It encompasses the main repo and the set of all defined external
repos.

As external repositories are repositories themselves, they often contain a
`WORKSPACE` file as well. However, these additional `WORKSPACE` files are
ignored by Bazel. In particular, repositories depended upon transitively are not
added automatically.
Note that historically the concepts of "repository" and "workspace" have been
conflated; the term "workspace" has often been used to refer to the main
repository, and sometimes even used as a synonym of "repository".

## Packages {:#packages}

Expand Down
8 changes: 4 additions & 4 deletions site/en/concepts/platforms.md
Original file line number Diff line number Diff line change
Expand Up @@ -175,7 +175,7 @@ For your project to build, you need to check the following:
1. The toolchains you want to use must exist. If using stock toolchains, the
language owners should include instructions for how to register them. If
writing your own custom toolchains, you need to [register](https://bazel.build/extending/toolchains#registering-building-toolchains) them in your
`WORKSPACE` or with [`--extra_toolchains`](https://bazel.build/reference/command-line-reference#flag--extra_toolchains).
`MODULE.bazel` file or with [`--extra_toolchains`](https://bazel.build/reference/command-line-reference#flag--extra_toolchains).

1. `select()`s and [configuration transitions][Starlark transitions] must
resolve properly. See [select()](#select) and [Transitions](#transitions).
Expand Down Expand Up @@ -374,8 +374,8 @@ When building `$ bazel build //:myproject --platforms=//:myplatform`, Bazel
automatically selects a toolchain that can run on the build machine and
build binaries for `//:myplatform`. This is known as *toolchain resolution*.

The set of available toolchains can be registered in the `WORKSPACE` with
[`register_toolchains`][register_toolchains Function] or at the
The set of available toolchains can be registered in the `MODULE.bazel` file
with [`register_toolchains`][register_toolchains Function] or at the
command line with [`--extra_toolchains`][extra_toolchains Flag].

For more information see [here][Toolchains].
Expand Down Expand Up @@ -419,7 +419,7 @@ contact [bazel-dev]{: .external}.
[Platforms examples]: https://github.com/hlopko/bazel_platforms_examples
[platform mappings design]: https://docs.google.com/document/d/1Vg_tPgiZbSrvXcJ403vZVAGlsWhH9BUDrAxMOYnO0Ls/edit
[platform Rule]: /reference/be/platforms-and-toolchains#platform
[register_toolchains Function]: /rules/lib/globals/workspace#register_toolchains
[register_toolchains Function]: /rules/lib/globals/module#register_toolchains
[Rust rules]: https://github.com/bazelbuild/rules_rust
[select()]: /docs/configurable-attributes
[select() Platforms]: /docs/configurable-attributes#platforms
Expand Down
11 changes: 7 additions & 4 deletions site/en/configure/attributes.md
Original file line number Diff line number Diff line change
Expand Up @@ -916,9 +916,12 @@ def selecty_genrule(name, select_cmd):
### Why doesn't select() work with bind()? {:#faq-select-bind}
Because [`bind()`](/reference/be/workspace#bind) is a WORKSPACE rule, not a BUILD rule.
First of all, do not use `bind()`. It is deprecated in favor of `alias()`.
Workspace rules do not have a specific configuration, and aren't evaluated in
The technical answer is that [`bind()`](/reference/be/workspace#bind) is a repo
rule, not a BUILD rule.
Repo rules do not have a specific configuration, and aren't evaluated in
the same way as BUILD rules. Therefore, a `select()` in a `bind()` can't
actually evaluate to any specific branch.
Expand All @@ -927,8 +930,6 @@ the `actual` attribute, to perform this type of run-time determination. This
works correctly, since `alias()` is a BUILD rule, and is evaluated with a
specific configuration.
You can even have a `bind()` target point to an `alias()`, if needed.
```sh
$ cat WORKSPACE
workspace(name = "myapp")
Expand Down Expand Up @@ -957,6 +958,8 @@ With this setup, you can pass `--define ssl_library=alternative`, and any target
that depends on either `//:ssl` or `//external:ssl` will see the alternative
located at `@alternative//:ssl`.
But really, stop using `bind()`.
### Why doesn't my select() choose what I expect? {:#faq-select-choose-condition}
If `//myapp:foo` has a `select()` that doesn't choose the condition you expect,
Expand Down
2 changes: 1 addition & 1 deletion site/en/configure/windows.md
Original file line number Diff line number Diff line change
Expand Up @@ -268,7 +268,7 @@ To enable the Clang toolchain for building C++, there are several situations.
--extra_toolchains=@local_config_cc//:cc-toolchain-x64_windows-clang-cl --extra_execution_platforms=//:x64_windows-clang-cl
```

* Register the platform and toolchain in your `WORKSPACE` file:
* Register the platform and toolchain in your `MODULE.bazel` file:

```
register_execution_platforms(
Expand Down
89 changes: 36 additions & 53 deletions site/en/contribute/codebase.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,10 +47,10 @@ and some are after (`-c opt`); the former kind is called a "startup option" and
affects the server process as a whole, whereas the latter kind, the "command
option", only affects a single command.

Each server instance has a single associated source tree ("workspace") and each
workspace usually has a single active server instance. This can be circumvented
by specifying a custom output base (see the "Directory layout" section for more
information).
Each server instance has a single associated workspace (collection of source
trees known as "repositories") and each workspace usually has a single active
server instance. This can be circumvented by specifying a custom output base
(see the "Directory layout" section for more information).

Bazel is distributed as a single ELF executable that is also a valid .zip file.
When you type `bazel`, the above ELF executable implemented in C++ (the
Expand Down Expand Up @@ -96,8 +96,9 @@ from the client are handled by `GrpcServerImpl.run()`.
Bazel creates a somewhat complicated set of directories during a build. A full
description is available in [Output directory layout](/remote/output-directories).

The "workspace" is the source tree Bazel is run in. It usually corresponds to
something you checked out from source control.
The "main repo" is the source tree Bazel is run in. It usually corresponds to
something you checked out from source control. The root of this directory is
known as the "workspace root".

Bazel puts all of its data under the "output user root". This is usually
`$HOME/.cache/bazel/_bazel_${USER}`, but can be overridden using the
Expand Down Expand Up @@ -226,22 +227,19 @@ Bazel, in contrast, supports projects whose source code spans multiple
repositories. The repository from which Bazel is invoked is called the "main
repository", the others are called "external repositories".

A repository is marked by a file called `WORKSPACE` (or `WORKSPACE.bazel`) in
its root directory. This file contains information that is "global" to the whole
build, for example, the set of available external repositories. It works like a
regular Starlark file which means that one can `load()` other Starlark files.
This is commonly used to pull in repositories that are needed by a repository
that's explicitly referenced (we call this the "`deps.bzl` pattern")
A repository is marked by a repo boundary file (`MODULE.bazel`, `REPO.bazel`, or
in legacy contexts, `WORKSPACE` or `WORKSPACE.bazel`) in its root directory. The
main repo is the source tree where you're invoking Bazel from. External repos
are defined in various ways; see [external dependencies
overview](/external/overview) for more information.

Code of external repositories is symlinked or downloaded under
`$OUTPUT_BASE/external`.

When running the build, the whole source tree needs to be pieced together; this
is done by `SymlinkForest`, which symlinks every package in the main repository
to `$EXECROOT` and every external repository to either `$EXECROOT/external` or
`$EXECROOT/..` (the former of course makes it impossible to have a package
called `external` in the main repository; that's why we are migrating away from
it)
`$EXECROOT/..`.

### Packages {:#packages}

Expand Down Expand Up @@ -277,18 +275,19 @@ Globbing is implemented in the following classes:
the legacy globber in order to avoid "Skyframe restarts" (described below)

The `Package` class itself contains some members that are exclusively used to
parse the WORKSPACE file and which do not make sense for real packages. This is
parse the "external" package (related to external dependencies) and which do not
make sense for real packages. This is
a design flaw because objects describing regular packages should not contain
fields that describe something else. These include:

* The repository mappings
* The registered toolchains
* The registered execution platforms

Ideally, there would be more separation between parsing the WORKSPACE file from
parsing regular packages so that `Package`does not need to cater for the needs
of both. This is unfortunately difficult to do because the two are intertwined
quite deeply.
Ideally, there would be more separation between parsing the "external" package
from parsing regular packages so that `Package` does not need to cater for the
needs of both. This is unfortunately difficult to do because the two are
intertwined quite deeply.

### Labels, Targets, and Rules {:#labels-targets-rules}

Expand Down Expand Up @@ -417,25 +416,21 @@ implementation used in Bazel is currently an interpreter.

Starlark is used in several contexts, including:

1. **The `BUILD` language.** This is where new rules are defined. Starlark code
running in this context only has access to the contents of the `BUILD` file
itself and `.bzl` files loaded by it.
2. **Rule definitions.** This is how new rules (such as support for a new
language) are defined. Starlark code running in this context has access to
the configuration and data provided by its direct dependencies (more on this
later).
3. **The WORKSPACE file.** This is where external repositories (code that's not
in the main source tree) are defined.
4. **Repository rule definitions.** This is where new external repository types
are defined. Starlark code running in this context can run arbitrary code on
the machine where Bazel is running, and reach outside the workspace.
1. **`BUILD` files.** This is where new build targets are defined. Starlark
code running in this context only has access to the contents of the `BUILD`
file itself and `.bzl` files loaded by it.
2. **The `MODULE.bazel` file.** This is where external dependencies are
defined. Starlark code running in this context only has very limited access
to a few predefined directives.
3. **`.bzl` files.** This is where new build rules, repo rules, module
extensions are defined. Starlark code here can define new functions and load
from other `.bzl` files.

The dialects available for `BUILD` and `.bzl` files are slightly different
because they express different things. A list of differences is available
[here](/rules/language#differences-between-build-and-bzl-files).

More information about Starlark is available
[here](/rules/language).
More information about Starlark is available [here](/rules/language).

## The loading/analysis phase {:#loading-phase}

Expand Down Expand Up @@ -721,7 +716,7 @@ time to load.

Execution platforms are specified in one of the following ways:

1. In the WORKSPACE file using the `register_execution_platforms()` function
1. In the MODULE.bazel file using the `register_execution_platforms()` function
2. On the command line using the --extra\_execution\_platforms command line
option

Expand All @@ -736,7 +731,7 @@ yet.
The set of toolchains to be used for a configured target is determined by
`ToolchainResolutionFunction`. It is a function of:

* The set of registered toolchains (in the WORKSPACE file and the
* The set of registered toolchains (in the MODULE.bazel file and the
configuration)
* The desired execution and target platforms (in the configuration)
* The set of toolchain types that are required by the configured target (in
Expand Down Expand Up @@ -1444,6 +1439,11 @@ This is implemented in the `build.lib.buildeventservice` and

## External repositories {:#external-repos}

Note: The information in this section is out of date, as code in this area has
undergone extensive change in the past couple of years. Please refer to
[external dependencies overview](/external/overview) for more up-to-date
information.

Whereas Bazel was originally designed to be used in a monorepo (a single source
tree containing everything one needs to build), Bazel lives in a world where
this is not necessarily true. "External repositories" are an abstraction used to
Expand Down Expand Up @@ -1515,23 +1515,6 @@ invalidated when the definition of the repository they are in changes. Thus,
`FileStateValue`s for an artifact in an external repository need to depend on
their external repository. This is handled by `ExternalFilesHelper`.

### Managed directories {:#managed-directories}

Sometimes, external repositories need to modify files under the workspace root
(such as a package manager that houses the downloaded packages in a subdirectory of
the source tree). This is at odds with the assumption Bazel makes that source
files are only modified by the user and not by itself and allows packages to
refer to every directory under the workspace root. In order to make this kind of
external repository work, Bazel does two things:

1. Allows the user to specify subdirectories of the workspace Bazel is not
allowed to reach into. They are listed in a file called `.bazelignore` and
the functionality is implemented in `BlacklistedPackagePrefixesFunction`.
2. We encode the mapping from the subdirectory of the workspace to the external
repository it is handled by into `ManagedDirectoriesKnowledge` and handle
`FileStateValue`s referring to them in the same way as those for regular
external repositories.

### Repository mappings {:#repo-mappings}

It can happen that multiple repositories want to depend on the same repository,
Expand Down
9 changes: 7 additions & 2 deletions site/en/docs/configurable-attributes.md
Original file line number Diff line number Diff line change
Expand Up @@ -916,9 +916,12 @@ def selecty_genrule(name, select_cmd):
### Why doesn't select() work with bind()? {:#faq-select-bind}
Because [`bind()`](/reference/be/workspace#bind) is a WORKSPACE rule, not a BUILD rule.
First of all, do not use `bind()`. It is deprecated in favor of `alias()`.
Workspace rules do not have a specific configuration, and aren't evaluated in
The technical answer is that [`bind()`](/reference/be/workspace#bind) is a repo
rule, not a BUILD rule.
Repo rules do not have a specific configuration, and aren't evaluated in
the same way as BUILD rules. Therefore, a `select()` in a `bind()` can't
actually evaluate to any specific branch.
Expand Down Expand Up @@ -957,6 +960,8 @@ With this setup, you can pass `--define ssl_library=alternative`, and any target
that depends on either `//:ssl` or `//external:ssl` will see the alternative
located at `@alternative//:ssl`.
But really, stop using `bind()`.
### Why doesn't my select() choose what I expect? {:#faq-select-choose-condition}
If `//myapp:foo` has a `select()` that doesn't choose the condition you expect,
Expand Down
5 changes: 2 additions & 3 deletions site/en/extending/macros.md
Original file line number Diff line number Diff line change
Expand Up @@ -138,8 +138,7 @@ def my_macro(name, visibility=None):

If you need to know the package name (for example, which `BUILD` file is calling the
macro), use the function [native.package_name()](/rules/lib/toplevel/native#package_name).
Note that `native` can only be used in `.bzl` files, and not in `WORKSPACE` or
`BUILD` files.
Note that `native` can only be used in `.bzl` files, and not in `BUILD` files.

## Label resolution in macros {:#label-resolution}

Expand All @@ -164,7 +163,7 @@ def my_cc_wrapper(name, deps = [], **kwargs):
Label("//config:needs_foo"): [
# Due to the use of Label, this label will resolve to the correct target
# even if the canonical name of @dep_of_my_ruleset should be different
# in the main workspace, such as due to repo mappings.
# in the main repo, such as due to repo mappings.
Label("@dep_of_my_ruleset//tools:foo"),
],
"//conditions:default": [],
Expand Down
Loading

0 comments on commit e253020

Please sign in to comment.