Skip to content

Commit

Permalink
Update repo rule docs to mention watch and remove restarts
Browse files Browse the repository at this point in the history 
The remarks about files being watched only if they're labels are no longer accurate.

Repo rule restarts are gone by default (`--experimental_worker_for_repo_fetching` defaults to `auto`), so the section doesn't really apply anymore.

Fixes bazelbuild#22494

PiperOrigin-RevId: 668317824
Change-Id: Iba380b944cce2b5c8e7bd07606b89035ab401e72
  • Loading branch information
@Wyverald @copybara-github
Wyverald authored and copybara-github committed Aug 28, 2024
1 parent 623c24a commit 1ab2c82
Showing 1 changed file with 62 additions and 69 deletions.
131 changes: 62 additions & 69 deletions site/en/extending/repo.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@ Book: /_book.yaml

{% include "_buttons.html" %}

This page covers how to define repository rules and provides examples for
more details.
This page covers how to define repository rules and provides examples for more
details.

An [external repository](/external/overview#repository) is a directory tree,
containing source files usable in a Bazel build, which is generated on demand by
Expand Down Expand Up @@ -64,8 +64,8 @@ the canonical repo name.

## Implementation function

Every repo rule requires an `implementation` function. It contains the
actual logic of the rule and is executed strictly in the Loading Phase.
Every repo rule requires an `implementation` function. It contains the actual
logic of the rule and is executed strictly in the Loading Phase.

The function has exactly one input parameter, `repository_ctx`. The function
returns either `None` to signify that the rule is reproducible given the
Expand All @@ -75,11 +75,10 @@ example, for a rule tracking a git repository that would mean returning a
specific commit identifier instead of a floating branch that was originally
specified.

The input parameter `repository_ctx` can be used to
access attribute values, and non-hermetic functions (finding a binary,
executing a binary, creating a file in the repository or downloading a file
from the Internet). See [the API docs](/rules/lib/builtins/repository_ctx) for
more context. Example:
The input parameter `repository_ctx` can be used to access attribute values, and
non-hermetic functions (finding a binary, executing a binary, creating a file in
the repository or downloading a file from the Internet). See [the API
docs](/rules/lib/builtins/repository_ctx) for more context. Example:

```python
def _impl(repository_ctx):
Expand All @@ -92,78 +91,72 @@ local_repository = repository_rule(

## When is the implementation function executed?

The implementation function of a repo rule is executed when Bazel needs a
target from that repository, for example when another target (in another
repo) depends on it or if it is mentioned on the command line. The
implementation function is then expected to create the repo in the file
system. This is called "fetching" the repo.
The implementation function of a repo rule is executed when Bazel needs a target
from that repository, for example when another target (in another repo) depends
on it or if it is mentioned on the command line. The implementation function is
then expected to create the repo in the file system. This is called "fetching"
the repo.

In contrast to regular targets, repos are not necessarily re-fetched when
something changes that would cause the repo to be different. This is
because there are things that Bazel either cannot detect changes to or it would
cause too much overhead on every build (for example, things that are fetched
from the network). Therefore, repos are re-fetched only if one of the
following things changes:

* The attributes passed to the repo rule invocation.
* The Starlark code comprising the implementation of the repo rule.
* The value of any environment variable passed to `repository_ctx`'s
`getenv()` method or declared with the `environ` attribute of the
[`repository_rule`](/rules/lib/globals/bzl#repository_rule). The values
of these environment variables can be hard-wired on the command line with the
[`--repo_env`](/reference/command-line-reference#flag--repo_env) flag.
* The content of any file passed to the `read()`, `execute()` and similar
methods of `repository_ctx` which is referred to by a label (for example,
`//mypkg:label.txt` but not `mypkg/label.txt`)
* When `bazel fetch --force` is executed.
something changes that would cause the repo to be different. This is because
there are things that Bazel either cannot detect changes to or it would cause
too much overhead on every build (for example, things that are fetched from the
network). Therefore, repos are re-fetched only if one of the following things
changes:

* The attributes passed to the repo rule invocation.
* The Starlark code comprising the implementation of the repo rule.
* The value of any environment variable passed to `repository_ctx`'s
`getenv()` method or declared with the `environ` attribute of the
[`repository_rule`](/rules/lib/globals/bzl#repository_rule). The values of
these environment variables can be hard-wired on the command line with the
[`--repo_env`](/reference/command-line-reference#flag--repo_env) flag.
* The existence, contents, and type of any paths being
[`watch`ed](/rules/lib/builtins/repository_ctx#watch) in the implementation
function of the repo rule.
* Certain other methods of `repository_ctx` with a `watch` parameter, such
as `read()`, `execute()`, and `extract()`, can also cause paths to be
watched.
* Similarly, [`repository_ctx.watch_tree`](/rules/lib/builtins/repository_ctx#watch_tree)
and [`path.readdir`](/rules/lib/builtins/path#readdir) can cause paths
to be watched in other ways.
* When `bazel fetch --force` is executed.

There are two parameters of `repository_rule` that control when the repositories
are re-fetched:

* If the `configure` flag is set, the repository is only re-fetched on
`bazel fetch` when the` --configure` parameter is passed to it (if the
attribute is unset, this command will not cause a re-fetch)
* If the `local` flag is set, in addition to the above cases, the repo is
also re-fetched when the Bazel server restarts.

## Restarting the implementation function

The implementation function can be _restarted_ while a repo is being
fetched if a dependency it requests is _missing_. In that case, the execution of
the implementation function will stop, the missing dependency is resolved and
the function will be re-executed after the dependency has been resolved. To
avoid unnecessary restarts (which are expensive, as network access might
have to be repeated), label arguments are prefetched, provided all
label arguments can be resolved to an existing file. Note that resolving
a path from a string or a label that was constructed only during execution
of the function might still cause a restart.
* If the `configure` flag is set, the repository is only re-fetched on `bazel
fetch` when the` --configure` parameter is passed to it (if the attribute is
unset, this command will not cause a re-fetch)
* If the `local` flag is set, in addition to the above cases, the repo is also
re-fetched when the Bazel server restarts.

## Forcing refetch of external repos

Sometimes, an external repo can become outdated without any change to its
definition or dependencies. For example, a repo fetching sources might
follow a particular branch of a third-party repository, and new commits are
available on that branch. In this case, you can ask bazel to refetch all
external repos unconditionally by calling `bazel fetch --force --all`.

Moreover, some repo rules inspect the local machine and might become
outdated if the local machine was upgraded. Here you can ask Bazel to
only refetch those external repos where the
[`repository_rule`](/rules/lib/globals#repository_rule)
definition has the `configure` attribute set, use
`bazel fetch --all --configure`.
definition or dependencies. For example, a repo fetching sources might follow a
particular branch of a third-party repository, and new commits are available on
that branch. In this case, you can ask bazel to refetch all external repos
unconditionally by calling `bazel fetch --force --all`.

Moreover, some repo rules inspect the local machine and might become outdated if
the local machine was upgraded. Here you can ask Bazel to only refetch those
external repos where the [`repository_rule`](/rules/lib/globals#repository_rule)
definition has the `configure` attribute set, use `bazel fetch --all
--configure`.

## Examples

- [C++ auto-configured toolchain](https://cs.opensource.google/bazel/bazel/+/master:tools/cpp/cc_configure.bzl;drc=644b7d41748e09eff9e47cbab2be2263bb71f29a;l=176):
it uses a repo rule to automatically create the
C++ configuration files for Bazel by looking for the local C++ compiler, the
environment and the flags the C++ compiler supports.
- [C++ auto-configured
toolchain](https://cs.opensource.google/bazel/bazel/+/master:tools/cpp/cc_configure.bzl;drc=644b7d41748e09eff9e47cbab2be2263bb71f29a;l=176):
it uses a repo rule to automatically create the C++ configuration files for
Bazel by looking for the local C++ compiler, the environment and the flags
the C++ compiler supports.

- [Go repositories](https://github.com/bazelbuild/rules_go/blob/67bc217b6210a0922d76d252472b87e9a6118fdf/go/private/go_repositories.bzl#L195)
uses several `repository_rule` to defines the list of dependencies
needed to use the Go rules.
- [Go repositories](https://github.com/bazelbuild/rules_go/blob/67bc217b6210a0922d76d252472b87e9a6118fdf/go/private/go_repositories.bzl#L195)
uses several `repository_rule` to defines the list of dependencies needed to
use the Go rules.

- [rules_jvm_external](https://github.com/bazelbuild/rules_jvm_external) creates
an external repository called `@maven` by default that generates build targets
for every Maven artifact in the transitive dependency tree.
- [rules_jvm_external](https://github.com/bazelbuild/rules_jvm_external)
creates an external repository called `@maven` by default that generates
build targets for every Maven artifact in the transitive dependency tree.

0 comments on commit 1ab2c82

Please sign in to comment.