From 1ab2c824e54fd27896f14c5257ab737815463dc1 Mon Sep 17 00:00:00 2001 From: Googler Date: Wed, 28 Aug 2024 00:32:36 -0700 Subject: [PATCH] Update repo rule docs to mention `watch` and remove restarts The remarks about files being watched only if they're labels are no longer accurate. Repo rule restarts are gone by default (`--experimental_worker_for_repo_fetching` defaults to `auto`), so the section doesn't really apply anymore. Fixes https://github.com/bazelbuild/bazel/issues/22494 PiperOrigin-RevId: 668317824 Change-Id: Iba380b944cce2b5c8e7bd07606b89035ab401e72 --- site/en/extending/repo.md | 131 ++++++++++++++++++-------------------- 1 file changed, 62 insertions(+), 69 deletions(-) diff --git a/site/en/extending/repo.md b/site/en/extending/repo.md index b680e8a782ef6d..ab14227c989707 100644 --- a/site/en/extending/repo.md +++ b/site/en/extending/repo.md @@ -5,8 +5,8 @@ Book: /_book.yaml {% include "_buttons.html" %} -This page covers how to define repository rules and provides examples for -more details. +This page covers how to define repository rules and provides examples for more +details. An [external repository](/external/overview#repository) is a directory tree, containing source files usable in a Bazel build, which is generated on demand by @@ -64,8 +64,8 @@ the canonical repo name. ## Implementation function -Every repo rule requires an `implementation` function. It contains the -actual logic of the rule and is executed strictly in the Loading Phase. +Every repo rule requires an `implementation` function. It contains the actual +logic of the rule and is executed strictly in the Loading Phase. The function has exactly one input parameter, `repository_ctx`. The function returns either `None` to signify that the rule is reproducible given the @@ -75,11 +75,10 @@ example, for a rule tracking a git repository that would mean returning a specific commit identifier instead of a floating branch that was originally specified. -The input parameter `repository_ctx` can be used to -access attribute values, and non-hermetic functions (finding a binary, -executing a binary, creating a file in the repository or downloading a file -from the Internet). See [the API docs](/rules/lib/builtins/repository_ctx) for -more context. Example: +The input parameter `repository_ctx` can be used to access attribute values, and +non-hermetic functions (finding a binary, executing a binary, creating a file in +the repository or downloading a file from the Internet). See [the API +docs](/rules/lib/builtins/repository_ctx) for more context. Example: ```python def _impl(repository_ctx): @@ -92,78 +91,72 @@ local_repository = repository_rule( ## When is the implementation function executed? -The implementation function of a repo rule is executed when Bazel needs a -target from that repository, for example when another target (in another -repo) depends on it or if it is mentioned on the command line. The -implementation function is then expected to create the repo in the file -system. This is called "fetching" the repo. +The implementation function of a repo rule is executed when Bazel needs a target +from that repository, for example when another target (in another repo) depends +on it or if it is mentioned on the command line. The implementation function is +then expected to create the repo in the file system. This is called "fetching" +the repo. In contrast to regular targets, repos are not necessarily re-fetched when -something changes that would cause the repo to be different. This is -because there are things that Bazel either cannot detect changes to or it would -cause too much overhead on every build (for example, things that are fetched -from the network). Therefore, repos are re-fetched only if one of the -following things changes: - -* The attributes passed to the repo rule invocation. -* The Starlark code comprising the implementation of the repo rule. -* The value of any environment variable passed to `repository_ctx`'s - `getenv()` method or declared with the `environ` attribute of the - [`repository_rule`](/rules/lib/globals/bzl#repository_rule). The values - of these environment variables can be hard-wired on the command line with the - [`--repo_env`](/reference/command-line-reference#flag--repo_env) flag. -* The content of any file passed to the `read()`, `execute()` and similar - methods of `repository_ctx` which is referred to by a label (for example, - `//mypkg:label.txt` but not `mypkg/label.txt`) -* When `bazel fetch --force` is executed. +something changes that would cause the repo to be different. This is because +there are things that Bazel either cannot detect changes to or it would cause +too much overhead on every build (for example, things that are fetched from the +network). Therefore, repos are re-fetched only if one of the following things +changes: + +* The attributes passed to the repo rule invocation. +* The Starlark code comprising the implementation of the repo rule. +* The value of any environment variable passed to `repository_ctx`'s + `getenv()` method or declared with the `environ` attribute of the + [`repository_rule`](/rules/lib/globals/bzl#repository_rule). The values of + these environment variables can be hard-wired on the command line with the + [`--repo_env`](/reference/command-line-reference#flag--repo_env) flag. +* The existence, contents, and type of any paths being + [`watch`ed](/rules/lib/builtins/repository_ctx#watch) in the implementation + function of the repo rule. + * Certain other methods of `repository_ctx` with a `watch` parameter, such + as `read()`, `execute()`, and `extract()`, can also cause paths to be + watched. + * Similarly, [`repository_ctx.watch_tree`](/rules/lib/builtins/repository_ctx#watch_tree) + and [`path.readdir`](/rules/lib/builtins/path#readdir) can cause paths + to be watched in other ways. +* When `bazel fetch --force` is executed. There are two parameters of `repository_rule` that control when the repositories are re-fetched: -* If the `configure` flag is set, the repository is only re-fetched on - `bazel fetch` when the` --configure` parameter is passed to it (if the - attribute is unset, this command will not cause a re-fetch) -* If the `local` flag is set, in addition to the above cases, the repo is - also re-fetched when the Bazel server restarts. - -## Restarting the implementation function - -The implementation function can be _restarted_ while a repo is being -fetched if a dependency it requests is _missing_. In that case, the execution of -the implementation function will stop, the missing dependency is resolved and -the function will be re-executed after the dependency has been resolved. To -avoid unnecessary restarts (which are expensive, as network access might -have to be repeated), label arguments are prefetched, provided all -label arguments can be resolved to an existing file. Note that resolving -a path from a string or a label that was constructed only during execution -of the function might still cause a restart. +* If the `configure` flag is set, the repository is only re-fetched on `bazel + fetch` when the` --configure` parameter is passed to it (if the attribute is + unset, this command will not cause a re-fetch) +* If the `local` flag is set, in addition to the above cases, the repo is also + re-fetched when the Bazel server restarts. ## Forcing refetch of external repos Sometimes, an external repo can become outdated without any change to its -definition or dependencies. For example, a repo fetching sources might -follow a particular branch of a third-party repository, and new commits are -available on that branch. In this case, you can ask bazel to refetch all -external repos unconditionally by calling `bazel fetch --force --all`. - -Moreover, some repo rules inspect the local machine and might become -outdated if the local machine was upgraded. Here you can ask Bazel to -only refetch those external repos where the -[`repository_rule`](/rules/lib/globals#repository_rule) -definition has the `configure` attribute set, use -`bazel fetch --all --configure`. +definition or dependencies. For example, a repo fetching sources might follow a +particular branch of a third-party repository, and new commits are available on +that branch. In this case, you can ask bazel to refetch all external repos +unconditionally by calling `bazel fetch --force --all`. + +Moreover, some repo rules inspect the local machine and might become outdated if +the local machine was upgraded. Here you can ask Bazel to only refetch those +external repos where the [`repository_rule`](/rules/lib/globals#repository_rule) +definition has the `configure` attribute set, use `bazel fetch --all +--configure`. ## Examples -- [C++ auto-configured toolchain](https://cs.opensource.google/bazel/bazel/+/master:tools/cpp/cc_configure.bzl;drc=644b7d41748e09eff9e47cbab2be2263bb71f29a;l=176): - it uses a repo rule to automatically create the - C++ configuration files for Bazel by looking for the local C++ compiler, the - environment and the flags the C++ compiler supports. +- [C++ auto-configured + toolchain](https://cs.opensource.google/bazel/bazel/+/master:tools/cpp/cc_configure.bzl;drc=644b7d41748e09eff9e47cbab2be2263bb71f29a;l=176): + it uses a repo rule to automatically create the C++ configuration files for + Bazel by looking for the local C++ compiler, the environment and the flags + the C++ compiler supports. -- [Go repositories](https://github.com/bazelbuild/rules_go/blob/67bc217b6210a0922d76d252472b87e9a6118fdf/go/private/go_repositories.bzl#L195) - uses several `repository_rule` to defines the list of dependencies - needed to use the Go rules. +- [Go repositories](https://github.com/bazelbuild/rules_go/blob/67bc217b6210a0922d76d252472b87e9a6118fdf/go/private/go_repositories.bzl#L195) + uses several `repository_rule` to defines the list of dependencies needed to + use the Go rules. -- [rules_jvm_external](https://github.com/bazelbuild/rules_jvm_external) creates - an external repository called `@maven` by default that generates build targets - for every Maven artifact in the transitive dependency tree. +- [rules_jvm_external](https://github.com/bazelbuild/rules_jvm_external) + creates an external repository called `@maven` by default that generates + build targets for every Maven artifact in the transitive dependency tree. \ No newline at end of file