Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(bzlmod): Register a default toolchain #1259

Merged
merged 5 commits into from
Jun 12, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 0 additions & 14 deletions .github/workflows/create_archive_and_notes.sh
Original file line number Diff line number Diff line change
Expand Up @@ -40,20 +40,6 @@ pip.parse(
)

use_repo(pip, "pip")

# (Optional) Register a specific python toolchain instead of using the host version
python = use_extension("@rules_python//python:extensions.bzl", "python")

python.toolchain(
name = "python3_9",
python_version = "3.9",
)

use_repo(python, "python3_9_toolchains")

register_toolchains(
"@python3_9_toolchains//:all",
)
\`\`\`

## Using WORKSPACE
Expand Down
24 changes: 24 additions & 0 deletions BZLMOD_SUPPORT.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,3 +35,27 @@ This rule set does not have full feature partity with the older `WORKSPACE` type
2. Gazelle does not support finding deps in sub-modules. For instance we can have a dep like ` "@our_other_module//other_module/pkg:lib",` in a `py_test` definition.

Check ["issues"](/bazelbuild/rules_python/issues) for an up to date list.

## Differences in behavior from WORKSPACE

### Default toolchain is not the local system Python

Under bzlmod, the default toolchain is no longer based on the locally installed
system Python. Instead, a recent Python version using the pre-built,
standalone runtimes are used.

If you need the local system Python to be your toolchain, then it's suggested
that you setup and configure your own toolchain and register it. Note that using
the local system's Python is not advised because will vary between users and
platforms.

If you want to use the same toolchain as what WORKSPACE used, then manually
register the builtin Bazel Python toolchain by doing
`register_toolchains("@bazel_tools//tools/python:autodetecting_toolchain")`.
**IMPORTANT: this should only be done in a root module, and may intefere with
the toolchains rules_python registers**.

NOTE: Regardless of your toolchain, due to
[#691](https://github.com/bazelbuild/rules_python/issues/691), `rules_python`
still relies on a local Python being available to bootstrap the program before
handing over execution to the toolchain Python.
9 changes: 9 additions & 0 deletions MODULE.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,15 @@ use_repo(
# We need to do another use_extension call to expose the "pythons_hub"
# repo.
python = use_extension("@rules_python//python/extensions:python.bzl", "python")

# The default toolchain to use if nobody configures a toolchain.
# NOTE: This is not a stable version. It is provided for convenience, but will
# change frequently to track the most recent Python version.
# NOTE: The root module can override this.
python.toolchain(
is_default = True,
python_version = "3.11",
)
use_repo(python, "pythons_hub")

# This call registers the Python toolchains.
Expand Down
37 changes: 29 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,31 +45,52 @@ the older way of configuring bazel with a `WORKSPACE` file.

### Using bzlmod

NOTE: bzlmod support is still experimental; APIs subject to change.

To import rules_python in your project, you first need to add it to your
`MODULE.bazel` file, using the snippet provided in the
[release you choose](https://github.com/bazelbuild/rules_python/releases).

Once the dependency is added, a Python toolchain will be automatically
registered and you'll be able to create runnable programs and tests.


#### Toolchain registration with bzlmod

To register a hermetic Python toolchain rather than rely on a system-installed interpreter for runtime execution, you can add to the `MODULE.bazel` file:
NOTE: bzlmod support is still experimental; APIs subject to change.

A default toolchain is automatically configured for by depending on
`rules_python`. Note, however, the version used tracks the most recent Python
release and will change often.

If you want to register specific Python versions, then use
`python.toolchain()` for each version you need:

```starlark
# Find the latest version number here: https://github.com/bazelbuild/rules_python/releases
# and change the version number if needed in the line below.
bazel_dep(name = "rules_python", version = "0.21.0")
python = use_extension("@rules_python//python:extensions.bzl", "python")

python.toolchain(
python_version = "3.9",
)
```

### Using pip with bzlmod

NOTE: bzlmod support is still experimental; APIs subject to change.

To use dependencies from PyPI, the `pip.parse()` extension is used to
convert a requirements file into Bazel dependencies.

```starlark
python = use_extension("@rules_python//python/extensions:python.bzl", "python")
python.toolchain(
name = "python",
configure_coverage_tool = True,
is_default = True,
python_version = "3.9",
)

interpreter = use_extension("@rules_python//python/extensions:interpreter.bzl", "interpreter")
interpreter.install(
name = "interpreter",
python_name = "python",
python_name = "python_3_9",
)
use_repo(interpreter, "interpreter")

Expand Down
19 changes: 13 additions & 6 deletions python/extensions/python.bzl
Original file line number Diff line number Diff line change
Expand Up @@ -98,12 +98,20 @@ def _python_impl(module_ctx):
module_name = mod.name,
)

# Only the root module is allowed to set the default toolchain
# to prevent submodules from clobbering each other.
# A single toolchain in the root module is treated as the default
# because it's unambigiuous.
# Only the root module and rules_python are allowed to specify the default
# toolchain for a couple reasons:
# * It prevents submodules from specifying different defaults and only
# one of them winning.
# * rules_python needs to set a soft default in case the root module doesn't,
# e.g. if the root module doesn't use Python itself.
# * The root module is allowed to override the rules_python default.
if mod.is_root:
# A single toolchain is treated as the default because it's unambiguous.
is_default = toolchain_attr.is_default or len(mod.tags.toolchain) == 1
elif mod.name == "rules_python" and not default_toolchain:
# We don't do the len() check because we want the default that rules_python
# sets to be clearly visible.
is_default = toolchain_attr.is_default
else:
is_default = False

Expand All @@ -129,8 +137,7 @@ def _python_impl(module_ctx):
# A default toolchain is required so that the non-version-specific rules
# are able to match a toolchain.
if default_toolchain == None:
fail("No default toolchain found: exactly one toolchain must have " +
"is_default=True set")
fail("No default Python toolchain configured. Is rules_python missing `is_default=True`?")

# The last toolchain in the BUILD file is set as the default
# toolchain. We need the default last.
Expand Down