rules_python bzlmod should register a default toolchain

[rickeylev]

🚀 feature request

Well, sort of a feature-parity / bug report, too; see below.

Relevant Rules

bzlmod module

Description

Today, users must manually specify a toolchain and manually register it. This adds friction and causes problems for a few reasons:

1. User configs are more verbose. They end up having to specify a version and toolchain to use.
2. What they specify above is somewhat misleading and can lead to surprising behavior because another module can specify a different toolchain and version, and it might be used instead (it depends on the order of register_toolchain calls)
3. It's just a general barrier to entry; configuring the toolchain is extra steps that get in the way to just getting started. I, myself, have put off trying out various ideas and bug fixes because setting up a MODULE with all the boiler plate is a chore.

It's also worth noting that WORKSPACE builds already register a default toolchain -- whatever is found on the local system (@bazel_tools//tools/python:autodetecting_toolchain). This toolchain is problematic for various reasons, but that is the existing WORKSPACE-based behavior.

Describe the solution you'd like

Instead, rules_python should register a toolchain by default. This avoids the three issues above.

I propose that it register the most recent version of Python available. We should be clear that this is for convenience -- the default provided shouldn't be relied upon.

If a module needs a specific version, then it should use the multi-version rules.

[chrislovecnm]

@rickeylev and I have discussed this a bunch, we are uncertain if we are going to do this. We need to have the rules register a toolchain, and usually not the user. But autoregistering may cause issues.

[Clement-Jean]

This could also avoid problems like the following: [Bug]: rules_python@0.21.0 'python/extensions' is not a package. Because the rules_python repo changed faster than the bzlmod registry it seems we can't use the last bzlmod dep. Less config in the MODULE.bazel, the less problems we will have for updating rules_python.

[rickeylev]

I came back to this issues today when thinking about what happens if the root module doesn't use Python, but a submodule does. For example, if I'm a JavaScript project and don't use Python myself, I shouldn't have to setup Python because a module I depend on happens to use Python.

My basic conclusion is rules_python should register a default toolchain. The root module can, of course, still override it.

Sub-modules still aren't allowed to establish a default, for the same reasons as before. That, plus I think it's easier to understand and manage "rules python sets the default" instead of "the first module that tries to use python in my module dependency graph sets the default". With the former, you just know exactly where the default is being set. If there's a problem, it's "Module A doesn't support Python version X", which is fairly straight forward. With the latter, you have to try and figure out where in your transitive module graph it might be coming from. Once you find it, the problem is "Module A wants Python version X, while module B wants Python version Y, and this combination isn't compatible", which is harder to solve.

[rickeylev]

Another thing that has occurred to me: there's an interaction with pip here, too. A module calling pip.parse() needs some way to refer to the default python version so that its pip resolution uses the same python version as what its programs will end up selecting through the toolchain.

So we'll need something like a way to make interpreter.install() able to refer to the default interpreter, or a way to make pip.parse() auto-select the default python interpreter. The latter sounds a bit more appealing.

[chrislovecnm]

@rickeylev is this where you want now?

[rickeylev]

We're close (requested toolchains are auto-registered), but rules_python isn't setting a default.

[chrislovecnm]

@rickeylev Can we close this now?

[rickeylev]

Yes.