Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Introduce an experimental uv toolchain #1989

Merged
merged 74 commits into from
Jul 12, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
74 commits
Select commit Hold shift + click to select a range
4a816ca
Spike uv
groodt Jun 18, 2024
6d0bdeb
Spike uv
groodt Jun 18, 2024
c5c3f88
Adds Windows
groodt Jun 18, 2024
f038ce8
buildifier
groodt Jun 18, 2024
8086333
buildifier
groodt Jun 18, 2024
2591363
chore: add uv to rules_python bzlmod
aignas May 6, 2024
2797a46
adapt the uv directly from http_archive
aignas Jun 19, 2024
308fe32
bump uv version
aignas Jun 19, 2024
04e5e6c
move the usage of the uv extension around
aignas Jun 19, 2024
0fcdf65
fixup the buildozer fixup command
aignas Jun 19, 2024
6d1743f
Merge branch 'main' into groodt-uv-hacks
groodt Jun 20, 2024
aea2ef9
Merge branch 'groodt-uv-hacks' of github.com:bazelbuild/rules_python …
groodt Jun 20, 2024
9b1f5fd
Hacky toolchain. Needs py interpreter.
groodt Jun 22, 2024
abbcc97
Hacky toolchain. Needs py interpreter.
groodt Jun 22, 2024
4a01495
Kinda works
groodt Jun 22, 2024
850f254
Merge branch 'main' into groodt-uv-hacks
groodt Jun 23, 2024
c6bddea
.
groodt Jun 23, 2024
750d8da
Init toolchain
groodt Jun 23, 2024
3a1f889
Init toolchain
groodt Jun 23, 2024
37859b8
Init toolchain
groodt Jun 23, 2024
0ff0003
Init toolchain
groodt Jun 23, 2024
fb73a9f
Init toolchain
groodt Jun 23, 2024
aefd672
Init toolchain
groodt Jun 23, 2024
8002c0e
Working toolchain
groodt Jun 24, 2024
1c8bf77
Working toolchain
groodt Jun 24, 2024
4356066
Working executable rule
groodt Jun 26, 2024
26a87f2
Working executable rule
groodt Jun 26, 2024
eea3f34
Working executable rule
groodt Jun 26, 2024
2cede41
Fix copyright date
rickeylev Jun 28, 2024
76b381a
address some nit/style type of comments
rickeylev Jun 28, 2024
c4b01c7
Merge branch 'main' into groodt-uv-hacks
groodt Jun 28, 2024
6041414
.
groodt Jun 28, 2024
ba09edc
Address review comments
groodt Jun 29, 2024
ec0f6fa
Address review comments
groodt Jun 29, 2024
4cb451a
Address review comments
groodt Jun 29, 2024
07c1ff6
Address review comments
groodt Jun 29, 2024
a60a308
Address review comments
groodt Jun 29, 2024
ec0f9f4
Address review comments
groodt Jun 29, 2024
014afbb
Address review comments
groodt Jun 29, 2024
e8fbfa1
Address review comments
groodt Jun 29, 2024
23628ea
Address review comments
groodt Jun 29, 2024
c2d7694
Address review comments
groodt Jun 29, 2024
fd9eebf
Address review comments
groodt Jun 29, 2024
a59ab24
Address review comments
groodt Jun 29, 2024
431c79f
Address review comments
groodt Jun 29, 2024
e67fdc8
Address review comments
groodt Jun 29, 2024
d141195
Address review comments
groodt Jun 29, 2024
3a0441b
Address review comments
groodt Jun 29, 2024
40ccfb0
Address review comments
groodt Jun 29, 2024
7750195
Address review comments
groodt Jun 29, 2024
8342931
Address review comments
groodt Jun 30, 2024
d98ba65
Address review comments
groodt Jun 30, 2024
dc0df0b
Address review comments
groodt Jun 30, 2024
4c8c490
Address review comments
groodt Jun 30, 2024
ae45048
Address review comments
groodt Jun 30, 2024
0b0d49c
Address review comments
groodt Jun 30, 2024
93e81c6
Address review comments
groodt Jun 30, 2024
5ecc345
Address review comments
groodt Jun 30, 2024
5733118
Address review comments
groodt Jun 30, 2024
ec10150
Address review comments
groodt Jul 1, 2024
6adac07
Merge branch 'main' into groodt-uv-hacks
groodt Jul 9, 2024
8e3cbe9
Bump to uv 0.2.23
groodt Jul 9, 2024
7dd729c
Address review comments
groodt Jul 9, 2024
bb2996c
Address review comments
groodt Jul 9, 2024
c31f6f9
Address review comments
groodt Jul 9, 2024
6f22655
Add EXPERIMENTAL comments and visibility restrictions
groodt Jul 9, 2024
22ad69f
Add EXPERIMENTAL comments and visibility restrictions
groodt Jul 9, 2024
f4c8790
Address review comments
groodt Jul 9, 2024
02c537d
Address review comments
groodt Jul 9, 2024
dfa24fe
Address review comments
groodt Jul 9, 2024
5c11f75
Address review comments
groodt Jul 9, 2024
93829ca
Address review comments
groodt Jul 10, 2024
a0f3d5c
Address review comments
groodt Jul 10, 2024
7fb7af1
Address review comments
groodt Jul 10, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions examples/bzlmod/MODULE.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,13 @@ python.toolchain(
# rules based on the `python_version` arg values.
use_repo(python, "python_3_10", "python_3_9", "python_versions")

# EXPERIMENTAL: This is experimental and may be removed without notice
uv = use_extension("@rules_python//python/uv:extensions.bzl", "uv")
uv.toolchain(uv_version = "0.2.23")
use_repo(uv, "uv_toolchains")

register_toolchains("@uv_toolchains//:all")

# This extension allows a user to create modifications to how rules_python
# creates different wheel repositories. Different attributes allow the user
# to modify the BUILD file, and copy files.
Expand Down
2 changes: 2 additions & 0 deletions python/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

"""This package contains two sets of rules:

1) the "core" Python rules, which were historically bundled with Bazel and
Expand Down Expand Up @@ -41,6 +42,7 @@ filegroup(
"//python/pip_install:distribution",
"//python/private:distribution",
"//python/runfiles:distribution",
"//python/uv:distribution",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure if having python/uv/private is better than python/private/uv but I don't we can discuss/bikeshed it later, we can merge it as is right now.

],
visibility = ["//:__pkg__"],
)
Expand Down
17 changes: 2 additions & 15 deletions python/private/python.bzl
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
load("@bazel_features//:features.bzl", "bazel_features")
load("//python:repositories.bzl", "python_register_toolchains")
load(":pythons_hub.bzl", "hub_repo")
load(":text_util.bzl", "render")
load(":toolchains_repo.bzl", "multi_toolchain_aliases")
load(":util.bzl", "IS_BAZEL_6_4_OR_HIGHER")

Expand All @@ -25,20 +26,6 @@ load(":util.bzl", "IS_BAZEL_6_4_OR_HIGHER")
_MAX_NUM_TOOLCHAINS = 9999
_TOOLCHAIN_INDEX_PAD_LENGTH = len(str(_MAX_NUM_TOOLCHAINS))

def _toolchain_prefix(index, name):
"""Prefixes the given name with the index, padded with zeros to ensure lexicographic sorting.

Examples:
_toolchain_prefix( 2, "foo") == "_0002_foo_"
_toolchain_prefix(2000, "foo") == "_2000_foo_"
"""
return "_{}_{}_".format(_left_pad_zero(index, _TOOLCHAIN_INDEX_PAD_LENGTH), name)

def _left_pad_zero(index, length):
if index < 0:
fail("index must be non-negative")
return ("0" * length + str(index))[-length:]

# Printing a warning msg not debugging, so we have to disable
# the buildifier check.
# buildifier: disable=print
Expand Down Expand Up @@ -202,7 +189,7 @@ def _python_impl(module_ctx):
name = "pythons_hub",
default_python_version = default_toolchain.python_version,
toolchain_prefixes = [
_toolchain_prefix(index, toolchain.name)
render.toolchain_prefix(index, toolchain.name, _TOOLCHAIN_INDEX_PAD_LENGTH)
for index, toolchain in enumerate(toolchains)
],
toolchain_python_versions = [t.python_version for t in toolchains],
Expand Down
16 changes: 16 additions & 0 deletions python/private/text_util.bzl
Original file line number Diff line number Diff line change
Expand Up @@ -99,11 +99,27 @@ def _render_tuple(items, *, value_repr = repr):
")",
])

def _toolchain_prefix(index, name, pad_length):
"""Prefixes the given name with the index, padded with zeros to ensure lexicographic sorting.

Examples:
toolchain_prefix( 2, "foo", 4) == "_0002_foo_"
toolchain_prefix(2000, "foo", 4) == "_2000_foo_"
"""
return "_{}_{}_".format(_left_pad_zero(index, pad_length), name)

def _left_pad_zero(index, length):
if index < 0:
fail("index must be non-negative")
return ("0" * length + str(index))[-length:]

render = struct(
alias = _render_alias,
dict = _render_dict,
indent = _indent,
left_pad_zero = _left_pad_zero,
list = _render_list,
select = _render_select,
tuple = _render_tuple,
toolchain_prefix = _toolchain_prefix,
)
78 changes: 78 additions & 0 deletions python/uv/BUILD.bazel
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
# Copyright 2024 The Bazel Authors. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# EXPERIMENTAL: This is experimental and may be removed without notice

load("@bazel_skylib//:bzl_library.bzl", "bzl_library")
load("//python/uv/private:current_toolchain.bzl", "current_toolchain")

package(default_visibility = ["//:__subpackages__"])

filegroup(
name = "distribution",
srcs = glob(["**"]) + [
"//python/uv/private:distribution",
],
visibility = ["//:__subpackages__"],
)

# For stardoc to reference the files
exports_files(["defs.bzl"])

toolchain_type(
name = "uv_toolchain_type",
visibility = ["//visibility:public"],
)

current_toolchain(
name = "current_toolchain",
# Marked manual so that `bazel test //...` passes
# even if no toolchain is registered.
tags = ["manual"],
# EXPERIMENTAL: Visibility is restricted to allow for changes.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you want to factor visibility into a variable? I just mention it in case you plan to patch-out the visibility restrictions to experiment with. It'd be easier to patch the variable than all the various visibility lines

visibility = ["@rules_python//examples:__subpackages__"],
)

bzl_library(
name = "defs",
srcs = ["defs.bzl"],
# EXPERIMENTAL: Visibility is restricted to allow for changes.
visibility = ["//:__subpackages__"],
)

bzl_library(
name = "extensions",
srcs = ["extensions.bzl"],
# EXPERIMENTAL: Visibility is restricted to allow for changes.
visibility = ["//:__subpackages__"],
deps = [":repositories"],
)

bzl_library(
name = "repositories",
srcs = ["repositories.bzl"],
# EXPERIMENTAL: Visibility is restricted to allow for changes.
visibility = ["//:__subpackages__"],
deps = [
"//python/uv/private:toolchains_repo",
"//python/uv/private:versions",
],
)

bzl_library(
name = "toolchain",
srcs = ["toolchain.bzl"],
# EXPERIMENTAL: Visibility is restricted to allow for changes.
visibility = ["//:__subpackages__"],
)
23 changes: 23 additions & 0 deletions python/uv/defs.bzl
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# Copyright 2024 The Bazel Authors. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

"""
EXPERIMENTAL: This is experimental and may be removed without notice

A toolchain for uv
"""

load("//python/uv/private:providers.bzl", _UvToolchainInfo = "UvToolchainInfo")

UvToolchainInfo = _UvToolchainInfo
50 changes: 50 additions & 0 deletions python/uv/extensions.bzl
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
# Copyright 2024 The Bazel Authors. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

"""
EXPERIMENTAL: This is experimental and may be removed without notice

A module extension for working with uv.
"""

load("//python/uv:repositories.bzl", "uv_register_toolchains")

_DOC = """\
A module extension for working with uv.
"""

uv_toolchain = tag_class(attrs = {
"uv_version": attr.string(doc = "Explicit version of uv.", mandatory = True),
})

def _uv_toolchain_extension(module_ctx):
for mod in module_ctx.modules:
for toolchain in mod.tags.toolchain:
if not mod.is_root:
fail(
"Only the root module may configure the uv toolchain.",
"This prevents conflicting registrations with any other modules.",
"NOTE: We may wish to enforce a policy where toolchain configuration is only allowed in the root module, or in rules_python. See https://github.com/bazelbuild/bazel/discussions/22024",
)

uv_register_toolchains(
uv_version = toolchain.uv_version,
register_toolchains = False,
)

uv = module_extension(
doc = _DOC,
implementation = _uv_toolchain_extension,
tag_classes = {"toolchain": uv_toolchain},
)
45 changes: 45 additions & 0 deletions python/uv/private/BUILD.bazel
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
# Copyright 2024 The Bazel Authors. All rights reserved.
groodt marked this conversation as resolved.
Show resolved Hide resolved
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

load("@bazel_skylib//:bzl_library.bzl", "bzl_library")

filegroup(
name = "distribution",
srcs = glob(["**"]),
visibility = ["//python/uv:__pkg__"],
)

bzl_library(
name = "current_toolchain",
srcs = ["current_toolchain.bzl"],
visibility = ["//python/uv:__subpackages__"],
)

bzl_library(
name = "toolchain_types",
srcs = ["toolchain_types.bzl"],
visibility = ["//python/uv:__subpackages__"],
)

bzl_library(
name = "toolchains_repo",
srcs = ["toolchains_repo.bzl"],
visibility = ["//python/uv:__subpackages__"],
)

bzl_library(
name = "versions",
srcs = ["versions.bzl"],
visibility = ["//python/uv:__subpackages__"],
)
56 changes: 56 additions & 0 deletions python/uv/private/current_toolchain.bzl
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
# Copyright 2024 The Bazel Authors. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

"""This module implements an alias rule to the resolved toolchain.
"""

load("//python/uv/private:toolchain_types.bzl", "UV_TOOLCHAIN_TYPE")

_DOC = """\
Exposes a concrete toolchain which is the result of Bazel resolving the
toolchain for the execution or target platform.
Workaround for https://github.com/bazelbuild/bazel/issues/14009
"""

# Forward all the providers
def _current_toolchain_impl(ctx):
toolchain_info = ctx.toolchains[UV_TOOLCHAIN_TYPE]

# Bazel requires executable rules to create the executable themselves,
# so we create a symlink in this rule so that it appears this rule created its executable.
original_uv_executable = toolchain_info.uv_toolchain_info.uv[DefaultInfo].files_to_run.executable
symlink_uv_executable = ctx.actions.declare_file("uv_symlink_{}".format(original_uv_executable.basename))
ctx.actions.symlink(output = symlink_uv_executable, target_file = original_uv_executable)

new_default_info = DefaultInfo(
files = depset([symlink_uv_executable]),
runfiles = toolchain_info.default_info.default_runfiles,
executable = symlink_uv_executable,
)

return [
toolchain_info,
new_default_info,
toolchain_info.template_variable_info,
toolchain_info.uv_toolchain_info,
]

# Copied from java_toolchain_alias
# https://cs.opensource.google/bazel/bazel/+/master:tools/jdk/java_toolchain_alias.bzl
current_toolchain = rule(
implementation = _current_toolchain_impl,
toolchains = [UV_TOOLCHAIN_TYPE],
doc = _DOC,
executable = True,
)
31 changes: 31 additions & 0 deletions python/uv/private/providers.bzl
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
# Copyright 2024 The Bazel Authors. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

"""This module contains the definitions of all providers."""

UvToolchainInfo = provider(
doc = "Information about how to invoke the uv executable.",
fields = {
"uv": """
:type: Target

The uv binary `Target`
""",
"version": """
:type: str

The uv version
""",
},
)
Loading