-
Notifications
You must be signed in to change notification settings - Fork 614
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Make some edits to the workspace concept documentation #6223
Merged
Merged
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,75 +1,173 @@ | ||
# Workspaces | ||
|
||
Workspaces help organize large codebases by splitting them into multiple packages with independent | ||
dependencies. Each package in a workspace has its own `pyproject.toml`, but they are all locked | ||
together in a shared lockfile and installed to shared virtual environment. | ||
Inspired by the [Cargo](https://doc.rust-lang.org/cargo/reference/workspaces.html) concept of the | ||
same name, a workspace is "a collection of one or more packages, called _workspace members_, that | ||
are managed together." | ||
|
||
Using the project interface, `uv run` and `uv sync` will install all packages of the workspace, | ||
unless you select a single workspace member with `--package`. When using the `uv pip` interface, | ||
workspace dependencies behave like editable path dependencies. | ||
Workspaces organize large codebases by splitting them into multiple packages with common | ||
dependencies. Think: a FastAPI-based web application, alongside a series of libraries that are | ||
versioned and maintained as separate Python packages, all in the same Git repository. | ||
|
||
## When (not) to use workspaces | ||
|
||
One common use case for a workspace is that the codebase grows large, and eventually you want some | ||
modules to become independent packages with their own dependency specification. Other use cases are | ||
separating parts of the codebase with different responsibilities, e.g. in a repository with a | ||
library package and CLI package, where the CLI package makes features of the library available but | ||
has additional dependencies, a webserver with a backend and an ingestion package, or a library that | ||
has a performance-critical subroutine implemented in a native language. | ||
In a workspace, each package defines its own `pyproject.toml`, but the workspace shares a single | ||
lockfile, ensuring that the workspace operates with a consistent set of dependencies. | ||
|
||
Workspaces are not suited when you don't want to install all members together, members have | ||
conflicting requirements, or you simply want individual virtual environments per project. In this | ||
case, use regular (editable) relative path dependencies. | ||
As such, `uv lock` operates on the entire workspace at once, while `uv run` and `uv sync` operate on | ||
the workspace root by default, though both accept a `--package` argument, allowing you to run a | ||
command in a particular workspace member from any workspace directory. | ||
|
||
Currently, workspace don't properly support different members having different `requires-python` | ||
values, we apply the highest of all `requires-python` lower bounds to the entire workspace. You need | ||
to use a `uv pip` to install individual member in an older virtual environment. | ||
## Getting started | ||
|
||
!!! note | ||
To create a workspace, add a `tool.uv.workspace` table to a `pyproject.toml`, which will implicitly | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We could also suggest in a !!! tip that |
||
create a workspace rooted at that package. | ||
|
||
As Python does not provide dependency isolation, uv can't ensure that a package uses only the dependencies it has declared, and not also imports a package that was installed for another dependency. For workspaces specifically, uv can't ensure that packages don't import dependencies declared by another workspace member. | ||
!!! tip | ||
|
||
## Usage | ||
By default, running `uv init` inside an existing package will add the newly created member to the workspace, creating a `tool.uv.workspace` table in the workspace root if it doesn't already exist. | ||
|
||
A workspace can be created by adding a `tool.uv.workspace` table to a `pyproject.toml` that will | ||
become the workspace root. This table contains `members` (mandatory) and `exclude` (optional), with | ||
lists of globs of directories: | ||
In defining a workspace, you must specify the `members` (required) and `exclude` (optional) keys, | ||
which direct the workspace to include or exclude specific directories as members respectively, and | ||
accept lists of globs: | ||
|
||
```toml title="pyproject.toml" | ||
[tool.uv.workspace] | ||
members = ["packages/*", "examples/*"] | ||
exclude = ["example/excluded_example"] | ||
``` | ||
|
||
`uv.lock` and `.venv` for the entire workspace are created next to this `pyproject.toml`. All | ||
members need to be in directories below it. | ||
In this example, the workspace includes all packages in the `packages` directory and all examples in | ||
the `examples` directory, with the exception of the `example/excluded_example` directory. | ||
|
||
Every directory included by the `members` globs (and not excluded by the `exclude` globs) must | ||
contain a `pyproject.toml` file; in other words, every member must be a valid Python package, or | ||
workspace discovery will raise an error. | ||
|
||
## Workspace roots | ||
|
||
Every workspace needs a workspace root, which can either be explicit or "virtual". | ||
|
||
An explicit root is a directory that is itself a valid Python package, and thus a valid workspace | ||
member, as in: | ||
|
||
```toml title="pyproject.toml" | ||
[project] | ||
name = "albatross" | ||
version = "0.1.0" | ||
requires-python = ">=3.12" | ||
dependencies = ["bird-feeder", "tqdm>=4,<5"] | ||
|
||
[tool.uv.sources] | ||
bird-feeder = { workspace = true } | ||
|
||
[tool.uv.workspace] | ||
members = ["packages/*"] | ||
|
||
[build-system] | ||
requires = ["hatchling"] | ||
build-backend = "hatchling.build" | ||
``` | ||
|
||
A virtual root is a directory that is _not_ a valid Python package, but contains a `pyproject.toml` | ||
with a `tool.uv.workspace` table. In other words, the `pyproject.toml` exists to define the | ||
workspace, but does not itself define a package, as in: | ||
|
||
```toml title="pyproject.toml" | ||
[tool.uv.workspace] | ||
members = ["packages/*"] | ||
``` | ||
|
||
A virtual root _must not_ contain a `[project]` table, as the inclusion of a `[project]` table | ||
implies the directory is a package, and thus an explicit root. As such, virtual roots cannot define | ||
their own dependencies; however, they _can_ define development dependencies as in: | ||
|
||
```toml title="pyproject.toml" | ||
[tool.uv.workspace] | ||
members = ["packages/*"] | ||
|
||
[tool.uv] | ||
dev-dependencies = ["ruff==0.5.0"] | ||
``` | ||
|
||
By default, `uv run` and `uv sync` operates on the workspace root, if it's explicit. For example, in | ||
the above example, `uv run` and `uv run --package albatross` would be equivalent. For virtual | ||
workspaces, `uv run` and `uv sync` instead sync all workspace members, since the root is not a | ||
member itself. | ||
|
||
## Workspace sources | ||
|
||
Within a workspace, dependencies on workspace members are facilitated via | ||
[`tool.uv.sources`](./dependencies.md), as in: | ||
|
||
```toml title="pyproject.toml" | ||
[project] | ||
name = "albatross" | ||
version = "0.1.0" | ||
requires-python = ">=3.12" | ||
dependencies = ["bird-feeder", "tqdm>=4,<5"] | ||
|
||
[tool.uv.sources] | ||
bird-feeder = { workspace = true } | ||
|
||
[tool.uv.workspace] | ||
members = ["packages/*"] | ||
|
||
[build-system] | ||
requires = ["hatchling"] | ||
build-backend = "hatchling.build" | ||
``` | ||
|
||
In this example, the `albatross` package depends on the `bird-feeder` package, which is a member of | ||
the workspace. The `workspace = true` key-value pair in the `tool.uv.sources` table indicates the | ||
`bird-feeder` dependency should be provided by the workspace, rather than fetched from PyPI or | ||
another registry. | ||
|
||
Any `tool.uv.sources` definitions in the workspace root apply to all members, unless overridden in | ||
the `tool.uv.sources` of a specific member. For example, given the following `pyproject.toml`: | ||
|
||
```toml title="pyproject.toml" | ||
[project] | ||
name = "albatross" | ||
version = "0.1.0" | ||
requires-python = ">=3.12" | ||
dependencies = ["bird-feeder", "tqdm>=4,<5"] | ||
|
||
[tool.uv.sources] | ||
bird-feeder = { workspace = true } | ||
tqdm = { git = "https://github.com/tqdm/tqdm" } | ||
|
||
[tool.uv.workspace] | ||
members = ["packages/*"] | ||
|
||
If `tool.uv.sources` is defined in the workspace root, it applies to all members, unless overridden | ||
in the `tool.uv.sources` of a specific member. | ||
[build-system] | ||
requires = ["hatchling"] | ||
build-backend = "hatchling.build" | ||
``` | ||
|
||
Using `uv init` inside a workspace will add the newly created package to `members`. | ||
Every workspace member would, by default, install `tqdm` from GitHub, unless a specific member | ||
overrides the `tqdm` entry in its own `tool.uv.sources` table. | ||
|
||
## Common structures | ||
## Workspace layouts | ||
|
||
There a two main workspace structures: A **root package with helpers** and a **flat workspace**. | ||
In general, there are two common layouts for workspaces, which map to the two kinds of workspace | ||
roots: a **root package with helpers** (for explicit roots) and a **flat workspace** (for virtual | ||
roots). | ||
|
||
The root workspace layout defines one main package in the root of the repository, with helper | ||
packages in `packages`. In this example `albatross/pyproject.toml` has both a `project` section and | ||
a `tool.uv.workspace` section. | ||
In the former case, the workspace includes an explicit workspace root, with peripheral packages or | ||
libraries defined in `packages`. For example, here, `albatross` is an explicit workspace root, and | ||
`bird-feeder` and `seeds` are workspace members: | ||
|
||
```text | ||
albatross | ||
├── packages | ||
│ ├── provider_a | ||
│ ├── bird-feeder | ||
│ │ ├── pyproject.toml | ||
│ │ └── src | ||
│ │ └── provider_a | ||
│ │ └── bird_feeder | ||
│ │ ├── __init__.py | ||
│ │ └── foo.py | ||
│ └── provider_b | ||
│ └── seeds | ||
│ ├── pyproject.toml | ||
│ └── src | ||
│ └── provider_b | ||
│ └── seeds | ||
│ ├── __init__.py | ||
│ └── bar.py | ||
├── pyproject.toml | ||
|
@@ -80,9 +178,8 @@ albatross | |
└── main.py | ||
``` | ||
|
||
In the flat layout, all packages are in the `packages` directory, and the root `pyproject.toml` | ||
defines a so-called virtual workspace. In this example `albatross/pyproject.toml` has only a | ||
`tool.uv.workspace` section, but no `project`. | ||
In the latter case, _all_ members are located in the `packages` directory, and the root | ||
`pyproject.toml` comprises a virtual root: | ||
|
||
```text | ||
albatross | ||
|
@@ -93,32 +190,70 @@ albatross | |
│ │ └── albatross | ||
│ │ ├── __init__.py | ||
│ │ └── foo.py | ||
│ ├── provider_a | ||
│ ├── bird-feeder | ||
│ │ ├── pyproject.toml | ||
│ │ └── src | ||
│ │ └── provider_a | ||
│ │ └── bird_feeder | ||
│ │ ├── __init__.py | ||
│ │ └── foo.py | ||
│ └── provider_b | ||
│ └── seeds | ||
│ ├── pyproject.toml | ||
│ └── src | ||
│ └── provider_b | ||
│ └── seeds | ||
│ ├── __init__.py | ||
│ └── bar.py | ||
├── pyproject.toml | ||
├── README.md | ||
└── uv.lock | ||
``` | ||
|
||
In the flat layout, you may still define development dependencies in the workspace root | ||
`pyproject.toml`: | ||
## When (not) to use workspaces | ||
|
||
Workspaces are intended to facilitate the development of multiple interconnected packages within a | ||
single repository. As a codebase grows in complexity, it can be helpful to split it into smaller, | ||
composable packages, each with their own dependencies and version constraints. | ||
|
||
Workspaces help enforce isolation and separation of concerns. For example, in uv, we have separate | ||
packages for the core library and the command-line interface, enabling us to test the core library | ||
independently of the CLI, and vice versa. | ||
|
||
Other common use cases for workspaces include: | ||
|
||
- A library with a performance-critical subroutine implemented in an extension module (Rust, C++, | ||
etc.). | ||
- A library with a plugin system, where each plugin is a separate workspace package with a | ||
dependency on the root. | ||
|
||
Workspaces are _not_ suited for cases in which members have conflicting requirements, or desire a | ||
separate virtual environment for each member. In this case, path dependencies are often preferable. | ||
For example, rather than grouping `albatross` and its members in a workspace, you can always define | ||
each package as its own independent project, with inter-package dependencies defined as path | ||
dependencies in `tool.uv.sources`: | ||
|
||
```toml title="pyproject.toml" | ||
[tool.uv.workspace] | ||
members = ["packages/*"] | ||
[project] | ||
name = "albatross" | ||
version = "0.1.0" | ||
requires-python = ">=3.12" | ||
dependencies = ["bird-feeder", "tqdm>=4,<5"] | ||
|
||
[tool.uv] | ||
dev-dependencies = [ | ||
"pytest >=8.3.2,<9" | ||
] | ||
[tool.uv.sources] | ||
bird-feeder = { path = "packages/bird-feeder" } | ||
|
||
[build-system] | ||
requires = ["hatchling"] | ||
build-backend = "hatchling.build" | ||
``` | ||
|
||
This approach conveys many of the same benefits, but allows for more fine-grained control over | ||
dependency resolution and virtual environment management (with the downside that `uv run --package` | ||
is no longer available; instead, commands must be run from the relevant package directory). | ||
|
||
Finally, uv's workspaces enforce a single `requires-python` for the entire workspace, taking the | ||
intersection of all members' `requires-python` values. If you need to support testing a given member | ||
on a Python version that isn't supported by the rest of the workspace, you may need to use `uv pip` | ||
to install that member in a separate virtual environment. | ||
|
||
!!! note | ||
|
||
As Python does not provide dependency isolation, uv can't ensure that a package uses its declared dependencies and nothing else. For workspaces specifically, uv can't ensure that packages don't import dependencies declared by another workspace member. |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This seems vaguely conflicting with the above
Maybe we should say "independent requirements" above? or "consistent set of dependency versions" here? I'm not sure.