feat(cli): introduce python management commands

[abn]

This is a highly experimental implementation of Python installation management features for Poetry. That said, more testers the merrier. Feedback around functionality and output ux are welcome. The intent of the PR, if made ready would be to introduce the feature as an experimental feature meeting the following basic requirements:

1. Use Python Standalone Builds with pbs-installer to provide Poetry users ability install/list/remove Python versions available.
2. Make such Python versions accessibly for Project virtual environment creation.

Depends-on: #10097 - While, this is not a hard dependency, having findpython in the mix definitely makes discovery easier. There will have to be changes once that PR is finalized.

Resolves: #9905

Testing Changes

Using pipx

pipx install --suffix=@10112 'poetry @ git+https://github.com/python-poetry/poetry.git@refs/pull/10112/head'
alias poetry=poetry@10112
poetry python list
poetry python list --managed
poetry python list --all
poetry python install 3.13.0
poetry python list
poetry python remove 3.13.0

Using a container (podman | docker)

podman run --rm -i --entrypoint bash python:latest <<EOF
set -xe
python -m pip install --disable-pip-version-check -q git+https://github.com/python-poetry/poetry.git@refs/pull/10112/head
poetry python list
poetry python list --managed
poetry python list --all
poetry python install 3.13.0
poetry python list
poetry python remove 3.13.0
EOF

Summary by Sourcery

Add commands to manage Python installations using standalone builds. Install, list, and remove Python versions using poetry python install, poetry python list, and poetry python remove.

New Features:

• Add experimental support for managing Python installations using standalone builds and pbs-installer.

Tests:

• Add tests for new Python installation management commands.

Summary by Sourcery

Add commands to manage Python installations using standalone builds. Install, list, and remove Python versions using the poetry python command group.

New Features:

• Add experimental support for managing Python installations using standalone builds and pbs-installer.

Tests:

• Add tests for new Python installation management commands.

[sourcery-ai]

Reviewer's Guide by Sourcery

This PR introduces experimental support for managing Python installations in Poetry via a new 'python' CLI namespace. The implementation leverages standalone Python builds with pbs-installer to allow users to install, list, and remove Python versions. The changes involve modifications to the environment manager, new CLI command implementations, enhancements to configuration management, tests, and documentation updates to support the new features.

Sequence diagram for Python Install Command

sequenceDiagram
  actor User
  participant PIC as "PythonInstallCommand"
  participant PI as "PythonInstaller"
  participant PBI as "pbs_installer"

  User->>PIC: Run 'poetry python install <version>'
  PIC->>PIC: Parse version & options
  PIC->>PI: Create PythonInstaller instance
  PI->>PBI: Call install() to download & install Python
  PBI-->>PI: Return installation status
  PI->>PIC: Report success or error
  PIC->>User: Display success/error messages

Loading

Class Diagram for Python Management Commands

classDiagram
    class Python {
      +find_all() : Iterator<Python>
      +find_all_versions(constraint, implementation) : Iterator<PythonInfo>
      +find_downloadable_versions(constraint) : Iterator<PythonInfo>
    }

    class PythonInfo {
      -major: int
      -minor: int
      -patch: int
      -implementation: string
      -executable: Path
    }

    class PythonInstaller {
      +request: str
      +implementation: str
      +free_threaded: bool
      +installation_directory: Path
      +exists() : bool
      +install() : None
      +version : Version
    }

    class PythonInstallCommand {
      +handle() : int
    }

    class PythonListCommand {
      +handle() : int
    }

    class PythonRemoveCommand {
      +handle() : int
      +remove_python_installation(request: str, implementation: str, io: IO) : int
    }

    class PoetryPythonPathProvider {
      +installation_dir(version: Version, implementation: string) : Path
      +installation_bin_paths(version: Version, implementation: string) : list~Path~
    }

    PythonInstallCommand --> PythonInstaller : uses
    PythonListCommand --> Python : accesses
    PythonRemoveCommand --> PoetryPythonPathProvider : uses

Loading

File-Level Changes

Change	Details	Files
Introduce new CLI commands for Python installation management.	Added 'python install' command to download and install Python versions. Added 'python list' command to display installed and available Python versions. Added 'python remove' command to remove Poetry-managed Python versions. Integrated new commands into the application command loader. Provided command descriptions and options (e.g., --clean, --free-threaded, --implementation, --reinstall).	src/poetry/console/commands/python/install.py src/poetry/console/commands/python/list.py src/poetry/console/commands/python/remove.py src/poetry/console/application.py
Enhance Python environment management implementation.	Extended the Python manager with methods for finding all pythons, Poetry-managed pythons, and downloadable versions. Introduced a PythonInfo NamedTuple for version details extraction. Integrated pbs-installer by importing constants like THIS_ARCH, THIS_PLATFORM and using PYTHON_VERSIONS for downloadable version discovery.	src/poetry/utils/env/python/manager.py src/poetry/utils/env/python/installer.py
Update configuration to support Python installations.	Added new configuration options for python installation directory ("python.installation-dir") and updated data-dir references. Enhanced the Config class with a python_installation_dir property. Updated tests and documentation to reflect the new configuration settings.	src/poetry/config/config.py docs/configuration.md tests/console/commands/test_config.py
Document and test the new Python management features.	Updated the CLI documentation to introduce the experimental 'python' command group with usage examples. Amended the configuration documentation for data-dir and python.installation-dir. Added comprehensive tests covering installation, listing and removal of Python versions, as well as handling of installer errors.	docs/cli.md pyproject.toml tests/console/commands/python/test_python_install.py tests/console/commands/python/test_python_list.py tests/console/commands/python/test_python_remove.py tests/utils/env/python/test_manager.py tests/utils/env/python/test_python_installer.py tests/utils/env/python/test_python_providers.py tests/types.py

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!
• Generate a plan of action for an issue: Comment @sourcery-ai plan on
  an issue to generate a plan of action for it.

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[jefer94]

In my opinion, poetry python list --all could be the default behavior of poetry python list and the last one could be poetry python list --installed or something like that

For poetry python install 3.13.1 is it covering the experimental death of GIL?

poetry install does not take into consideration the python versions installed with poetry python, it could install them in any directory and then create a venv using something like /??/python -m venv ??? right now poetry remember what was the last virtualenv installed

It seems good

[abn]

In my opinion, poetry python list --all could be the default behavior of poetry python list and the last one could be poetry python list --installed or something like that

Personally, I think that is a noisy default. I would really just want to know what is installed when I do poetry python list.

For poetry python install 3.13.1 is it covering the experimental death of GIL?

You will need to add the --free-threaded flag during install.

poetry install does not take into consideration the python versions installed with poetry python

In theory it should. But the "preference" is still the same "active python" -> "poetry python" -> "discovered python" (in this implementation the order is version based, no separate priority to managed by poetry versions). However once #10097 is finalised we can tackle that part better.

This said you can do

poetry python install 3.10.0
poetry env use 3.10.0
poetry install

And it should work.

right now poetry remember what was the last virtualenv installed

Yes, this will not change. As poetry python commands are not project specific. You have to use poetry env use command to change python version for a project's active env.

[github-actions]

Deploy preview for website ready!

✅ Preview
https://website-dabd8jkth-python-poetry.vercel.app

Built with commit b907d1f.
This pull request is being automatically deployed with vercel-action

[abn]

@sourcery-ai review

[sourcery-ai]

Hey @abn - I've reviewed your changes - here's some feedback:

Overall Comments:

• Consider adding more documentation about system requirements, limitations and potential issues users may encounter with this experimental feature, particularly around the Python Standalone Builds dependencies.

Here's what I looked at during the review

• 🟡 General issues: 1 issue found
• 🟢 Security: all looks good
• 🟢 Testing: all looks good
• 🟢 Complexity: all looks good
• 🟢 Documentation: all looks good

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

[sourcery-ai]

Hey @abn - I've reviewed your changes and found some issues that need to be addressed.

Blocking issues:

• Potential hardcoded secret (link)

Here's what I looked at during the review

• 🟡 General issues: 3 issues found
• 🔴 Security: 1 blocking issue
• 🟡 Testing: 1 issue found
• 🟢 Complexity: all looks good
• 🟢 Documentation: all looks good

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

[radoering]

As requested, I have taken over the PR. I made the changes in additional commits so it is easier to follow. (I think we can squash it in the end.)

I added tests for the major part of the implementation, fixed some minor bugs and changed two things:

• The location (data-dir vs. cache-dir) was inconsistently defined. I decided for cache-dir because it made sense to have Python installations next to virtualenvs.
• I changed the shortcut for --free-threaded from -f to -t because the executables or suffixed with t and -f felt like --force to me.

A real integration test, which installs a Python is still missing but I am fine without it, especially since the feature is experimental.

One thing still seems a bit strange to me (but that is something we can also change later). When listing the installed python versions, you will often get multiple lines per Python version (one for each executable), e.g.:

>poetrye python list
 Version Implementation Manager Path
 3.13.2  CPython        Poetry  C:/Users/Randy/AppData/Local/pypoetry/Cache/python/cpython@3.13.2/python.exe
 3.13.0  CPython        System  C:/Python313/python.exe
 3.12.2  CPython        System  C:/Python312/python.exe
 3.11.8  CPython        System  C:/Python311/python.exe
 3.10.14 PyPy           Poetry  C:/Users/Randy/AppData/Local/pypoetry/Cache/python/pypy@3.10.14/python3.10.exe
 3.10.14 PyPy           Poetry  C:/Users/Randy/AppData/Local/pypoetry/Cache/python/pypy@3.10.14/pypy3.10.exe
 3.10.14 PyPy           Poetry  C:/Users/Randy/AppData/Local/pypoetry/Cache/python/pypy@3.10.14/python3.exe
 3.10.14 PyPy           Poetry  C:/Users/Randy/AppData/Local/pypoetry/Cache/python/pypy@3.10.14/python.exe
 3.10.14 PyPy           Poetry  C:/Users/Randy/AppData/Local/pypoetry/Cache/python/pypy@3.10.14/pypy3.exe
 3.10.14 PyPy           Poetry  C:/Users/Randy/AppData/Local/pypoetry/Cache/python/pypy@3.10.14/pypy.exe

(This is just one installed PyPy version!)

[abn]

Thanks for taking this on @radoering

The data directory is what we should use I reckon. As cache is meant for non essentials. If someone wipes the cache directory, all in-project environments will get corrupted.

There is a single base directory relative to which user-specific non-essential (cached) data should be written.

https://specifications.freedesktop.org/basedir-spec/latest/

As for the duplicates, yeah that was something I was not sure if we should collapse or not. Because each install will have 2 symlinks and 1 binary. For example a 3.13 install will have the following symlinks.

1. python -> python3
2. python3 -> python3.13

I had a patch that checked if the symlink was at the same level, if so return that as the executable path. But the risk there is we might end up with some unexpected errors elsewhere. We also cannot simply use the resolved paths as this could cause issues for system managed installations.

We can defer this decision for later as well.

[radoering]

If someone wipes the cache directory, all in-project environments will get corrupted.

Good point. I will change it.

[radoering]

@sourcery-ai review

[sourcery-ai]

Hey @abn - I've reviewed your changes - here's some feedback:

Overall Comments:

• Extract duplicated config path logic (e.g. constructing data-dir/python) into a shared utility to reduce repetition across config, providers, and tests.
• Simplify the nested loops and set comprehensions in Python.find_downloadable_versions for improved readability.

Here's what I looked at during the review

• 🟡 General issues: 1 issue found
• 🟢 Security: all looks good
• 🟡 Testing: 1 issue found
• 🟢 Complexity: all looks good
• 🟢 Documentation: all looks good

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

[radoering]

@sourcery-ai review

[sourcery-ai]

Hey @abn - I've reviewed your changes - here's some feedback:

Overall Comments:

• Avoid writing empty error lines in the Python install command; remove or consolidate the self.io.write_error_line("") calls for clearer error output.
• Consider deferring provider registration from module import time to a dedicated initialization routine to prevent unexpected side effects during import.

Here's what I looked at during the review

• 🟡 General issues: 2 issues found
• 🟢 Security: all looks good
• 🟡 Testing: 1 issue found
• 🟢 Complexity: all looks good
• 🟢 Documentation: all looks good

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

[radoering]

@sourcery-ai review

[sourcery-ai]

Hey @abn - I've reviewed your changes - here's some feedback:

Overall Comments:

• Ensure that the use of Path.is_relative_to in find_all() is compatible with all supported Python versions, as it was introduced in Python 3.9.
• Refactor the nested set comprehension in find_downloadable_versions for better readability and to simplify the iteration logic.

Here's what I looked at during the review

• 🟡 General issues: 1 issue found
• 🟢 Security: all looks good
• 🟡 Testing: 1 issue found
• 🟢 Complexity: all looks good
• 🟢 Documentation: all looks good

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}