Add better way to do editable python install in project

[pavelzw]

Problem description

Related to #175

Currently, the only way to do an editable install of a python project with pixi is to have something like

[tasks]
postinstall = "pip install --no-build-isolation --no-deps --disable-pip-version-check -e ."

in your pixi.toml.

This is not really nice since your environment is not "ready to use" after pixi install but only after pixi install && pixi run postinstall.

[baszalmstra]

Maybe it would be nice to implement something similar to Lifecycle Scripts in yarn and npm.

[srilman]

+1 I was initially confused by the examples, since postinstall is the same name used in yarn/npm. Happy to contribute given a little advice!

[pavelzw]

Also being able to install wheels in some form using rip would be nice for my production use case:

I'm currently using this workaround (with the annoyance that pip is still in my prod environment):

[project]
name = "my-project"
channels = ["conda-forge"]
platforms = ["osx-arm64", "osx-64", "linux-64", "linux-aarch64"]

[dependencies]
python = ">=3.12"

[host-dependencies]
pip = "*"

[feature.test.dependencies]
pytest = ">=6"
[feature.test.tasks]
test = "pytest"

[feature.build.dependencies]
python-build = "*"
setuptools = ">=61"
setuptools-scm = "*"
[feature.build.tasks]
postinstall = "pip install --no-build-isolation --no-deps --disable-pip-version-check -e ."
build-wheel = "python -m build --no-isolation --wheel ."

[feature.prod.tasks]
postinstall-production = "pip install --no-deps --disable-pip-version-check dist/my_project-*.whl"

[environments]
default = ["test", "build"]
build = ["build"]
prod = ["prod"]

FROM ubuntu:22.04 AS build

RUN apt update && apt install -y curl git
RUN curl -Ls "https://github.com/prefix-dev/pixi/releases/latest/download/pixi-$(uname -m)-unknown-linux-musl" \
    -o /usr/local/bin/pixi \
    && chmod +x /usr/local/bin/pixi
COPY . /app
WORKDIR /app
RUN pixi run -e build build-wheel
RUN pixi install -e prod
RUN pixi run -e prod postinstall-production
RUN pixi shell-hook -e prod > /shell-hook
RUN echo "python -m my_project" >> /shell-hook

FROM ubuntu:22.04 AS production

COPY --from=build /app/.pixi/envs/prod /app/.pixi/envs/prod
COPY --from=build /shell-hook /shell-hook
WORKDIR /app
ENTRYPOINT ["sh", "/shell-hook"]

[ruben-arts]

@srilman This isn't really a simple issue, we're working hard on improving our rip library to support more features. But if you really want to help you should be following the work we're doing there!

[srilman]

@ruben-arts Thanks for the advice! Correct me if I'm wrong, but I don't see how rip is related? Assuming that pixi implements something similar to Lifecycle Scripts from npm or yarn, we could execute any arbitrary shell command, not just pip install .... So the workflow when running pixi install would be:

• Do all the current actions of pixi install
• If a postinstall task is defined, then run pixi run postinstall
• Otherwise, exit

[ruben-arts]

Ah sorry I though it was specifically a request for the pip install --no-build-isolation --no-deps --disable-pip-version-check -e . command. I also like bas' proposal for the lifecycle scripts. Feel free to give it a try.

We changed the logic so much in the last few PR's that I'm not sure what the best strategy is but reusing as much as you can from the src/cli/run.rs is probably important to keep it simple. @baszalmstra do you have some design tips?

For the commands I would start with only postinstall and see how we can implement this the best we can before we implement the logic in more places.

[Emsu]

For those of us needing editable installs, is the official recommendation to wait for uv integration?

I read this blog post and it seems like we should wait until the integration is complete and editable installs will work out of the box? https://prefix.dev/blog/uv_in_pixi

[baszalmstra]

Short answer: yes.

Longer answer, we are relatively close to a first version with uv integrated. This offers us all the features uv provides including editable installs.

However, some features including editable installs require a bit more work. We have to modify the lock file and the pixi.toml to accommodate this. But we'll get there!

[pavelzw]

xref #879

[srilman]

Opened a PR with a MVP working. Had a couple of open questions before opening it for review that I put in the PR. If anyone could take a look, that'd be helpful!

[ruben-arts]

I would like to close this feature in favor of #879

[MatusGasparik]

Also being able to install wheels in some form using rip would be nice for my production use case:

I'm currently using this workaround (with the annoyance that pip is still in my prod environment):

[project]
name = "my-project"
channels = ["conda-forge"]
platforms = ["osx-arm64", "osx-64", "linux-64", "linux-aarch64"]

[dependencies]
python = ">=3.12"

[host-dependencies]
pip = "*"

[feature.test.dependencies]
pytest = ">=6"
[feature.test.tasks]
test = "pytest"

[feature.build.dependencies]
python-build = "*"
setuptools = ">=61"
setuptools-scm = "*"
[feature.build.tasks]
postinstall = "pip install --no-build-isolation --no-deps --disable-pip-version-check -e ."
build-wheel = "python -m build --no-isolation --wheel ."

[feature.prod.tasks]
postinstall-production = "pip install --no-deps --disable-pip-version-check dist/my_project-*.whl"

[environments]
default = ["test", "build"]
build = ["build"]
prod = ["prod"]

Thanks, @pavelzw for this suggestion. One question though - how could you accomodate a following use case: I have a foo package withing a src folder which I want to build as a package. This is how I would set it up in pyproject.toml:

[tool.setuptools.packages.find]
where = ["src"]
include = ["foo"]

Is there a way to do this without having the pyproject.toml in addition to pixi.toml?

[pavelzw]

pixi.toml now (not released yet) also supports pyproject.toml. I would wait for the new release and then figure out best practices