Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pixi upgrade --dry-run fails with "cannot update pypi dependencies" #2770

Open
2 tasks done
lucascolley opened this issue Dec 26, 2024 · 3 comments · May be fixed by #2771
Open
2 tasks done

pixi upgrade --dry-run fails with "cannot update pypi dependencies" #2770

lucascolley opened this issue Dec 26, 2024 · 3 comments · May be fixed by #2771

Comments

@lucascolley
Copy link
Contributor

Checks

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pixi, using pixi --version.

Reproducible example

% pixi upgrade
# works fine
% pixi upgrade --dry-run

  × Cannot update pypi dependencies without first installing a conda prefix that includes python.
[project]
...
requires-python = ">=3.10"
...
dynamic = ["version"]
dependencies = ["array-api-compat>=1.10.0"]

[project.optional-dependencies]
tests = [
    "pytest >=6",
    "pytest-cov >=3",
    "array-api-strict",
    "numpy",
]
docs = [
    "sphinx>=7.0",
    "myst_parser>=0.13",
    "sphinx_copybutton",
    "sphinx_autodoc_typehints",
    "furo>=2023.08.17",
]
...
[tool.pixi.project]
channels = ["https://prefix.dev/conda-forge"]
platforms = ["linux-64", "osx-arm64", "win-64"]

[tool.pixi.dependencies]
python = ">=3.10.16,<3.14"
array-api-compat = ">=1.10.0,<2"

[tool.pixi.pypi-dependencies]
array-api-extra = { path = ".", editable = true }

[tool.pixi.feature.lint.dependencies]
pre-commit = "*"
pylint = "*"
basedmypy = "*"
basedpyright = "*"
numpydoc = ">=1.8.0,<2"
# import dependencies for mypy:
array-api-strict = "*"
numpy = "*"
pytest = "*"

[tool.pixi.feature.lint.tasks]
pre-commit-install = "pre-commit install"
pre-commit = "pre-commit run --all-files"
mypy = "mypy"
pylint = { cmd = "pylint array_api_extra", cwd = "src" }
pyright = "basedpyright"
lint = { depends-on = ["pre-commit", "pylint", "mypy", "pyright"] }

[tool.pixi.feature.tests.dependencies]
pytest = ">=6"
pytest-cov = ">=3"
array-api-strict = "*"
numpy = "*"

[tool.pixi.feature.tests.tasks]
tests = "pytest -v"
tests-cov = "pytest -v -ra --cov --cov-report=xml --cov-report=term --durations=20"

clean-vendor-compat = "rm -rf vendor_tests/array_api_compat"
clean-vendor-extra = "rm -rf vendor_tests/array_api_extra"
copy-vendor-compat = { cmd = "cp -r $(python -c 'import site; print(site.getsitepackages()[0])')/array_api_compat vendor_tests/", depends-on = ["clean-vendor-compat"] }
copy-vendor-extra = { cmd = "cp -r src/array_api_extra vendor_tests/", depends-on = ["clean-vendor-extra"] }
tests-vendor = { cmd = "pytest -v vendor_tests", depends-on = ["copy-vendor-compat", "copy-vendor-extra"] }

tests-ci = { depends-on = ["tests-cov", "tests-vendor"] }
coverage = { cmd = "coverage html", depends-on = ["tests-cov"] }
open-coverage = { cmd = "open htmlcov/index.html", depends-on = ["coverage"] }

[tool.pixi.feature.docs.dependencies]
sphinx = ">=7.0"
furo = ">=2023.08.17"
myst-parser = ">=0.13"
sphinx-copybutton = "*"
sphinx-autodoc-typehints = "*"

[tool.pixi.feature.docs.tasks]
docs = { cmd = "sphinx-build . build/", cwd = "docs" }
open-docs = { cmd = "open build/index.html", cwd = "docs", depends-on = ["docs"] }

[tool.pixi.feature.dev.dependencies]
ipython = "*"

[tool.pixi.feature.dev.tasks]
ipython = { cmd = "ipython" }

[tool.pixi.feature.py310.dependencies]
python = "~=3.10.0"

[tool.pixi.feature.py313.dependencies]
python = "~=3.13.0"

# Backends that can run on CPU-only hosts
[tool.pixi.feature.backends.target.linux-64.dependencies]
pytorch = "*"
dask = "*"
sparse = ">=0.15"
jax = "*"

[tool.pixi.feature.backends.target.osx-arm64.dependencies]
pytorch = "*"
dask = "*"
sparse = ">=0.15"
jax = "*"

[tool.pixi.feature.backends.target.win-64.dependencies]
# pytorch = "*"  # Package unavailable on Windows
dask = "*"
sparse = ">=0.15"
# jax = "*"  # Package unavailable on Windows

# Backends that require a GPU host and a CUDA driver
[tool.pixi.feature.cuda-backends.target.linux-64.dependencies]
cupy = "*"

[tool.pixi.feature.cuda-backends.target.osx-arm64.dependencies]
# cupy = "*"  # Package unavailable on macOSX

[tool.pixi.feature.cuda-backends.target.win-64.dependencies]
cupy = "*"

[tool.pixi.environments]
default = { solve-group = "default" }
lint = { features = ["lint"], solve-group = "default" }
tests = { features = ["tests"], solve-group = "default" }
docs = { features = ["docs"], solve-group = "default" }
dev = { features = ["lint", "tests", "docs", "dev"], solve-group = "default" }
ci-py310 = ["py310", "tests"]
ci-py313 = ["py313", "tests"]
# CUDA not available on free github actions
ci-backends = ["py310", "tests", "backends"]
tests-backends = ["py310", "tests", "backends", "cuda-backends"]

Issue description

dry run fails, but regular run succeeds

Expected behavior

behaviour should match the regular run

@lucascolley
Copy link
Contributor Author

I suppose this is related to

pixi/src/lock_file/update.rs

Lines 1164 to 1185 in a343eef

// Solving Pypi packages requires a python interpreter to be present in the
// prefix, therefore we first need to make sure we have conda packages
// available, then we can instantiate the prefix with at least the required
// conda packages (including a python interpreter) and then we can solve the
// Pypi packages using the installed interpreter.
//
// We only need to instantiate the prefix for the current platform.
for (environment, platforms) in self.outdated_envs.pypi.iter() {
// Only instantiate a prefix if any of the platforms actually contain pypi
// dependencies. If there are no pypi-dependencies than solving is also
// not required and thus a prefix is also not required.
if !platforms
.iter()
.any(|p| !environment.pypi_dependencies(Some(*p)).is_empty())
{
continue;
}
// If we are not allowed to install, we can't instantiate a prefix.
if self.no_install {
miette::bail!("Cannot update pypi dependencies without first installing a conda prefix that includes python.");
}

@lucascolley
Copy link
Contributor Author

I think this failure should be expected if one passes --no-install, but not if one passes --dry-run (at least that is what seems most intuitive to me)

@lucascolley
Copy link
Contributor Author

lucascolley commented Dec 27, 2024

PR at gh-2771, which avoids the error observed here, while keeping these assertions passing:

assert manifest_path.read_text() == manifest_content
assert lock_file_path.read_text() == lock_file_content

The one that doesn't pass is:

assert not os.path.exists(tmp_pixi_workspace / ".pixi")

So the question is just whether that is acceptable for a dry-run in this situation, I suppose?

Hofer-Julian added a commit to lucascolley/pixi that referenced this issue Dec 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant