Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Collapse redundant dependency clauses enumerating available versions #6160

Merged
merged 1 commit into from
Aug 19, 2024

Conversation

zanieb
Copy link
Member

@zanieb zanieb commented Aug 16, 2024

In #5046, we show the tautological proof:

  ╰─▶ Because colabfold[alphafold]==1.5.5 depends on jax>=0.4.20 and only the following versions of jax are available:
          jax<=0.4.20
          jax==0.4.21
          jax==0.4.22
          jax==0.4.23
          jax==0.4.24
          jax==0.4.25
          jax==0.4.26
          jax==0.4.27
          jax==0.4.28
          jax==0.4.29
          jax==0.4.30
          jax==0.4.31
      we can conclude that colabfold[alphafold]==1.5.5 depends on jax>=0.4.20.
      And because jax>=0.4.20 depends on numpy>=1.26.0, we can conclude that colabfold[alphafold]==1.5.5 depends on numpy>=1.26.0.
      (1)

This is a part of the error tree because the statement colabfold[alphafold]==1.5.5 depends on jax>=0.4.20 is actually a simplification of colabfold[alphafold]==1.5.5 depends on jax>=0.4.20,<0.5.0 and the no versions clause is a proof of that simplification.

Without simplification, the clause looks like:

  ╰─▶ Because colabfold[alphafold]==1.5.5 depends on jax>=0.4.20,<0.5.0 and only the following versions of jax are available:
          jax<=0.4.20
          jax==0.4.21
          jax==0.4.22
          jax==0.4.23
          jax==0.4.24
          jax==0.4.25
          jax==0.4.26
          jax==0.4.27
          jax==0.4.28
          jax==0.4.29
          jax==0.4.30
          jax==0.4.31
      we can conclude that colabfold[alphafold]==1.5.5 depends on one of:
          jax==0.4.20
          jax==0.4.21
          jax==0.4.22
          jax==0.4.23
          jax==0.4.24
          jax==0.4.25
          jax==0.4.26
          jax==0.4.27
          jax==0.4.28
          jax==0.4.29
          jax==0.4.30
          jax==0.4.31
      And because jax>=0.4.20 depends on numpy>=1.26.0, we can conclude that colabfold[alphafold]==1.5.5 depends on numpy>=1.26.0.

I don't think we have a great way to avoid performing the simplification of the range conditionally and it makes the error simpler to just jump straight to colabfold[alphafold]==1.5.5 depends on jax>=0.4.20.

The derivation for this clause looks like:

          jax==0.4.20 | ==0.4.21 | ==0.4.22 | ==0.4.23 | ==0.4.24 | ==0.4.25 | ==0.4.26 | ==0.4.27 | ==0.4.28 | ==0.4.29 | ==0.4.30 | ==0.4.31 depends on numpy>=1.26.0
            no versions of jax>0.4.20, <0.4.21 | >0.4.21, <0.4.22 | >0.4.22, <0.4.23 | >0.4.23, <0.4.24 | >0.4.24, <0.4.25 | >0.4.25, <0.4.26 | >0.4.26, <0.4.27 | >0.4.27, <0.4.28 | >0.4.28, <0.4.29 | >0.4.29, <0.4.30 | >0.4.30, <0.4.31 | >0.4.31, <0.5.0
            colabfold[alphafold]==1.5.5 depends on jax>=0.4.20, <0.5.0

So it looks like we can take trees of this form and drop the "no versions" clause if the ranges are compatible[*]. See this comment for a simpler explanation.

With this pull request, the clause simplifies to

╰─▶ Because colabfold[alphafold]==1.5.5 depends on jax>=0.4.20 and jax>=0.4.20 depends on numpy>=1.26.0,
     we can conclude that colabfold[alphafold]==1.5.5 depends on numpy>=1.26.0. (1)

Unfortunately, this doesn't change any snapshots in our test suite so I'm uncertain if the strategy generalizes. In some incorrect iterations of this logic, the snapshots did reveal my mistakes.

[*] "if the ranges are compatible" includes a bit of hand-waving. I'm not 100% sure if I've chosen the correct range heuristic here.

@zanieb zanieb added the error messages Messaging when something goes wrong label Aug 16, 2024
@zanieb zanieb changed the base branch from main to zb/simplify-unavailable August 16, 2024 18:12
@zanieb zanieb changed the title zb/collapse no versions redundant Collapse redundant resolver error clauses enumerating available versions Aug 16, 2024
@zanieb zanieb force-pushed the zb/collapse-no-versions-redundant branch 6 times, most recently from ea15e4e to 31230e1 Compare August 16, 2024 19:39
@zanieb zanieb marked this pull request as ready for review August 16, 2024 19:39
@zanieb zanieb changed the title Collapse redundant resolver error clauses enumerating available versions Collapse redundant dependency clauses enumerating available versions Aug 16, 2024
@zanieb
Copy link
Member Author

zanieb commented Aug 16, 2024

I think this is quite a bit like PubGrub's collapse_no_versions concept but much narrower.

Comment on lines 368 to 384
/// Given a [`DerivationTree`], collapse `NoVersions` incompatibilities that are redundant children
/// of a dependency. For example, if we have a tree like:
///
/// A>=1,<2 depends on B
/// A has no versions >1,<2
/// C depends on A>=1,<2
///
/// We can simplify this to `C depends A>=1 and A>=1 depends on B so C depends on B` without
/// explaining that there are no other versions of A. This is dependent on the first range of A
/// being a subset of the second. For example, in a tree like:
///
/// A>=1,<3 depends on B
/// A has no versions >2,<3
/// C depends on A>=2,<3
///
/// We cannot say `C depends on A>=2 and A>=1 depends on B so C depends on B` because there is a
/// hole in the range.
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This explainer might be easier to grok than the real-world example.

@zanieb zanieb force-pushed the zb/simplify-unavailable branch 2 times, most recently from e01d3fa to 4685604 Compare August 16, 2024 20:42
Base automatically changed from zb/simplify-unavailable to main August 16, 2024 20:56
@zanieb zanieb requested a review from charliermarsh August 16, 2024 21:05
@zanieb zanieb force-pushed the zb/collapse-no-versions-redundant branch from 31230e1 to 482bf17 Compare August 16, 2024 23:16
@zanieb zanieb requested a review from konstin August 19, 2024 14:29
crates/uv-resolver/src/error.rs Outdated Show resolved Hide resolved
@konstin
Copy link
Member

konstin commented Aug 19, 2024

Can you open an issue in pubgrub to make sure it's tracked upstream too?

@zanieb zanieb force-pushed the zb/collapse-no-versions-redundant branch from 482bf17 to de281ef Compare August 19, 2024 17:55
@zanieb
Copy link
Member Author

zanieb commented Aug 19, 2024

@konstin I think we need to come up with a test scenario first. I'll open an issue for that.

@zanieb zanieb enabled auto-merge (squash) August 19, 2024 17:56
@zanieb zanieb merged commit f6f2c5b into main Aug 19, 2024
56 checks passed
@zanieb zanieb deleted the zb/collapse-no-versions-redundant branch August 19, 2024 18:02
tmeijn pushed a commit to tmeijn/dotfiles that referenced this pull request Aug 21, 2024
This MR contains the following updates:

| Package | Update | Change |
|---|---|---|
| [astral-sh/uv](https://github.com/astral-sh/uv) | minor | `0.2.37` -> `0.3.0` |

MR created with the help of [el-capitano/tools/renovate-bot](https://gitlab.com/el-capitano/tools/renovate-bot).

**Proposed changes to behavior should be submitted there as MRs.**

---

### Release Notes

<details>
<summary>astral-sh/uv (astral-sh/uv)</summary>

### [`v0.3.0`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#030)

[Compare Source](astral-sh/uv@0.2.37...0.3.0)

This release introduces the uv [project](https://docs.astral.sh/uv/guides/projects/),
[tool](https://docs.astral.sh/uv/guides/tools/),
[script](https://docs.astral.sh/uv/guides/scripts/), and
[python](https://docs.astral.sh/uv/guides/install-python/) interfaces. If you've been following
uv's development, you've probably seen these new commands behind a preview flag. Now, the
interfaces are stable and ready for production-use.

These features are all documented in [new, comprehensive
documentation](https://docs.astral.sh/uv/).

This release also stabilizes preview functionality in `uv venv`:

-   `uv venv --python <version>` will [automatically
    download](https://docs.astral.sh/uv/concepts/python-versions/#requesting-a-version) the Python
    version if required
-   `uv venv` will read the required Python version from the `.python-version` file or
    `pyproject.toml`

The `uv pip` interface should not be affected by any breaking changes.

Note the following changelog entries does not include all the new features since they were added
incrementally as preview features. See the
[feature page](https://docs.astral.sh/uv/getting-started/features/) in the documentation for a
comprehensive listing, or read the [blog post](https://astral.sh/blog/uv-unified-python-packaging)
for more context on the new features.

##### Breaking changes

-   Migrate to XDG and Linux strategy for macOS directories ([#&#8203;5806](astral-sh/uv#5806))
-   Move concurrency settings to top-level ([#&#8203;4257](astral-sh/uv#4257))
-   Apply system Python filtering to executable name requests ([#&#8203;4309](astral-sh/uv#4309))
-   Remove `--legacy-setup-py` command-line argument ([#&#8203;4255](astral-sh/uv#4255))
-   Stabilize preview features ([#&#8203;6166](astral-sh/uv#6166))

##### Enhancements

-   Add 32-bit Windows target ([#&#8203;6252](astral-sh/uv#6252))
-   Add support for `python_version in ...` markers ([#&#8203;6172](astral-sh/uv#6172))
-   Allow user to constrain supported lock environments ([#&#8203;6210](astral-sh/uv#6210))
-   Lift requirement that .egg-info filenames must include version ([#&#8203;6179](astral-sh/uv#6179))
-   Change "any of" to "all of" in error messages ([#&#8203;6222](astral-sh/uv#6222))
-   Collapse redundant dependency clauses enumerating available versions ([#&#8203;6160](astral-sh/uv#6160))
-   Collapse unavailable packages in resolver errors ([#&#8203;6154](astral-sh/uv#6154))
-   Fix messages for unavailable packages when range is plural ([#&#8203;6221](astral-sh/uv#6221))
-   Improve resolver error messages when `--offline` is used ([#&#8203;6156](astral-sh/uv#6156))
-   Avoid overwriting dependencies with different markers in `uv add` ([#&#8203;6010](astral-sh/uv#6010))
-   Simplify available package version ranges when the name includes markers or extras ([#&#8203;6162](astral-sh/uv#6162))
-   Simplify version ranges reported for unavailable packages ([#&#8203;6155](astral-sh/uv#6155))
-   Rename `environment-markers` to `resolution-markers` ([#&#8203;6240](astral-sh/uv#6240))
-   Support `uv add -r requirements.txt` ([#&#8203;6005](astral-sh/uv#6005))

##### CLI

-   Hide global options in `uv generate-shell-completion` ([#&#8203;6170](astral-sh/uv#6170))
-   Show generate-shell-completion command in `uv help` ([#&#8203;6180](astral-sh/uv#6180))
-   Special-case reinstalls in environment update summaries ([#&#8203;6243](astral-sh/uv#6243))
-   Add output when `uv add` and `uv remove` update scripts ([#&#8203;6231](astral-sh/uv#6231))
-   Add support for `package@latest` in `tool run` ([#&#8203;6138](astral-sh/uv#6138))
-   Show `python find` output with `-q` ([#&#8203;6256](astral-sh/uv#6256))
-   Warn when `--upgrade` is passed to `tool run` ([#&#8203;6140](astral-sh/uv#6140))

##### Configuration

-   Allow customizing the tool install directory with `UV_TOOL_BIN_DIR` ([#&#8203;6207](astral-sh/uv#6207))

##### Performance

-   Use `FxHash` in `uv-auth` ([#&#8203;6149](astral-sh/uv#6149))

##### Bug fixes

-   Avoid panicking when the resolver thread encounters a closed channel ([#&#8203;6182](astral-sh/uv#6182))
-   Respect release-only semantics of `python_full_version` when constructing markers ([#&#8203;6171](astral-sh/uv#6171))
-   Tolerate missing `[project]` table in `uv venv` ([#&#8203;6178](astral-sh/uv#6178))
-   Avoid using workspace `lock_path` as relative root ([#&#8203;6157](astral-sh/uv#6157))

##### Documentation

-   Preview changes are now included in the standard changelog ([#&#8203;6259](astral-sh/uv#6259))
-   Document dynamic metadata behavior for cache ([#&#8203;5993](astral-sh/uv#5993))
-   Document the effect of ordering on package priority ([#&#8203;6211](astral-sh/uv#6211))
-   Make some edits to the workspace concept documentation ([#&#8203;6223](astral-sh/uv#6223))
-   Update environment variables doc ([#&#8203;5994](astral-sh/uv#5994))
-   Disable collapsible navigation in the documentation ([#&#8203;5674](astral-sh/uv#5674))
-   Document `uv add` and `uv remove` behavior with markers ([#&#8203;6163](astral-sh/uv#6163))
-   Document the Python installation directory ([#&#8203;6227](astral-sh/uv#6227))
-   Document the `uv.pip` section semantics ([#&#8203;6225](astral-sh/uv#6225))
-   Document the cache directory ([#&#8203;6229](astral-sh/uv#6229))
-   Document the tools directory ([#&#8203;6228](astral-sh/uv#6228))
-   Document yanked packages caveat during sync ([#&#8203;6219](astral-sh/uv#6219))
-   Link to persistent configuration options in Python versions document ([#&#8203;6226](astral-sh/uv#6226))
-   Link to the projects concept from the dependencies concept ([#&#8203;6224](astral-sh/uv#6224))
-   Improvements to the Docker installation guide ([#&#8203;6216](astral-sh/uv#6216))
-   Increase the size of navigation entries ([#&#8203;6233](astral-sh/uv#6233))
-   Install `ca-certificates` in docker and use pipefail ([#&#8203;6208](astral-sh/uv#6208))
-   Add script support to feature highlights in index ([#&#8203;6251](astral-sh/uv#6251))
-   Show `uv generate-shell-completion` in CLI documentation reference ([#&#8203;6146](astral-sh/uv#6146))
-   Update Docker guide for projects ([#&#8203;6217](astral-sh/uv#6217))
-   Use `uv add --script` in guide ([#&#8203;6215](astral-sh/uv#6215))
-   Show pinned version example on in GitHub Actions integration guide ([#&#8203;6234](astral-sh/uv#6234))

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever MR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this MR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this MR, check this box

---

This MR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy40NDAuNyIsInVwZGF0ZWRJblZlciI6IjM3LjQ0MC43IiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJSZW5vdmF0ZSBCb3QiXX0=-->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
error messages Messaging when something goes wrong
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants