Remove extra duplication from lockfile #4888
Assignees
Labels
preview
Experimental behavior
Comments
|
Is this limited to the specific example shown here, or in general? In general, each of those dependency entries can have other fields on them that are different. For example, markers. In theory, I think, other fields could be different too, such as version and source. Or at least, that's what our data model supports. But I think the markers really could be different? |
|
This should only collapse entries with the same markers. Say we have dependencies = [
"pandas[excel,html]; python_version == '3.13'",
"pandas[excel,plot]",
]we currently get dependencies = [
{ name = "pandas" },
{ name = "pandas", extra = "excel" },
{ name = "pandas", extra = "html", marker = "python_version == '3.13'" },
{ name = "pandas", extra = "plot" },
]which we could collapse to dependencies = [
{ name = "pandas", extras = ["excel", "plot"] },
{ name = "pandas", extras = ["html"], marker = "python_version == '3.13'" },
] |
|
Right, okay. I think this LGTM. Although I don't totally mind our existing format, I am very sympathetic to the fact that it leaks our "virtual package" abstraction. So I think I would on balance favor your suggestion here to the status quo. |
This was referenced Jul 8, 2024
konstin
added a commit
that referenced
this issue
Jul 18, 2024
As user, you specify a list of extras. Internally, we decompose this into one virtual package per extra. We currently leak this abstraction by writing one entry per extra to the lockfile:
```toml
[[distribution]]
name = "foo"
version = "4.39.0.dev0"
source = { editable = "." }
dependencies = [
{ name = "pandas" },
{ name = "pandas", extra = "excel" },
{ name = "pandas", extra = "hdf5" },
{ name = "pandas", extra = "html", marker = "os_name != 'posix'" },
{ name = "pandas", extra = "output-formatting", marker = "os_name == 'posix'" },
{ name = "pandas", extra = "plot", marker = "os_name == 'posix'" },
]
```
Instead, we should merge the extras into a list of extras, creating a more concise lockfile:
```toml
[[distribution]]
name = "foo"
version = "4.39.0.dev0"
source = { editable = "." }
dependencies = [
{ name = "pandas", extra = ["excel", "hdf5"] },
{ name = "pandas", extra = ["html"], marker = "os_name != 'posix'" },
{ name = "pandas", extra = ["output-formatting", "plot"], marker = "os_name == 'posix'" },
]
```
Fixes #4888
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
When declaring a dependency
pandas[excel,html,plot], the lockfile shows:We should collapse those entries:
This makes more sense to the user (we don't install pandas four times, we install it once and then those extra packages, that's our internal abstraction into virtual packages leaking) and makes the lockfile more concise.
The text was updated successfully, but these errors were encountered: