-
Notifications
You must be signed in to change notification settings - Fork 615
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merge extras in lockfile #5181
Merge extras in lockfile #5181
Conversation
As user, you specify a list of extras. Internally, we decompose this into one virtual package per extra. We currently leak this abstraction by writing one entry per extra to the lockfile: ```toml [[distribution]] name = "foo" version = "4.39.0.dev0" source = { editable = "." } dependencies = [ { name = "pandas" }, { name = "pandas", extra = "excel" }, { name = "pandas", extra = "hdf5" }, { name = "pandas", extra = "html", marker = "os_name != 'posix'" }, { name = "pandas", extra = "output-formatting", marker = "os_name == 'posix'" }, { name = "pandas", extra = "plot", marker = "os_name == 'posix'" }, ] ``` Instead, we should merge the extras into a list of extras, creating a more concise lockfile: ```toml [[distribution]] name = "foo" version = "4.39.0.dev0" source = { editable = "." } dependencies = [ { name = "pandas", extra = ["excel", "hdf5"] }, { name = "pandas", extra = ["html"], marker = "os_name != 'posix'" }, { name = "pandas", extra = ["output-formatting", "plot"], marker = "os_name == 'posix'" }, ] ``` Fixes #4888
let new_dep = Dependency::from_annotated_dist(annotated_dist, marker); | ||
for existing_dep in &mut self.dependencies { | ||
if existing_dep.distribution_id == new_dep.distribution_id | ||
&& existing_dep.marker == new_dep.marker |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This makes add_dependency
quadratic, right?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I can't figure out a clean way around this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is only called from Lock::from_resolution_graph
and it's un-exported, so it should be fine to build up some intermediate state that replaces this loop with a O(1)
or O(logn)
check.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Totally agree. I briefly tried but had some trouble making it work.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As user, you specify a list of extras. Internally, we decompose this into one virtual package per extra. We currently leak this abstraction by writing one entry per extra to the lockfile:
Instead, we should merge the extras into a list of extras, creating a more concise lockfile:
The base package is now implicitly included, as it is in PEP 508.
Fixes #4888