Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix GaussianFilters dependency requirements #33273

Closed
wants to merge 1 commit into from

Conversation

radioflash
Copy link

GaussianFilters had a compatibility entry for its "Distributions" dependency that was NOT specified in the package itself,
but ONLY in G/GaussianFilters/Compat.toml.

AFAICT this had been automatically added.
This needs to be removed so that GaussianFilters can coexist e.g. with Distributions@0.24.15

On a sidenote:
This was incredibly frustrating to hunt down because GaussianFilters itself DOES not require any specific "Distributions" version.
It took forever to trace this back to the registry itself.

/rant
Dependency requirement management is just fundamentally broken (by concept) whenever packages use pre-1.0.0 versions, because it becomes impossible to express forward compatibility ("we need Distributions@v0.X, with X >= 23"), and thus requires ALL dependents to change with EVERY minor release of EVERY dependency (or to not specify dependency requirements at all).
I realise that this is how semantic versioning semantics are, but the resulting workflow is just broken.

GaussianFilters had a compatibility entry for its distributions dependency that was NOT specified in the package itself,
but ONLY in G/GaussianFilters/Compat.toml.

AFAICT this had been automatically added.
This needs to be removed so that GaussianFilters can coexist e.g. with Distributions@0.24.15
@StefanKarpinski
Copy link
Contributor

StefanKarpinski commented Mar 31, 2021

We don't generally encourage or allow unbounded compatibility. It doesn't make sense to say that GaussianFilters does not require any specific Distributions version: you can only say that for Distributions versions that have been released so far. If Distributions makes a breaking release tomorrow that breaks GaussianFilters then this will cause people to have a broken configuration. That said, Distributions is largely at fault here because it has not yet made a stable 1.0 release so every feature release is potentially broken according to semantic versioning, even if it doesn't break anything.

@StefanKarpinski
Copy link
Contributor

So the correct version of this change is to bump the upper bounds on these dependencies rather than deleting them.

@radioflash
Copy link
Author

radioflash commented Apr 1, 2021

I can see your point.

IMO there is only one proper fix: To make all libraries >v1.0.0 (this does not affect applications/non-dependencies).
This may sound drastic, and developers may be a bit hesitant to call something "1.0.0" when they are not yet fully satisfied with test coverage/documentation/"polish", but as soon as a package hits the registry, it has a de facto public API. And the semantic versioning specification clearly states:

Version 1.0.0 defines the public API

So this is indirectly required by SemVer already anyway.

This also makes a lot of sense regarding versioning semantics:
It is impossible to cleanly indicate "feature growth" (without breaking changes) in pre-1.0.0 versions, and this is where most of the trouble comes from: Currently many pre-1.0.0 libraries indicate feature growth by incrementing the minor version, but this is a liability for the whole ecosystem, because at that point, every minor release of the library requires an explicit update in ALL other modules that use it (even when the version increase just means "added features").

I also wanted to stress how very unpleasant the "hidden version requirements by the package registry" is:

  • This erodes trust in the whole package management infrastructure, because it soon becomes obvious that the package manager is betraying you (by enforcing hidden version constraints behind your back, that are completely invisible to a novice/intermediate user)
  • The resulting symptoms are very confusing and make it hard to blame anything but Julia core components, because all the dependencies work by themselves, and even ] dev <library> works fine (!!), because that circumvents the version constraints from the registry AFAICT.

TL;DR: My goal is to achieve >v1.0.0 for every public (registered) library in the Julia ecosystem (by convention).
Because libraries <v1.0.0 are a liability for the Julia ecosystem.

Easter holidays are coming; I'm going on a crusade.
My next steps:

  • Collect/create some docs to explain problem/solution (for library maintainers/Julia devs in general)
  • Preach to individual package maintainers about this
  • Get this into the language doc, preferably together with a strong encouragement to version libraries only >=1.0.0 on public release
  • Convince package registry to auto-enforce this in the future (bestcase)
  • ???
  • Jerusalem Profit

Sorry for the long rant; feedback/discussion is always very welcome.

Please don't read this in an accusatory tone-- I have just been bitten by this a few times already but I'm otherwise deliriously happy with Julia, its package system and the whole ecosystem in general (y'all did excellent work 👍); I'm just trying to help.

@StefanKarpinski
Copy link
Contributor

There was a longish discussion about this on the JuliaLang Slack yesterday if you care to check it out. Forcing everyone to go to 1.0 was discussed, but seems pretty disruptive. Urging packages to go to 1.0 is good, of course. If they won't do that, then they should use 0.x releases only for breaking changes and 0.x.y releases for features and bugfixes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants