new dependency for linear-1.18.3 broke our lts-based builds

[mwu-tow]

Hi,
recently old releases of linear on Hackage were updated and dependency on base-orphans was introduced. This is bad, as it breaks stack's lts and our builds.
Why does almost one-year old package release suddenly has its dependencies changed?

We use linear-1.18.3, as part of lts-2.22. See http://hackage.haskell.org/package/linear-1.18.3/revisions/ for the problematic revision.

[kgadek]

I'd want to further discuss the benefits of using cabal-revisions (a.k.a. metadata-revisions), because as of now I see none. Usage of any sane versioning scheme - most notably semver - seems to overlap with that mechanism, but also [seems] to be strictly better than cabal-revisions.

[bergey]

The key to Hackage metadata revisions is that cabal will only ever pick the latest available revision of a given version. cabal is perfectly willing to pick 1.8.0.5 when 1.8.0.6 exists, so a 4th- or 5th-number release that tightens bounds doesn't actually prevent cabal from attempting known-broken combinations. We could probably get the same effect by deprecating 1.8.0.5 at the same time as releasing 1.8.0.6. I suspect that would cause more pain overall.

In practice, Hackage trustees also seem more comfortable making metadata edits than new numbered releases. That's what happened here.

[glguy]

@hvr ping.

@mwu-tow can you include the resulting error message?

[bergmark]

The problem occurs with stack because the added package is not in the lts-2.22 snapshot. You can work around this by specifying the package on the command line or adding it as an extra-dep.

$ stack build linear-1.18.3 --resolver lts-2.22
Run from outside a project, using implicit global project config
Using resolver: lts-2.22 specified on command line
While constructing the BuildPlan the following exceptions were encountered:

--  While attempting to add dependency,
    Could not find package base-orphans in known packages

--  Failure when adding dependencies:
      base-orphans: needed (<0.5), stack configuration has no specified version (latest applicable is 0.4.5)
    needed for package linear-1.18.3

Recommended action: try adding the following to your extra-deps in /Users/adam/.stack/global-project/stack.yaml
- base-orphans-0.4.5

$ stack build linear-1.18.3 --resolver lts-2.22 base-orphans-0.4.5

<OK>

[bergmark]

I think this has been fixed in stack-1.1.2 via commercialhaskell/stack#2070, but the lts-2.22 snapshot is too old and doesn't contain the needed checksums.

[hvr]

@kgadek We do have a rather elaborate versioning scheme which predates SemVer. However, orphan instances cause problems that can't be properly expressed in our current cabal meta-data framework. I proposed one way to extend the meta-data to handle the case of orphan-instances at haskell/cabal#3061 which explains the issue more in-depth. While that proposal is still in the air, the specific case of base-orphans was handled in a pragmatic way, as install-plans on Hackage involving base-orphans and linear were breaking. It's unfortunate that improving the situation for cabal-users causes regressions for Stack users, but the way I understand it, this is something that Stack may be able to fix by a clever heuristic for LTS snapshots that predate recording the cabal-revision checksums

[ekmett]

@kgadek We use cabal revisions because if we don't then we wind up with packages that retroactively wind up with bad build plans that we know can't succeed when we get an updated version of one of our dependencies that while a valid minor revision w.r.t the PVP happens to bring into scope a conflicting name, or a new instance.

The base-orphans adoption here was IIRC, because both linear and base-orphans, which is needed to build most of linears dependencies in most build configurations provided orphans for the same things, namely Complex numbers, so it fixed many users of linear that were getting bad build plans that would just fail complaining about conflicting instances on newer GHCs.

An alternate revision that could work would be to mark almost every single historical version of linear as having a severely limited known-bad upper bound on most of its dependencies, which would rule out a lot more build plans.

I don't have a dog in this fight. We took the edit that seemed to be the least invasive at the time, and which was most consistent with other edits being done to other packages. If someone wants to chase down precisely what revisions would have to be made to avoid having newer base-orphans and older linear built together, we could possibly go that way instead.

[kgadek]

Thanks all for explaining the difficulties, reasons behind, et al. Pitty that we have kind of "cabal vs stack" choice sometimes.

[ekmett]

Closing this as there is nothing we can really do here but try to be better in the future.