Build error against transformers-0.6 with GHC-9.0

[phadej]

% cabal build -w ghc-9.0.2 --constraint='transformers >=0.6'
Resolving dependencies...
Build profile: -w ghc-9.0.2 -O1
In order, the following will be built (use -v for more details):
 - mtl-2.3.1 (lib) (first run)
Configuring library for mtl-2.3.1..
Preprocessing library for mtl-2.3.1..
Building library for mtl-2.3.1..
[ 1 of 24] Compiling Control.Monad.Accum ( Control/Monad/Accum.hs, /codetmp/mtl-2.3.1/dist-newstyle/build/x86_64-linux/ghc-9.0.2/mtl-2.3.1/build/Control/Monad/Accum.o, /codetmp/mtl-2.3.1/dist-newstyle/build/x86_64-linux/ghc-9.0.2/mtl-2.3.1/build/Control/Monad/Accum.dyn_o )
[ 2 of 24] Compiling Control.Monad.Cont.Class ( Control/Monad/Cont/Class.hs, /codetmp/mtl-2.3.1/dist-newstyle/build/x86_64-linux/ghc-9.0.2/mtl-2.3.1/build/Control/Monad/Cont/Class.o, /codetmp/mtl-2.3.1/dist-newstyle/build/x86_64-linux/ghc-9.0.2/mtl-2.3.1/build/Control/Monad/Cont/Class.dyn_o )

Control/Monad/Cont/Class.hs:205:3: error:
    • Could not deduce (Monad (t m'))
      from the context: (MonadTrans t, Monad m)
        bound by the type signature for:
                   liftCallCC :: forall (t :: (* -> *) -> * -> *) (m :: * -> *) a b.
                                 (MonadTrans t, Monad m,
                                  forall (m' :: * -> *). Monad m' => Monad (t m')) =>
                                 CallCC m (t m a) b -> CallCC (t m) a b
        at Control/Monad/Cont/Class.hs:(205,3)-(207,40)
      or from: Monad m'
        bound by a quantified context
        at Control/Monad/Cont/Class.hs:(205,3)-(207,40)
    • In the ambiguity check for ‘liftCallCC’
      To defer the ambiguity check to use sites, enable AllowAmbiguousTypes
      In the type signature:
        liftCallCC :: forall (t :: (Type -> Type) -> Type -> Type)
                             (m :: Type -> Type)
                             (a :: Type)
                             (b :: Type).
                      (MonadTrans t,
                       Monad m,
                       forall (m' :: Type -> Type). Monad m' => Monad (t m')) =>
                      CallCC m (t m a) b -> CallCC (t m) a b
    |
205 |   forall (t :: (Type -> Type) -> Type -> Type) (m :: Type -> Type) (a :: Type) (b :: Type) . 
    |   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^...

EDIT: Doesn't happen with GHC-9.2.5 or other major versions for example, so probably some GHC bug?

           9.4.3  9.2.5  9.0.2  8.10.7  8.8.4  8.6.5  8.4.4  8.2.2  8.0.2
mtl-2.3.1  OK     OK     FAIL   OK      OK     NO-IP  NO-IP  NO-IP  NO-IP

I'm not making revision without knowing what is the cause of this problem.

Note: mtl-2.3 is compiling fine with GHC-9.0. So there's some change in mtl-2.3.1 which triggered this. EDIT: liftCallCC was added in mtl-2.3.1

[phadej]

The issue seems to be https://gitlab.haskell.org/ghc/ghc/-/issues/20582 as there are two ways to solve Monad (t m') when transformers-0.6 is used. Either via superclass (i.e. MonadTrans) or directly from given constraints.

Writing

liftCallCC :: 
  forall (t :: (Type -> Type) -> Type -> Type) (m :: Type -> Type) (a :: Type) (b :: Type) . 
  (MonadTrans t, Monad m
#if !MIN_VERSION_transformers(0,6,0)
  , forall (m' :: Type -> Type) . Monad m' => Monad (t m')
#endif
  ) => 
  CallCC m (t m a) b -> CallCC (t m) a b
liftCallCC f g = join . lift . f $ \exit -> pure $ g (lift . exit . pure)

seems to fix the issue, but I haven't tested that extensively. (Does mtl's CI test against transformers-0.6 on all GHCs?).

As I identified the GHC issue I made a revision to mtl-2.3.1.

[emilypi]

Thanks for noting the regression and build plan failure @phadej. Odd that this made it through CI without a hiccup - all builds are building for 2.3.1 with no issue. However, even in #131 for 9.0, the build passed, so something's fishy here.

[nomeata]

I ran into the same issue, but also had CI succeed. I think what happens is that when testing locally whether my code compiles with transformers-0.6, I used

--constraint='transformers >0.6'

and thus forced its use. But on CI, because transformersis a package that comes with GHC and thus is already installed, cabal prefers the older, installed version.

This highlights a fundamental issue with our dependency ranges: There is no (good) way to actually check on CI that the full range of allowed versions actually works! haskell/cabal#8387 would help at least with the upper end of the range, and would have caught this incompatibility before it would have hit hackage.

[phadej]

@nomeata Yes, it's an issue with dependency ranges, but there is no way to do better still using some ranges.

This an issue in GHC. Any library version (combination) may trigger a bug in GHC, so we cannot extrapolate using semantics of versioning system. But exhaustively checking all combinations is not realistic. So we approximate as well as we see fit.

And because this is an issue in GHC, everyone can run into this issue in their code - not only mtl. So one possible solution is simply disallow transformers-0.6 on GHC-9.0. Quite blunt, but effective. Alternatively everyone have to be aware of this, like everyone had to be aware to not use PolyKinds on GHC-7.4 (Everyone writing QuantifiedConstraints code has to be aware of this issue, but users may not need to be).

EDIT: Another option is to have GHC-9.0.3, fixing the issue, but AFAIK it's not possible.

[nomeata]

So one possible solution is simply disallow transformers-0.6 on GHC-9.0.

Yes, that’s actually quite reasonable. Is it just a matter of adding

if impl(ghc == 9.0.*)
  buildable: false

to the transformer’s cabal files, and this could be done by a hackage revision even, right?

[phadej]

@nomeata Yes, and no.

buildable: false doesn't have semantics you think. ^†

if impl(ghc ==9.0.*)
  build-depends: unbuildable <0

has correct one.

That cannot be added via revision though currently, as the structure of .cabal files cannot be changed. (IIRC there is issue somewhere on hackage-server or hackage-cli issue tracker to allow revisions adding conditional blocks which add build-depends to things already dependended upon - the general problem is that we don't want allow adding/removing arbitrary dependencies and revisions should be reversible, i.e. you should be able to undo any revision).

Luckily, you can amend build-depends: base to exclude base range used by particular GHC (for now, using base version as a proxy for GHC version).

---

^† The semantics for buildable is that it's checked only when package version is already selected to be included in install plan (a bit like extra-libraries are only checked at the build time). I.e. the cabal solver will happily select package version (with buildable: False), but then the build will fail when configuring. See https://cabal.readthedocs.io/en/stable/cabal-package.html#pkg-field-buildable example. That is bad, and I recommend to never use buildable:. (I suspect its semantics is legacy from ages before cabal-install had proper backtracking solver).

EDIT: there are also additional semantics where cabal toggles buildable: False test-suites and benchmarks for local packages, but I cannot find any documentation about that, and I only remember that it's confusing functionality.

[emilypi]

Ah right tranformers is bundled. Yeah, makes sense. i'll come up with a PR when my brain turns back on after the holidays. Right now, the revision is satisfactory. Thanks both of you! @kozross

[nomeata]

Luckily, you can amend build-depends: base to exclude base range used by particular GHC (for now, using base version as a proxy for GHC version).

Is someone going to that, or should I raise an issue at https://github.com/haskell-infra/hackage-trustees/issues?

[phadej]

Is someone going to that

That is very intrusive revision, as a trustee I won't do it. Pedantically saying, transformers isn't broken. You really should discuss with the maintainer first.

FWIW, GHC-9.2.1 and GHC-9.2.2 are also affected as the bug is fixed in ghc-9.2.3: https://www.haskell.org/ghc/blog/20220527-ghc-9.2.3-released.html (issue #20582)

[nomeata]

Sorry, quite right. Jumped the gun here. I'll create an issue upstream m

[nomeata]

I asked Ross at https://hub.darcs.net/ross/transformers/issue/90

[RossPaterson]

I've changed my mind on this. It seems a bit drastic to block GHC 9.0 from transformers-6.0, when as @phadej says, transformers isn't actually broken here. The above fix for mtl seems appropriate, since for transformers >= 6.0 the forall constraint is redundant, being implied by MonadTrans. It's true that the GHC bug may bite other users combining MonadTrans with type system extensions, but it seems excessive to make users of transformers who aren't using those extensions pay the price for that.

[andreasabel]

@emilypi : Currently mtl-2.3.1 is in an awkward state with a gap in the base versions it supports. Could we get mtl-2.3.1.1 fixing this issue soon?
It seems a fix is already outlined in #138 (comment).

Breakage in the wild:

• https://github.com/haskell-hvr/http-io-streams/actions/runs/4084912048/jobs/7042192308#step:25:29

rejecting: base-4.15.1.0/installed-4.15.1.0 (conflict: mtl => base>=4.12 && <4.15 || >=4.16 && <5)