Overhaul CI to use ghcup and not a mixture of obsolete sources of GHC, etc.

[Mikolaj]

We really need a proper overhaul moving from a mixture of containers, Ubuntu packages, curl, chocolatey and downloads of random old snapshots of packages to ghcup. E.g., we are stuck at 8.10.4 (or even much earlier, e.g., 8.6.5) due to that.

Related: #7747, #7742, #7708, #7655, #7315 and many others.

[phadej]

FWIW, I'd really appreciate if Cabal kept testing that it can control older GHCs: Currently we have

                    , GhcJob "8.8.4"  True  "--lib-only"                True  ["8.8.4"] $
                        libSteps ++
                        [ "lib-suite-extras --extra-hc /opt/ghc/7.0.4/bin/ghc-7.0.4"
                        , "lib-suite-extras --extra-hc /opt/ghc/7.2.2/bin/ghc-7.2.2"
                        , "lib-suite-extras --extra-hc /opt/ghc/7.4.2/bin/ghc-7.4.2"
                        ]

job, which tests that cabal-install & Cabal can use these old GHCs as compilers.

This is different then being buildable with them, that's not that important and there the 5/3 year policy is very conservative already.

[jneira]

My point of view about ci:

1. We should put as much logic as possible in the test suite itself: it is haskell code and it would help to migrate ci between providers (and cabal was moved several times) easier and use several providers at same time (as we are doing with github/gitlab)
2. I would try to minimize shell and other langs like python scripts, they are not always os independent and add complexity and cognitive load as you have to learn and maintain them. I would try to have only haskell code and ci configuration, nothing more in the middle
3. I would not use templates even written in haskell unless the complexity they add reduce data duplication and maintance cost dramatically (it may be the case with cabal, dont know). ci providers has their own ways to reduce duplication for things which cant be put in the test suite itself. Templates makes some ci providers features unusable see add dependabot: add Actions CI updates merge requests automation #7809

[jneira]

I would say we should use haskell/action/setup, abstracting away the way we get ghc's, even abstracting over ghcup, and try to make possible download older ghcs there (it it is not already possible). It would help all the haskell community

[Mikolaj]

I'm a fan of baby steps, but I'm not averse to dropping existing templates one by one, once we have a replacement and we've thought through the cost of each such switch.

Regarding a switch to haskell/action/setup, tempting as it is, it binds us to GHA. I remember when I thought I would use Travis CI forever...

[jneira]

Regarding a switch to haskell/action/setup, tempting as it is, it binds us to GHA. I remember when I thought I would use Travis CI forever...

Yeah it is against my point 1 in the list above. Bring it to the extreme we could put the ghc download logic in the test suite itself, in haskell code, using ghcup or whatever. The gh action is written in javascript, so it is against the 2 point.

But, well, i think it is not exactly the same scenario, we are already tied to github having the repo here and we are using all the features gh is offering us, like branch protection. So to be tied to gh workflow is not a qualitative but quantitative thing.

Otoh all ci providers are copying features between them and gitlab is offering something similar to gha. With a little bit of luck, porting it will be not extremely hard.

[andreabedini]

After #7952 we are now relying 100% on haskell/action/setup on all three platforms. I am not sure whether haskell/action/setup uses only ghcup or a combination of tools but what matters here is that we have one single of doing it and the rest can be improved elsewhere. I belive this issue can be closed.

This might not be the best place to discuss this but I need to take a pebble out of my shoe.

I don't find this discussion around TravisCI & GHA and "what happens if" useful. Yes, TravisCI story is unfortunate (I remember it clearly) but the rest of the OSS community has dealt with it. We can deal with it too.

I don't want to spend any time developing a 100% Haskell, self-maintained and self-hosted CI. The goal here is to develop cabal, not to chase autarky. We can assume we are always going to depend on other services (whatever the are paid or free) and I think we should use them anytime the help us to make a better cabal.

Apologies if this is coming out a bit like a rant. I appreciate it is a trade-off we make, but for me it's a very clear one.

[jneira]

After #7952 we are now relying 100% on haskell/action/setup on all three platforms. I am not sure whether haskell/action/setup uses only ghcup or a combination of tools but what matters here is that we have one single of doing it and the rest can be improved elsewhere. I belive this issue can be closed.

From my investigations in #8000 i am afraid we will have to rely in some older container and hvr/ppa to test ghcs < 7.10/7.8

[andreabedini]

From my investigations in #8000 i am afraid we will have to rely in some older container and hvr/ppa to test ghcs < 7.10/7.8

If we can't even install GHC 7.10 on modern Ubuntus, I think we should not do this at all.

[jneira]

linux is not my main os (i am on windows) so not sure if i am the right person to talk about that.
I guess you could install ghc-7.10 in modern ubuntus, i did not find a reliable way to install it and make work the existing validate script at same time

[andreabedini]

I guess you could install ghc-7.10 in modern ubuntus, i did not find a reliable way to install it and make work the existing validate script at same time

ghcup stops at 7.10.3 (so 7.8 is out) but the installer doesn't work https://gitlab.haskell.org/ghc/ghc/-/issues/20782 so yeah /shrugh

[jneira]

ghcup stops at 7.10.3 (so 7.8 is out) but the installer doesn't work https://gitlab.haskell.org/ghc/ghc/-/issues/20782 so yeah /shrugh

haskell setup-action is able to install older versions of ghc cause it falls back to hvr/ppa but it is not guaranteed (it is stated in its readme), see

• https://github.com/jneira/cabal/actions/runs/1874957564: it started to install in ubuntu-20.04, 7.10.3, 7.4.2, 7.2.2 and 7.0.4 but no 7.8.4 and 7.6.3 🤷
• https://github.com/jneira/cabal/runs/5272663216: it installed all old versions (7.10.3, 7.4.2, 7.2.2, 7.0.4, 7.8.4 and 7.6.3) on ubuntu-18.04

[Mikolaj]

If we can't even install GHC 7.10 on modern Ubuntus, I think we should not do this at all.

I'm using Ubuntu 16.04, which still has (optional) support and, believe me, some companies are even more conservative or even less eager to spend resources upgrading anything.

[andreabedini]

I'm using Ubuntu 16.04, which still has (optional) support and, believe me, some companies are even more conservative or even less eager to spend resources upgrading anything.

Fair enough.

[jneira]

haskell setup-action is able to install older versions of ghc cause it falls back to hvr/ppa

In fact is the other way around, it tries hvr/ppa first and then ghcup

[jneira]

Re: haskell/actions#97 (use ghcup in windows)
Also related: #8011

[Mikolaj]

I think we are hit by this issue right now, because our CI is failing due to base version being too hight, which probably stems from GHC upgrade on GHA that our build tools are not ready for. An example of failed CI jobs is at #7386.

[Mikolaj]

Actually, it seems this was not GHC upgrading GHC, but ghcup changing the "recommended" version and our scripts install that version. Huh.

[Mikolaj]

It seems we need to upgrade the doctest checker we use (or switch to another one) to handle GHC 9.2. So I'm sticking to GHC 8.10.7 until this is done.

BTW, it seems this ticket is not as related to the current CI breakdown as I thought. If anything, the current crisis shows ghcup does not solve the problem totally. Perhaps a mixture of GHC versions "recommended" by ghcup and of fixed versions is a good idea. Independently, we should upgrade our build/test/CI tools at our own pace. Ideally, we'd not release cabal officially supporting a particular GHC version until all our tools can handle that version. We are so far from that yet.

[ulysses4ever]

Haskell actions now have a steady leadership and follows the ecosystem in lock step (new GHC releases and even prereleases are readily available). I think we could close this issue for now.

[Mikolaj]

Sure. All the initially mentioned tickets are closed too.