Remove sandbox functionality

[phadej]

Related is e.g. #6444 (comment)

Maybe it's still a bit too early to remove v1-build, but sandboxes could IMHO be removed. If someone really needs them, they can be simulated.

Yet we need to do some cleanup, as v2-build is the default now.

[CMCDragonkai]

Just a question. In my Gitlab CI/CD I used .cabal-sandbox as the CI cache. Now if sandboxes are removed or I switch over to V2 commands. What is the appropriate thing to cache to speed up CI builds? Is it .cabal/store?

[phadej]

@CMCDragonkai yes. And .cabal/packages, when you clean a bit from there. See e.g. https://github.com/haskell-CI/haskell-ci/blob/f3e046c5ba5ba252ae44f82b2ecfb1e2d8760c0a/.travis.yml#L17-L31

EDIT: so packages .tar.gz aren't downloaded, also there's 01-index.tar.gz which would speed-up cabal update, as only delta is downloaded (and no need to cache quickly rebuildable stuff).

[Mistuke]

Maybe it's still a bit too early to remove v1-build, but sandboxes could IMHO be removed. If someone really needs them, they can be simulated.

How? :) I still use them to reproduce GHC issues people submit so I don't mess up my global config but genuinely unsure how to simulate them. The biggest thing I use them for is to have a package-db with all the dependencies of the repro the user submitted.

Since I often repro with ghc development builds I have to run with --allow-newer so I don't want these packages in my global store-dir as as soon as the ghc version changes I gotta rebuild again anyway (and the space isn't cleaned up automatically).

But new-install 's dist folder only contains a package-db with the package only, not it's dependencies.. so I end up having to use --store-dir to point it locally and then use two package-dbs where sandbox just has one so it's easier to use.

[hasufell]

Sandboxes are still used by some users and are very robust. Recovering from problems/failure is easy, while with v2-build, it basically boils down to removing the entire store. Semi-reproducible bugs like #6483 are a good example

[phadej]

My opinion is: people who need sandboxes could keep using old cabal-install versions. The less code there is blocking new development work, the better.

[hasufell]

The less code there is blocking new development work, the better.

I think that's a little quick. In case the code is really a problem, there should at least be a call for maintainers picking that up. We should know clearly whether there is no interest in this feature or not.

[Mistuke]

My opinion is: people who need sandboxes could keep using old cabal-install versions. The less code there is blocking new development work, the better.

I could.. If new GHC didn't often require bleeding edge Cabal.

[Mistuke]

In any case, whatever you guys decide as maintainers I'll live with :) especially if it's getting in the way of new things.

[hasufell]

Also note, you'll probably break tools like https://github.com/haskell-mafia/mafia when removing sandboxes.

[hvr]

Well, back in 2018 a ticket for "Investigate porting to use cabal new-build" was filed at haskell-mafia/mafia#240 -- but it doesn't seem like there has been much activity looking into how to adapt to cabal's announced v2-style UI. And I've got a hunch this isn't going to change unless there's a forcing function to do so.

So I'm for removing these cabal sandbox codepaths at last -- they're scheduled for removal anyway sooner or later (so the cost of migrating to a different mechanism will be incurred anyway); but delaying doing so translates into a maintainance/development cost for us, and consequently removing it sooner rather than later minimises the overall cost.

Fwiw, while I haven't used Mafia myself, I believe it should be possible to emulate most of cabal sandbox-like workflows atop of cabal v2-build by orchestrating cabal accordingly and controlling the paths where it stores things. I'm happy to help brainstorm w/ the Mafia devs how to do this if they can elaborate on Mafia's requirements and what operations it needs to effect on a cabal project.

[hasufell]

So there was no survey or announcement to figure out if sandboxes are still a feature the community wants?

[hvr]

We did have annoying warnings for all v1-* commands following a generic message-template in cabal 2.4 (released in 2018!) in order to raise some additional awareness (& discussions) that we did want to eventually decomission the v1-* commands, and to give people a chance to speak up...

$ /opt/cabal/2.4/bin/cabal sandbox 
Warning: The sandbox command is a part of the legacy v1 style of cabal usage.

Please switch to using either the new project style and the new-sandbox
command or the legacy v1-sandbox alias as new-style projects will become the
default in the next version of cabal-install. Please file a bug if you cannot
replicate a working v1- use case with the new-style commands.

For more information, see: https://wiki.haskell.org/Cabal/NewBuild

Moreover, the wording

For those who do not wish to use the new functionality, the classic project style will not be removed immediately, but these legacy commands will require the usage of the v1- prefix as of Cabal 3.0 and will be removed in a future release.

has been in the user guide since 2018 as well: https://cabal.readthedocs.io/en/latest/nix-local-build-overview.html

[hasufell]

We did have annoying warnings for all v1-* commands following a generic message-template in cabal 2.4 (released in 2018!) in order to raise some additional awareness (& discussions) that we did want to eventually decomission the v1-* commands, and to give people a chance to speak up...

The cabal help output is large and confusing. I remember people in #haskell largely had the impression that it's just about the deprecation of default behavior, not deprecation of the feature.

has been in the user guide since 2018 as well: https://cabal.readthedocs.io/en/latest/nix-local-build-overview.html

Again, I don't think that is a discussion or even a proper announcement.

[phadej]

I'm developing new feature: active-repositories. It would allow using multiple simultaneous repositories (e.g. hackage.haskell.org + head.hackage) more "deterministic", as the ordering
could be configured. One usage (e.g. for Nix users) is active-repositories: :none - i.e. you wouldn't need to spin complete new ~/.cabal/config just to make "offlline" (but not really) mode.

It's development is currently blocked transitively on this issue.

One needs to remove legacy local repositories (#6729), as they are unnamed so cannot be specified in active-repositories.

However, the above is again blocked on sandboxes, as legacy repositories seems to be used in the sandbox code.

---

One way forward is just to make sandbox code compile, and disable failing tests. It won't be hard removal, but most likely sandboxes will be broken in 3.4.

[phadej]

I'll proceed with above "just break sandboxes" approach in two weeks, i.e. Wednesday 2020-05-06.

[23Skidoo]

Shipping a known-broken implementation is probably worse than outright removing sandboxes.

[emilypi]

@phadej thank you for this. I'm 100% in support of your approach! If you need any help, please feel free to direct me towards anything you need done 😄

[hvr]

@23Skidoo just to clarify, by

Shipping a known-broken implementation is probably worse than outright removing sandboxes.

You do mean, that given that sandboxes will be "known-broken" in 3.4 anyway, it's reasonable (i.e. less "worse") to "outright remove" them with cabal 3.4 (which I'd agree with)?

[23Skidoo]

Yeah, if we don't have the bandwidth to keep supporting sandboxes, let's just remove them.

[phadej]

Sandboxes are removed, and I send a message to cabal-devel/

[xplat]

We did have annoying warnings for all v1-* commands following a generic message-template in cabal 2.4 (released in 2018!) in order to raise some additional awareness (& discussions) that we did want to eventually decomission the v1-* commands, and to give people a chance to speak up...

$ /opt/cabal/2.4/bin/cabal sandbox 
Warning: The sandbox command is a part of the legacy v1 style of cabal usage.

Please switch to using either the new project style and the new-sandbox
command or the legacy v1-sandbox alias as new-style projects will become the
default in the next version of cabal-install. Please file a bug if you cannot
replicate a working v1- use case with the new-style commands.

This message ends up looking really deceptive, referring to a new-sandbox command that did not exist and was never intended to exist. It's even more unfortunate given how many commands in the v2 series were intended to exist but took a long time to materialize. Water under the bridge now, but this understandably came as a surprise to a lot of people.

[phadej]

I can only repeat that changing to use cabal-install-3.4 is not mandatory. If you are happy with previous versions and there is no pressing need to upgrade, don't.

Also cabal-install-3.4 will only be released in about three months (the earliest!) so there is still time to prepare.

[ivanperez-keera]

We did have annoying warnings for all v1-* commands following a generic message-template in cabal 2.4 (released in 2018!) in order to raise some additional awareness (& discussions) that we did want to eventually decomission the v1-* commands, and to give people a chance to speak up...

From what I see, the decision is made, so I'm saying this with the hope to find a better way to stay on top of these discussions and contribute in a timely manner when and where they happen. And, just in case, I absolutely understand that maintaining these features can be a burden and as a user all I have is gratitude for cabal (and the whole Haskell ecosystem) existing in the first place, and I thank all who make it happen.

I saw these warnings years ago and I never interpreted this as "open to discussion", nor did I find a pointer to express an opinion.

As in prior occasions, I had to opt to try and "stay alive" in any way I could: the Haskell ecosystem generally changes so, so much that I've always found myself having to choose between creating up to date Haskell tools, or creating real-world applications with (not-so-new) Haskell. However, the burden of doing both has been increasingly higher in any of the environments where I worked. I made a decision to opt for the latter: it is a no-brainer IMO, but my opinion seems to counter that of others.

I'd like to inquire about the best way to stay on top of these discussions, and participate sooner if possible. For example, being subscribed to Cabal on github means having to process an average of 2x issues a day, just in case a discussion about a breaking feature comes up, and each issue will be an asynchronous conversation thread on its own. We also have GHC, base, plus the hundreds of libraries that projects tend to depend on these days because the general tendency for fine partitioning and additional levels of abstraction.

And just to be clear, I'm not trying to disguise a complaint as a question. I see that people seem to be able to cope with these levels of complexity, and I wonder what I'm missing, or what methods others may have found. In the last 20 years, projects have become much, much harder to maintain IMO.

[phadej]

@ivanperez-keera I mentioned this in https://mail.haskell.org/pipermail/cabal-devel/2020-May/010482.html and will soon (in July) post a call-for-features for 3.6 with preliminary list of what I know is worked on. cabal-devel is very low-traffic list.