How to correctly install a locally-built library in Cabal 3?

[acrylic-origami]

I'm having difficulties installing and using a library built from source. It's likely just a documentation/my understanding issue, but to do the following:

1. Build a cabal library package from source that depends on other [Hackage] cabal packages;
2. Install the package into either the user or global package db;
3. Depend on the package in another project on the same machine.

I could only find one working method that I cannot believe is the Right Way:

1. runhaskell Setup.hs configure --global; must be global. User installs aren't found by other projects (which I don't understand).
2. For all Hackage dependencies, manually register them into the global package db. That means ghc-pkg register --global /path/to/user/package.db/package-name.conf.
   • This was needed because setup build was failing with "Encountered missing dependencies". I saw the call it was making to discover packages was /usr/local/bin/ghc-pkg dump --global -v0, where the package (installed by cabal install --lib) was missing. Trying to install with cabal install --global --lib package-name still doesn't show up in ghc-pkg dump --global.
   • This is the main pain point for this otherwise documented workflow. Should Hackage packages be registered by default to the global db? Alternatively, should Setup look in the user package db too?
3. runhaskell Setup.hs build
4. sudo runhaskell Setup.hs install
5. Include the package elsewhere in build-depends as usual.

This is with Setup as Distribution.Simple.defaultMain on GHC 8.6.5 and Cabal 3.0.0.0. GHC 8.4.2 + Cabal 2.4.0.0 surprisingly works with a simple cabal build; cabal install;, so maybe it's a version thing?

Any other combinations seem to fail. In particular, the intuitive cabal new-install --lib: the package isn't found in ghc-pkg list nor cabal list --installed and is an "unknown package" to other libraries. Interestingly, trying to use the package via -fplugin without build-depends will instead say that it's a hidden module of our library.

In case this is rare workflow, at least to develop source plugins it's a vital one. This way, I don't have to specify a source-repository-package and build it again and again in every codebase I analyze, or upload it to Hackage for every edit I want to test.

[elliottslaughter]

I've had a similar issue, where I want to write "scripts" that make use of the Pandoc API. I have not been able to find a set of new-style commands that will let me use ghc --make to link against an installed copy of Pandoc. See jgm/pandoc#6066 for the details of what I've tried.

The only thing I've found that works is to go back to the v1- subcommands, i.e. cabal v1-install pandoc.

It would be nice if there were a way to use the new-style subcommands, otherwise I guess I'll just not be updating past the point where the v1- subcommmands are removed.

[phadej]

@elliottslaughter there is cabal install --lib, which will create a package environment with pandoc. This issue about the case, when you want to install the local (say modified) version of a library. I think that cabal install --lib should "work" in that case to (modulo the reported issues like #5559, where it fails when you try to update an environment)

[elliottslaughter]

@phadej If you see jgm/pandoc#6066, I did try this, with the result:

$ cabal install --lib pandoc
$ ghc-pkg list pandoc
.../ghc/8.8.1/lib/ghc-8.8.1/package.conf.d
    (no packages)
$ ghc --make behead.hs 
Loaded package environment from /Users/elliott/.ghc/x86_64-darwin-8.8.1/environments/default
[1 of 1] Compiling Main             ( behead.hs, behead.o )

behead.hs:4:1: error:
    Ambiguous module name ‘Text.Pandoc’:
      it was found in multiple packages: pandoc-2.9.1.1 pandoc-2.9.1.1
  |
4 | import Text.Pandoc
  | ^^^^^^^^^^^^^^^^^^

I also tried --global --lib (won't copy that output to save space, but see the other issue).

Is this a bug? Should I open a new issue?

[phadej]

$ ghc-pkg list pandoc

is wrong, cabal v2-install --lib pandoc doesn't register anything in global package database, that's the core point of v2-build approach.

If you instead look into ~/.ghc/ghc-8.8.1-.../environments/default, there should be pandoc.
Also if you look ghc-pkg list --package-db=~/.cabal/store/ghc-8.8.1/package-db there will be pandoc (and a lot of other stuff).

So, ghc -package-env=default --make behead.hs should work (at least in theory).

---

I'm really sorry that documentation is not up to date, if you find something missing the documentation PRs are more than welcome.

---

Arguably the cabal v2-install --lib libname should tell which environment file it rewrote, maybe it will clarify the situation further. That should be an easy change to make. (Currently it seems to end, like it did nothing further then building the library).

[elliottslaughter]

Ok, that works, though I had to also run cabal v2-install --lib pandoc-types because otherwise pandoc-types is a hidden package, apparently. I guess I'll need to do this with every Pandoc dependency that I use in my scripts.

I agree that modifying the output of cabal v2-install would help, and you could go even further and provide some usage hints (because after all, how is anyone supposed to use an installed library if the commands don't do so by default?).

I'm not honestly sure what to suggest for the documentation. It seems the new-style subcommands introduce a new abstraction (the environment), but this is something shared between Cabal and GHC. Presumably the Cabal documentation wouldn't be the place to find out that I need to run ghc -package-env=default --make .... But you're right, there is a general lack of information, and it's hard to know where to even look at the moment.

[acrylic-origami]

Hmm, I'm still not able to get it working with cabal v2-install --lib libname. Other packages can't see the lib once installed. I do see it in ~/.cabal/store, but it doesn't show up in cabal list --installed either. Maybe it isn't getting registered? This is the default GHC 8.6.5 + Cabal 3.0.0.0 combo installed by ghcup on a fresh Ubuntu image. Here's what I get:

$ mkdir issue6478-lib && cd issue6478-lib
$ touch LICENSE
$ cabal init --lib
$ cabal v2-install --lib issue6478-lib
Resolving dependencies...
Build profile: -w ghc-8.6.5 -O1
In order, the following will be built (use -v for more details):
 - issue6478-lib-0.1.0.0 (lib) (requires build)
Configuring library for issue6478-lib-0.1.0.0..
Preprocessing library for issue6478-lib-0.1.0.0..
Building library for issue6478-lib-0.1.0.0..
[1 of 1] Compiling MyLib            ( MyLib.hs, dist/build/MyLib.o )
Installing library in /home/leaf/.cabal/store/ghc-8.6.5/incoming/new-936/home/leaf/.cabal/store/ghc-8.6.5/issue6478-lib-0.1.0.0-18a99bae1973d0ff27a0d1eac6b057cf6cf68066ecae7c323cc34066a07a2f13/lib
$ cd ..
$ mkdir issue6478-test && cd issue6478-test
$ cabal init
$ sed -i 's/build-depends:/build-depends: issue6478-lib, /g' issue6478-test.cabal
$ cabal v2-build
Resolving dependencies...
cabal: Could not resolve dependencies:
[__0] trying: issue6478-test-0.1.0.0 (user goal)
[__1] unknown package: issue6478-lib (dependency of issue6478-test)
[__1] fail (backjumping, conflict set: issue6478-lib, issue6478-test)
After searching the rest of the dependency tree exhaustively, these were the
goals I've had most trouble fulfilling: issue6478-test, issue6478-lib
$ ls ~/.cabal/store/ghc-8.6.5
issue6478-lib-0.1.0.0-18a99bae1973d0ff27a0d1eac6b057cf6cf68066ecae7c323cc34066a07a2f13
incoming
package.db
$ ls ~/.cabal/store/ghc-8.6.5/package.db
issue6478-lib-0.1.0.0-18a99bae1973d0ff27a0d1eac6b057cf6cf68066ecae7c323cc34066a07a2f13.conf
package.cache
package.cache.lock

[acrylic-origami]

I see now that ghc --make does work on packages installed with v2-install --lib, just not when those packages are named in the Cabal config file. Is it really just that it isn't registered properly for Cabal to find it? This note from the first comment seems relevant:

Interestingly, trying to use the package via -fplugin without build-depends will instead say that it's a hidden module of our library.

Also, I couldn't pick this out from the conversation, so I reproduced @elliottslaughter 's secondary issue by installing the dependent package twice, i.e. two versions end up in ~/.cabal. This isn't solved by -package-env, but by removing the old versions. Sorry if this is old news.

[asr]

I'm getting a similar problem when installing the upstream versions of shellac and lambda-shell.

Cabal and GHC versions:

$ cabal --version
cabal-install version 3.0.0.0
compiled using version 3.0.0.0 of the Cabal library 

$ ghc --numeric-version
8.6.5

First, I install the shellac library:

$ git clone https://github.com/robdockins/shellac.git
$ cd shellac
$ cabal install --lib
Wrote tarball sdist to /tmp/shellac/dist-newstyle/sdist/Shellac-0.9.9.tar.gz
Wrote tarball sdist to
/tmp/shellac/dist-newstyle/sdist/Shellac-editline-0.9.9.tar.gz
Wrote tarball sdist to
/tmp/shellac/dist-newstyle/sdist/Shellac-readline-0.9.9.tar.gz
Wrote tarball sdist to
/tmp/shellac/dist-newstyle/sdist/Shellac-haskeline-0.2.1.tar.gz
Wrote tarball sdist to
/tmp/shellac/dist-newstyle/sdist/Shellac-compatline-0.9.9.tar.gz
Resolving dependencies...
Up to date

And the shellac library is in ~/.cabal/store/ghc-8.6.5:

$ cd ~/.cabal/store/ghc-8.6.5
$ ls | grep Shellac
Shellac-0.9.9-71a5d5c15c193a908448cf9a7f3239951c5c0f6bf20708c697ddd938c81ae597

Now, I try to install lambda-shell:

$ git clone https://github.com/robdockins/lambda-shell.git
$ cd lambda-shell
$ cabal install
Wrote tarball sdist to
/tmp/lambda-shell/dist-newstyle/sdist/LambdaShell-0.9.9.tar.gz
Resolving dependencies...
Build profile: -w ghc-8.6.5 -O1
In order, the following will be built (use -v for more details):
 - Shellac-0.9.9 (lib) (requires build)
 - readline-1.0.3.0 (lib:readline) (requires build)
 - Shellac-readline-0.9.9 (lib) (requires build)
 - LambdaShell-0.9.9 (exe:lambdaShell) (requires build)
Configuring library for Shellac-0.9.9..
Preprocessing library for Shellac-0.9.9..
Building library for Shellac-0.9.9..
...
[ 4 of 10] Compiling System.Console.Shell.PPrint ( src/System/Console/Shell/PPrint.hs, dist/build/System/Console/Shell/PPrint.o )

src/System/Console/Shell/PPrint.hs:67:21: error:
    Ambiguous occurrence ‘<>’
    It could refer to either ‘Prelude.<>’,
                             imported from ‘Prelude’ at src/System/Console/Shell/PPrint.hs:50:1-38
                             (and originally defined in ‘GHC.Base’)
                          or ‘System.Console.Shell.PPrint.<>’,
                             defined at src/System/Console/Shell/PPrint.hs:96:3
   |
67 |         []  -> left <> right
...

and the installation fails because cabal-install is trying of install shellac from Hackage (and this version does not support GHC 8.6.5).

Any ideas?

[acrylic-origami]

@asr Interesting, it also seems then that v1-install into the user package db won't be resolved by cabal elsewhere when the package is also published to Hackage. I was able to get your package to build by installing shellac into the global package db with runhaskell Setup.hs install.

[asr]

I was able to get your package to build by installing shellac into the global package db with runhaskell Setup.hs install.

I could reproduce the installation using the global package db. However the installation failed installing shellac in the user package db. Weird.

it also seems then that v1-install into the user package db won't be resolved by cabal elsewhere when the package is also published to Hackage.

No. Using v1-install worked for me.

[acrylic-origami]

@asr Sorry yes, I understand now: the second package must also be built with v1 commands to find the first installed by v1-install.

[jrp2014]

It would be v good to have some common use scenarios drawn out in the documentation. (I'd offer to help, but don't have enough knowledge to do so reliably.) For example:

• it would be good to have an upfront explanation of the hierarchy of basic concepts and how they inter-relate: environment (in various flavours: ghc, package, project, etc), profile, project, package, library, executable, test, ... global v local v user, tabulating the differences between v1 and v2

• Basic use case: building an executable from hackage (trivial)

• Including a hackage library in a project that you are developing (trivial)

• Including a local version of a library in a package that you are developing that is not on hackage

• Including a local version of a library in a package that you are developing that is also on hackage

• packages that have both libraries and

• Multi-package projects, etc

• Compiler plugins

• How to address the most common diagnostics (eg, hidden packages, version conflicts (less of a problem these days) etc

This task-based documentation would complement the reference material in the manual.

I'm sure that this would not be an arduous task and would significantly reduce the cost of entry to building robust cabal configurations.

Perhaps this already exists somewhere, but I wasn't able to find it in a way that I could grok.

[fgaz]

cabal install --lib puts the packages in an environment so that they are visible to ghc, but cabal still cannot refer to them from another project unless you have a way to unambiguously specify that package's hash, so I think op's problem can be solved with a cabal.project that refers to both packages:

packages: path/to/package1 path/to/package2

then cabal build/run/whatever package2

It would be v good to have some common use scenarios drawn out in the documentation

I also think the docs could benefit a lot from a cookbook if there isn't one already. Maybe you could open an issue with a list of common scenarios, then we can start filling out the "how"s?

[acrylic-origami]

@fgaz This is a pretty good solution. Somehow I had never seen this usage, although searching for similar topics I'm finally starting to notice it, albeit usually buried under some sandbox-related solution.

I also see that it won't work for more than one level of local dependency (see #5060). Also seen in that issue are some further difficulties of truly installing then depending on packages as opposed to building in every package the cabal.project way. But for my little project ecosystem the latter usually works out okay.

[fgaz]

@acrylic-origami is this solved?

There's also https://github.com/phadej/cabal-extras/tree/master/cabal-env , a prototype of what a future install --lib sould be (#6481)

[acrylic-origami]

@fgaz Yes! packages with local paths worked well for me, because my dependencies have so far always just been one level deep.

Thanks for linking the discussion around cabal env too, the local install options look very appealing to this, and fingers crossed they would also allow layered local deps easily (should they land).

[andreasabel]

I came here googling for solutions how to depend on a unreleased package.
I am trying to fix UlfNorell/agda-bench#1.
Thus, I need to depend on the yet unreleased Agda-2.6.2 in the cabal file of agda-bench.

With v1-cabal, I would simply have typed

cabal v1-install

in the root of my git working copy of Agda (development version, 2.6.2) and then Agda-2.6.2 would have been picked up when building agda-bench.

I am looking for a v2-cabal drop-in replacement for this workflow.
After poking round in issues I come to the conclusion that this does not exist, right?
I have to settle for a less principled workaround, like adding a cabal.project file that links to the Agda source directory. This is of course not portable to other users, e.g. when I make a PR on agda-bench, the receiver has to also do a similar setup to test my contribution. Further, when building agda-bench it will build its own copy of Agda-2.6.2 (takes 30-40 minutes).

This does not seem to scale. Imagine a scenario where you have M projects each depending on N local packages, each of which takes half an hour to build. Say the build time for the M projects alone (with the N packages already built) is negligible. Your total v2-build time for these M projects is then MN/2 hours, as opposed to just N/2 hours in v1-style.

Even just 1 additional build of Agda hurts badly...

[andreasabel]

I tried putting Agda as one of the packages in cabal.project:

$ cat > cabal.project
packages: ../../../agda-erasure, .

abel:00:01:38:0:~/bin/src/agda-bench$ cabal build
Resolving dependencies...
Build profile: -w ghc-8.10.3 -O1
In order, the following will be built (use -v for more details):
 - Agda-2.6.2 (lib:Agda) (first run)
 - abstract-deque-0.3 (lib) (requires build)
 - abstract-par-0.3.3 (lib) (requires build)
 - cereal-0.5.8.1 (lib) (requires build)
 - criterion-measurement-0.1.2.0 (lib) (requires build)
 - js-chart-2.9.4.1 (lib) (requires build)
 - microstache-1.0.1.2 (lib) (requires build)
 - math-functions-0.3.4.1 (lib) (requires build)
 - text-short-0.1.3 (lib) (requires build)
 - vector-binary-instances-0.2.5.1 (lib) (requires build)
 - vector-algorithms-0.8.0.4 (lib) (requires build)
 - vector-th-unbox-0.2.1.7 (lib) (requires build)
 - monad-par-extras-0.3.3 (lib) (requires build)
 - mwc-random-0.15.0.1 (lib) (requires build)
 - cassava-0.5.2.0 (lib) (requires build)
 - dense-linear-algebra-0.1.0.0 (lib) (requires build)
 - monad-par-0.3.5 (lib) (requires build)
 - statistics-0.15.2.0 (lib) (requires build)
 - criterion-1.5.9.0 (lib) (requires build)
 - agda-bench-1.1.0 (exe:agda-bench) (first run)
[1 of 1] Compiling Main             ( /Users/abel/bin/src/agda-bench/dist-newstyle/build/x86_64-osx/ghc-8.10.3/Agda-2.6.2/setup/setup.hs, /Users/abel/bin/src/agda-bench/dist-newstyle/build/x86_64-osx/ghc-8.10.3/Agda-2.6.2/setup/Main.o )
Linking /Users/abel/bin/src/agda-bench/dist-newstyle/build/x86_64-osx/ghc-8.10.3/Agda-2.6.2/setup/setup ...
Configuring Agda-2.6.2...
Preprocessing library for Agda-2.6.2..
unused rules: 6
Building library for Agda-2.6.2..
[  1 of 394] Compiling Agda.Interaction.ExitCode ( src/full/Agda/Interaction/ExitCode.hs, /Users/abel/bin/src/agda-bench/dist-newstyle/build/x86_64-osx/ghc-8.10.3/Agda-2.6.2/build/Agda/Interaction/ExitCode.o, /Users/abel/bin/src/agda-bench/dist-newstyle/build/x86_64-osx/ghc-8.10.3/Agda-2.6.2/build/Agda/Interaction/ExitCode.dyn_o )
...
[394 of 394] Compiling Agda.Main        ( src/full/Agda/Main.hs, /Users/abel/bin/src/agda-bench/dist-newstyle/build/x86_64-osx/ghc-8.10.3/Agda-2.6.2/build/Agda/Main.o, /Users/abel/bin/src/agda-bench/dist-newstyle/build/x86_64-osx/ghc-8.10.3/Agda-2.6.2/build/Agda/Main.dyn_o )

It doesn't say anything about Installing library in... but continues right away with the next dependency.

Configuring library for abstract-deque-0.3..
Preprocessing library for abstract-deque-0.3..
Building library for abstract-deque-0.3..
...
[12 of 12] Compiling Criterion        ( Criterion.hs, dist/build/Criterion.o, dist/build/Criterion.dyn_o )
Installing library in /Users/abel/.cabal/store/ghc-8.10.3/incoming/new-88687/Users/abel/.cabal/store/ghc-8.10.3/crtrn-1.5.9.0-d7f6dc56/lib
Configuring executable 'agda-bench' for agda-bench-1.1.0..
Preprocessing executable 'agda-bench' for agda-bench-1.1.0..
Building executable 'agda-bench' for agda-bench-1.1.0..
[1 of 1] Compiling Main             ( Main.hs, /Users/abel/bin/src/agda-bench/dist-newstyle/build/x86_64-osx/ghc-8.10.3/agda-bench-1.1.0/x/agda-bench/build/agda-bench/agda-bench-tmp/Main.o )

Main.hs:16:1: error:
    Could not load module ‘Agda.Interaction.Options.HasOptions’
    it is a hidden module in the package ‘Agda-2.6.2’
    it is a hidden module in the package ‘Agda-2.6.2’
    Use -v (or `:set -v` in ghci) to see a list of the files searched for.
   |
16 | import Agda.Interaction.Options.HasOptions (commandLineOptions)
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Main.hs:17:1: error:
    Could not load module ‘Agda.Interaction.Options.Base’
    it is a hidden module in the package ‘Agda-2.6.2’
    it is a hidden module in the package ‘Agda-2.6.2’
    Use -v (or `:set -v` in ghci) to see a list of the files searched for.
   |
17 | import Agda.Interaction.Options.Base       (optInputFile)
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
abel:01:01:14:1:~/bin/src/agda-bench$ 

(Note the whole build attempt took 1h 0min 36sec.)

What am I missing? Why isn't GHC finding the package Agda-2.6.2 it just spend >40 min to compile?

The environment file does contain an "inplace" version of Agda:

$ cat .ghc.environment.x86_64-darwin-8.10.3 
-- This is a GHC environment file written by cabal. This means you can
-- run ghc or ghci and get the environment of the project as a whole.
-- But you still need to use cabal repl $target to get the environment
-- of specific components (libs, exes, tests etc) because each one can
-- have its own source dirs, cpp flags etc.
--
clear-package-db
global-package-db
package-db /Users/abel/.cabal/store/ghc-8.10.3/package.db
package-db dist-newstyle/packagedb/ghc-8.10.3
package-id Agda-2.6.2-inplace
package-id sn-1.5.5.1-608e3df1
package-id ttprsc-0.13.2.4-a752863c
package-id array-0.5.4.0
...

[fgaz]

@andreasabel

What am I missing? Why isn't GHC finding the package Agda-2.6.2 it just spend >40 min to compile?

Not the package, the module:

Could not load module ‘Agda.Interaction.Options.Base’ it is a hidden module

That is, the module is listed in other-modules. You have to put it in exposed-modules to make it available to other components/packages.

[acrylic-origami]

@andreasabel As for trying to install Agda as a proper global package, I've had success (caveat: for packages with far fewer dependencies) with the Setup.hs-based install outlined in my first post. To elaborate on step "2" of registering all the package dependencies globally (which should really be the first step), this is my workflow:

1. Build all the dependencies of the package I want to install normally via cabal build in the package root.

   • Optionally, stop before/during the compilation of the package itself if it takes a long time — the package will be built separately in the Setup.hs flow

2. Keep running ghc-pkg register --global on all the packages in your package.db until they all find their dependencies and are registered. You'll eventually clear out all the "dependency ... doesn't exist (use --force to override)" errors because each pass should register at least some packages, which will satisfy more packages in later passes. Worst-case N passes. Something along the lines of:

   find ~/.cabal/store/ghc-version/package.db -type f -name "*.conf" | while read pkg; do
    ghc-pkg-register --global "$pkg";
   done;
   

3. If your GHC/Cabal is fresh, ensure capital-C Cabal is registered globally via cabal install --lib Cabal so you can run Setup scripts.

Then run the remaining steps, which will build and install the package globally. In the dependents, so long as your local version is versioned beyond the one on Hackage, constraining that package appropriately should convince Cabal to use your local version.

You may want to do this on a clean/sandboxed GHC+Cabal as updating those global registrations of all the transitive dependencies can be a big pain.

[fgaz]

@andreasabel re artifact sharing:

I am looking for a v2-cabal drop-in replacement for this workflow.
After poking round in issues I come to the conclusion that this does not exist, right?

No, v2- is fundamentally different. that's why it had a prefix and everything.
In general, the interface of v2- commands is a level of abstraction above package databases.

I have to settle for a less principled workaround

I'd say the principles are just different, less imperative and more functional/declarative. There are a few different ways you could declare dependencies in your scenario, with different artifact sharing properties.

• Adding Agda to the agda-bench cabal project (rebuilds Agda)
• Adding agda-bench to the Agda cabal project
• If the projects are tightly coupled, creating a top-level cabal project for both
• If agda-bench is an official Agda benchmark, adding it to the Agda package, in a benchmark stanza

As for the N*M problem, apart from putting them all in a big project, it looks like the best solution would be to treat the N packages as remote packages by adding them to a global file:// or file+noindex:// repository.

---

As @acrylic-origami wrote, there is one single exception to all this: system packages, which are installed in the global package db. cabal picks them up when possible so that linux distributions or other package managers (such as nix) can provide binary packages that can be used directly. This usually includes only base and a few more packages.

At the moment, only the global package db has this property.

Also, for @andreasabel use case I'd avoid messing with package dbs

[yaxu]

No, v2- is fundamentally different. that's why it had a prefix and everything.

And then it didn't.. and old commands silently stopped working, with old use cases no longer supported, and the new behaviour undocumented.

[donaldcallen]

This is such a fundamental operation. To me it borders on the insane that it isn't properly documented and/or working. I'm just throwing up my hands on Haskell. Brilliant language, amazing compiler, unusable toolset. There are other good choices for software development that don't inflict this level of pain.

[Mikolaj]

@donaldcallen: I thoroughly sympathise. Would you like to help improving the situation? Trying out the outlined solutions or solution prototypes and submitting documentation PRs are the ways in which you can have impact immediately.

[kaol]

I found this thread when I was searching why running cabal install --lib didn't seem to affect anything. I don't know what is the proper way of doing this, but what worked for me was to replace my ~/.ghc/x86_64-linux-8.8.4/package.conf.d with ln -s ~/.cabal/store/ghc-8.8.4/package.db/ ~/.ghc/x86_64-linux-8.8.4/package.conf.d. After that cabal build found the dependency.

[Mikolaj]

@kaol: thank you for sharing the tips. Just to make sure: you really do need to manually install libraries? Normally it's not necessary and actually causes problems.

[kaol]

I want to audit all my dependencies before even building them. I don't see that there's the option to download only with cabal install so this is what I'm doing.

[Mikolaj]

I'm sure there are many ways to do that more conveniently. How about --only-download?

[kaol]

Yes, it would fit my purpose. But it's not yet available in the Cabal version I'm using.

[Mikolaj]

Well, in that case, --dry-run or anything that creates a build plan and then cabal get for each individual package, audit it, move to the directory where cabal looks for sources (or specify in cabal.project where to look)?

[l8l]

After reading all of the above, I am more confused than before. Perhaps someone could help me with a summary for dummies of how to actually install a local library now. What I tried:

cd mylibrary
cabal build
cabal install --lib

This worked fine. Then I go to another project on the same machine. In the otherProject.cabal, I add, under build-depends the installed library mylibrary. Then I run cabal build and I get the error:

Resolving dependencies...
cabal: Could not resolve dependencies:
[__0] trying: otherProject-0.1.0.0 (user goal)
[__1] unknown package: mylibrary (dependency of otherProject)
[__1] fail (backjumping, conflict set: otherProject, mylibrary)
After searching the rest of the dependency tree exhaustively, these were the
goals I've had most trouble fulfilling: otherProject, mylibrary

However, if I do not include mylibrary in build-depends and only import the Module, I obtain the message

    Could not load module ‘System.Mylibrary’
    It is a member of the hidden package ‘mylibrary-2.0’.
    Perhaps you need to add ‘mylibrary’ to the build-depends in your .cabal file.

which means that the GHC does know of mylibrary and even recommends putting it into build-depends but when I do that, it throws an error. Why does it not work and how to fix it?

[Mikolaj]

@l8l: are you sure you need to install the library? It shouldn't be needed except for some rare cases and even there mostly for repl usage convenience.

[l8l]

@Mikolaj Thanks for the reply. I would like to use the functions in that library in other projects without copying them into those other projects. Is that possible without installing a library? Otherwise I do not undertand the question.

[Mikolaj]

Oh, yes, it's absolutely possible. cabal does that for you. Just mention this library in build-depends of all the projects. cabal will, as needed, build this library for the first time, use an already built version, build another copy with different dependency versions if some of the projects enforce it, etc. This should be well enough explained in https://cabal.readthedocs.io/en/stable/ and if it's not, please file a documentation improvement suggestion (just a ticket or PR in this repo).

[l8l]

@Mikolaj Perhaps I misunderstand the answer. But how is cabal supposed to find this library if it is not installed? It is just a local library which is not on hackage.

[Mikolaj]

That's a good question. Either take it from a local directory or directly from a git repo. Both are declared in a cabal.project file. Let us know if the syntax for that is too hard to find in the manual once the user knows what to look for.

[l8l]

@Mikolaj Thanks. With your hint, I now created a cabal.project file and inserted

packages: ./ ./../mylibrary

and now the project builds correctly.

However, this procedure requires me to specify a new relative path, or to copy an absolute path to my library around everytime I want to use it. Originally, I had hoped that running cabal install --lib in the library folder would automatically specify the path to the library once in some global cabal config file and then build this library only once such that I can just use it as any other library from hackage, simply by listing it under build-depends without specifying a path everytime or creating a project-file. Is that possible as well?

[yaxu]

Again when the majority of your work is in the REPL, this is not just about 'convenience'.

I agree that having to think ahead to manually make a text file containing all the libraries you are going to use in a session really gets in the way of getting things done. It's often just not possible - you might not know what libraries you will need until halfway through the day.

[Mikolaj]

I'm sure you are both right about your use cases. I don't have any similar workflows so can't advise.

@l8l: Does the first ticket or any ticket at https://github.com/haskell/cabal/projects/10 offer any help? E.g., would cabal-env make your workflow less impossible to sustain? Alternatively, there's v1- but it's unmaintained.

[andreasabel]

cabal v1-install mylib still works fine for me if I want GHC to know about it and don't want to write cabal/project files.

[fgaz]

Using globally installed packages is possible, with v1- or a combination of the new --package-db flag and Setup.hs, but I'd strongly advise against it. It will break your install plans and package databases. That's what the old "cabal hell" was about.

Instead, I think there are alternatives suitable to all your use cases:

• Source based techniques. Much more flexible and safe, eg. the package can be automatically rebuilt against dependencies specific to the project. To use your package as a dependency in another package use one of these:
  • To use two local packages together, put them in cabal.project under packages: , like you did in How to correctly install a locally-built library in Cabal 3? #6478 (comment)
  • To use a remote package that's not from hackage, use source-repository-package
  • To have a local package that you use frequently available in all your other projects, like you describe in How to correctly install a locally-built library in Cabal 3? #6478 (comment), sdist the package and put it in a local noindex repository
    • You could even use the dist-newstyle/sdist directory of the dependency as local noindex repo, to avoid the hassle of copying the sdist around, like another used did in confusion to use Local no-index repositories. parse error when copying from doc. #8425
• Environment files. Used to set up an environment with specific packages for direct use with ghc(i). Useful for experiments and debugging. Do NOT use this if you want to provide dependencies for a project. It won't be picked up. These are created with cabal install --lib or with tools like cabal-env. I think @yaxu's use case is better served by these.

[ulysses4ever]

I have a dream: the current cabal leadership convenes and decides where to put the marvelous text above in the user manual.

Also, I whole-heartedly support suggestion to avoid v1-install at all costs. Even if this requires you to learn a field or two of cabal.project or how GHC environments work. This is easier than it sounds. v1 is legacy and should be expelled imo.

[yaxu]

As above, cabal.project doesn't work well for my use case, and package environments constantly break. Until there is something that works better than v1-install, please don't remove it.

[donaldcallen]

I confess to not having read this thread in detail, but I would like to mention that I solved my confusion about dealing with local libraries with a cabal.project file. It really took me a while -- too long -- to dig this out of what passes for documentation of cabal (I got a big assist from searching, which turned up good advice from people who had been down this road). I hope this is helpful.

…

On Wed, Aug 24, 2022 at 11:09 AM Artem Pelenitsyn ***@***.***> wrote: I have a dream: the current cabal leadership convenes and decides where to put the marvelous text above in the user manual. Also, I whole-heartedly support suggestion to avoid v1-install at all costs. Even if this requires you to learn a field or two of cabal.project or how GHC environments work. This is easier than it sounds. v1 is legacy and should be expelled imo. — Reply to this email directly, view it on GitHub <#6478 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABGOGJYPLONK53F47AUF7T3V2Y3MFANCNFSM4KFT234A> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[yaxu]

Environment files. Used to set up an environment with specific packages for direct use with ghc(i). Useful for experiments and debugging. Do NOT use this if you want to provide dependencies for a project. It won't be picked up. These are created with cabal install --lib or with tools like cabal-env. I think @yaxu's use case is better served by these.

Yes this sounds good! Not only useful for experiments and debugging, but also using the libraries interactively, e.g. in live music performance.

[Mikolaj]

Would somebody please volunteer a PR to put @fgaz list into the https://cabal.readthedocs.io/en/stable manual? We can then work on the text together in the PR, not to miss anybody's perspective.

[l8l]

Thanks @fgaz for the detailed reply. Could you elaborate on your third option in your post? What does "sdist the package and put it in a local noindex repository" mean? And wouldn't it also be nice to have a very simple command that does that or provides some way to register a local library globally? I am not deep enough into the matter to understand why this is so complicated but I am somehow surprised that it is. Anyways, thanks again.

[juhp]

Using globally installed packages is possible, with v1- or a combination of the new --package-db flag and Setup.hs, but I'd strongly advise against it. It will break your install plans and package databases.

Is that actually true? v1 installation (ie using Setup) is what Linux distros still do for Haskell libraries and I think it actually plays nicely with v2 - best of both worlds in my opinion. But I am interested in hearing counter-examples and may be forgetting something.

[andreasabel]

@ulysses4ever wrote:

Also, I whole-heartedly support suggestion to avoid v1-install at all costs. Even if this requires you to learn a field or two of cabal.project or how GHC environments work. This is easier than it sounds. v1 is legacy and should be expelled imo.

v1-install is just too terribly useful to do GHC-only Haskell tinkering, so I do not want to see it go. For quick-and-dirty experiments, I don't want to write cabal files or set up GHC environments.

[Mikolaj]

I think Debian uses Setup.hs, not v1.

@andreasabel is ghc Setup.hs install (or whatever the exact command is; and you need to copy that default Setup.hs from somewhere) equally good for GHC-only Haskell tinkering? Does the v1 overlay add value in such context?

[andreasabel]

Personally, I never use the runghc Setup.hs method, I want to work at the abstraction of the package manager cabal. Also, this would require a download first (cabal get) with the trouble of manually deciding on a download location. cabal v1-install allows me to skip all that.

[Mikolaj]

Thank you. Let's hope v2 gets enough functionality at some point to retire v1.

[ivanperez-keera]

I have a dream: the current cabal leadership convenes and decides where to put the marvelous text above in the user manual.

Also, I whole-heartedly support suggestion to avoid v1-install at all costs. Even if this requires you to learn a field or two of cabal.project or how GHC environments work. This is easier than it sounds. v1 is legacy and should be expelled imo.

Note that this means we need to teach cabal and GHC environments to newcomers, including people who only intend to use an eDSL that happens to be embedded in Haskell.

Since DSLs are one of the main use cases of Haskell, I think we really need to support using runhaskell without having to create a cabal.project exposing a bunch of libraries that users know nothing about. Having to learn cabal or GHC environments at a deeper level is ofc out of the question if we mean to support such use cases.

I never use the runghc Setup.hs method, I want to work at the abstraction of the package manager cabal.

I use cabal v1-exec -- ghc Code.hs all the time (also with ghci and runhaskell). It exposes all packages installed (in the local dir or in my $HOME or globally, depending). For quick programs and especially for the use case I was talking about above, it's absolutely great.