Install packages to the snapshot database with profiling when they are present in the global database without profiling

[DanBurton]

Example: The data-default-class package is installed in the global database without profiling. My project depends on the cookie package, but when stack tries to install it with profiling into a snapshot database, it ends in failure:

Web/Cookie.hs:54:8:
    Could not find module ‘Data.Default.Class’
    Perhaps you haven't installed the profiling libraries for package ‘data-default-class-0.0.1@datad_9ouboI9uirTF3xjV57lONw’?
    Use -v to see a list of the files searched for. 

See #120 for details.

[chrisdone]

Researching the GHC-specific details for this (and relatedly #127):

• I already have data-default-class-0.0.1 installed in my local snapshot database.
• I install it also in my --global database without profiling.

I confirm that the global version doesn't have profiling:

chris@retina:~/Work/stack$ ghc-pkg describe --global data-default-class-0.0.1
name: data-default-class
version: 0.0.1
id: data-default-class-0.0.1-c90f2361289c9de494da7845c67d72e1
…
library-dirs: /usr/local/lib/x86_64-linux-ghc-7.8.4/data-default-class-0.0.1
chris@retina:~/Work/stack$ ls /usr/local/lib/x86_64-linux-ghc-7.8.4/data-default-class-0.0.1
Data  libHSdata-default-class-0.0.1.a  libHSdata-default-class-0.0.1-ghc7.8.4.so

So in getting the package list in Stack.GhcPkg, this package would be excluded. (Also, any package which was built with profiling must be built upon dependencies that have profiling. So if data-default-class had any dependencies, those also would show up under inspection as not being compiled with profiling either.) That would (hypothetically) cause the dependency installer to decide to install data-default-class (and any other non-profiled libraries) and go ahead and install with profiling.

And then we can see the snapshot one does have profiling:

chris@retina:~/Work/stack$ ghc-pkg describe -f ~/.stack/snapshots/x86_64-linux/lts-2.9/7.8.4/pkgdb/ -f .stack-work/install/x86_64-linux/lts-2.9/7.8.4/pkgdb/ data-default-class-0.0.1
name: data-default-class
version: 0.0.1
id: data-default-class-0.0.1-c90f2361289c9de494da7845c67d72e1
…
library-dirs: /home/chris/.stack/snapshots/x86_64-linux/lts-2.9/7.8.4/lib/x86_64-linux-ghc-7.8.4/data-default-class-0.0.1
chris@retina:~/Work/stack$ ls /home/chris/.stack/snapshots/x86_64-linux/lts-2.9/7.8.4/lib/x86_64-linux-ghc-7.8.4/data-default-class-0.0.1
Data                             libHSdata-default-class-0.0.1-ghc7.8.4.so
libHSdata-default-class-0.0.1.a  libHSdata-default-class-0.0.1_p.a

(Note: they both have the same package ID, so if we wanted to, it seems impossible to distinguish them.)

But seeing as we didn't touch the global database to remove the offending version, and in fact in reality in this scenario I actually installed data-class-default to the global database afterwards (so it is the latest), we should confirm that we're able to use the older snapshot one:

Just global:

chris@retina:~/Work/stack$ ghc --make X.hs -prof 
X.hs:2:8:
    Could not find module ‘Data.Default.Class’
    Perhaps you haven't installed the profiling libraries for package ‘data-default-class-0.0.1’?
    Use -v to see a list of the files searched for.

With snapshot db:

chris@retina:~/Work/stack$ ghc --make X.hs -prof -package-db /opt/ghc/7.8.4/lib/ghc-7.8.4/package.conf.d -package-db ~/.stack/snapshots/x86_64-linux/lts-2.9/7.8.4/pkgdb/ -package data-default-class-0.0.1
[1 of 1] Compiling X                ( X.hs, X.o )
chris@retina:~/Work/stack$ rm X.o
chris@retina:~/Work/stack$ ghc --make X.hs -prof -package-db ~/.stack/snapshots/x86_64-linux/lts-2.9/7.8.4/pkgdb/ -package-db /opt/ghc/7.8.4/lib/ghc-7.8.4/package.conf.d -package data-default-class-0.0.1
X.hs:2:8:
    Could not find module ‘Data.Default.Class’
    Perhaps you haven't installed the profiling libraries for package ‘data-default-class-0.0.1’?
    Use -v to see a list of the files searched for.
chris@retina:~/Work/stack$ 

So GHC chooses the packages based on the ordering of -package-db arguments, meaning we can indeed favour the snapshot release which has the proper profiling package.

[chrisdone]

@DanBurton Confirm fixed on master?

[DanBurton]

It appears that profiling libraries are no longer installed by default. I presume this is by design?

bash> stack --version
Version 0.0.0, Git revision 12a290029c3b758787958ddd793372c403e9b04f
bash> tree -L 1
.
├── dist-stack
├── prof.cabal
├── Setup.hs
├── stack.yaml
└── X.hs

1 directory, 4 files
bash> cat stack.yaml
resolver: lts-2.11
packages:
- .

I installed a non-profiling version of data-default-class into my global package database. (Strangely, cabal install --global didn't work, I had to give it the package-db argument instead because apparently the ghc I got from stackage-setup is in an unorthodox location.)

Note that data-default-class is not present in the snapshot package-db.

bash> stack exec -- ghc-pkg list data-default-class
/home/dan/.stackage/environment/ghc/ghc-7.8.4/lib/ghc-7.8.4/package.conf.d/
   data-default-class-0.0.1
/home/dan/.stack/snapshots/x86_64-linux/lts-2.11/7.8.4/pkgdb/
/home/dan/prof/.stack-work/install/x86_64-linux/lts-2.11/7.8.4/pkgdb/

I have a trivial file that pretends to depend on cookie.

bash> cat X.hs
import Web.Cookie
main = putStrLn "Hello, world"
bash> stack build # installs cookie to the snapshot db
bash> stack exec -- ghc-pkg list cookie
/home/dan/.stackage/environment/ghc/ghc-7.8.4/lib/ghc-7.8.4/package.conf.d/
/home/dan/.stack/snapshots/x86_64-linux/lts-2.11/7.8.4/pkgdb/
   cookie-0.4.1.5
/home/dan/prof/.stack-work/install/x86_64-linux/lts-2.11/7.8.4/pkgdb/

Now when we try to compile with profiling

bash> stack exec -- ghc --make X -prof

X.hs:1:8:
    Could not find module ‘Web.Cookie’
    Perhaps you haven't installed the profiling libraries for package ‘cookie-0.4.1.5’?
    Use -v to see a list of the files searched for.

So stack build does work, but it doesn't install profiling libraries by default.

bash> stack build --executable-profiling

This worked. (Yay!)

bash> stack exec -- ghc-pkg list data-default-class
/home/dan/.stackage/environment/ghc/ghc-7.8.4/lib/ghc-7.8.4/package.conf.d/
   data-default-class-0.0.1
/home/dan/.stack/snapshots/x86_64-linux/lts-2.11/7.8.4/pkgdb/
   data-default-class-0.0.1
/home/dan/prof/.stack-work/install/x86_64-linux/lts-2.11/7.8.4/pkgdb/