Test suite with a cyclic dependency no longer working

[tswelsh]

This was discovered when using stack, the relevant issue is here: commercialhaskell/stack#3892. Copying the summary from there:

We have a project with package A that exposes a library and has a test module. Package B depends on A and exposes a library that is used by A's test suite (the reason for this is that package B is used by other packages). So there is a circularish dependency going on, which works when building with GHC 8.0.2 but not GHC 8.2.2. Not sure if this is stack or cabal or GHC or what, but I thought that stack has code to deal with this situation (checking whether it can do an all in one build, I think?), so perhaps something changed in GHC or cabal that stops it from working.

Steps to reproduce

Minimal example here: https://github.com/dten/CouldntMatchTypeRepro. Changing the resolver in the stack.yaml between LTS 9 and LTS 10 exhibits the change in behaviour.

After poking around it seems like stack's behaviour hasn't changed. I tried to replicate what stack does under the hood with cabal install 2.0.0.0 on that project. Using cabal install in the model dir, then cabal install in the testing dir, then cabal configure --enable-tests && cabal build in the model dir gives the same error as when using stack, as expected. I haven't used the new-* ommands much before, but making a basic cabal.project and trying cabal new-configure in the model directory reuslts in:

Warning: The package list for 'hackage.haskell.org' is 97 days old.
Run 'cabal update' to get the latest list of available packages.
Resolving dependencies...
cabal.exe: Could not resolve dependencies:
trying: not-working-model-0.0.0 (user goal)
next goal: not-working-testing (user goal)
rejecting: not-working-testing-0.1.0.0 (cyclic dependencies; conflict set:
not-working-model, not-working-testing)
fail (backjumping, conflict set: not-working-model, not-working-testing)
After searching the rest of the dependency tree exhaustively, these were the
goals I've had most trouble fulfilling: not-working-model, not-working-testing

(I changed the names because it looks like model is a package already on Hackage, so didn't want to confuse the matter). Is this a supported project configuration/workflow? Will it be in the future? It felt quite natural and useful to do at the time.

[hvr]

related: #3422 & #1575

[tswelsh]

Ah OK, this has been raised before. The fairly recent comment here #1575 (comment) says:

cabal can't build the tests in one step, but it is possible to install foo, install bar, and then build foo with tests enabled. cabal new-build can't build the tests, though, because it can't install packages in one step and then use the installed packages as inputs when calculating the next install plan.

While using new-build failed as this comment suggests, it looks like the first part about installing things in order no longer works. Should this be considered a separate bug?

[23Skidoo]

Closing in favour of #3422 and #1575. This use case is not yet supported by new-build, but I believe that legacy-style sandboxes will still work.

[grayjay]

I looked into the old-build part of this issue. I can reproduce the behavior change between cabal-install-1.24 and cabal-install-2.0 by running the following commands with GHC 8.0.2 and no package index:

cd model/
cabal install
cd ../testing/
cabal install
cd ../model/
cabal configure --enable-tests
cabal build

cabal gives a warning when configuring model in both cases but continues:

Warning: The package list for 'hackage.haskell.org' does not exist. Run 'cabal
update' to download it.
Resolving dependencies...
Warning: solver failed to find a solution:
Could not resolve dependencies:
rejecting: model-0.0.0:!test (constraint from config file, command line flag,
or user target requires opposite flag selection)
trying: model-0.0.0:*test
next goal: testing (dependency of model-0.0.0:*test)
rejecting: testing-0.1.0.0/installed-H0e... (package is broken)
After searching the rest of the dependency tree exhaustively, these were the
goals I've had most trouble fulfilling: model, model-0.0.0:test, testing
Trying configure anyway.
Configuring model-0.0.0...

Then cabal build succeeds with cabal from the 1.24 branch:

Building model-0.0.0...
Preprocessing library model-0.0.0...
Preprocessing test suite 'test' for model-0.0.0...
[1 of 1] Compiling Main             ( test/Spec.hs, dist/build/test/test-tmp/Main.o )
Linking dist/build/test/test ...

The same command fails with cabal from the 2.0 branch:

Preprocessing library for model-0.0.0..
Building library for model-0.0.0..
[1 of 1] Compiling Types.Ids        ( src/Types/Ids.hs, dist/build/Types/Ids.o )
Preprocessing test suite 'test' for model-0.0.0..
Building test suite 'test' for model-0.0.0..
[1 of 2] Compiling NotWorking       ( test/NotWorking.hs, dist/build/test/test-tmp/NotWorking.o )

test/NotWorking.hs:7:13: error:
    • Couldn't match expected type ‘MyId’
                  with actual type ‘model-0.0.0@model-0.0.0-3PRplodnnvDBYrvoS4XINO:Types.Ids.MyId’
      NB: ‘model-0.0.0@model-0.0.0-3PRplodnnvDBYrvoS4XINO:Types.Ids.MyId’
            is defined in ‘Types.Ids’
                in package ‘model-0.0.0@model-0.0.0-3PRplodnnvDBYrvoS4XINO’
          ‘MyId’
            is defined in ‘Types.Ids’
                in package ‘model-0.0.0@model-0.0.0-GIZucc00QVGIRtXyMfDLQ8’
    • In the expression: makeId
      In an equation for ‘iDontWork’: iDontWork = makeId
[2 of 2] Compiling Main             ( test/Spec.hs, dist/build/test/test-tmp/Main.o )

The model test suite references the library's MyId type directly and through the testing package's makeId function, in the function iDontWork:

iDontWork :: MyId
iDontWork = makeId

I think that the cabal-install-2.0 error message means that the type of makeId from the installed model doesn't match the MyId type from the source model. I don't think that is a bug, since model could be configured differently in the two commands. In fact, cabal-install-1.24 doesn't handle the project correctly either, because when I modify Types.Ids between installing model and building the tests, I get a link error (I also needed to add a reference to the NotWorking module from Main, because NotWorking was unused.):

Building model-0.0.0...
Preprocessing library model-0.0.0...
[1 of 1] Compiling Types.Ids        ( src/Types/Ids.hs, dist/build/Types/Ids.o )
Preprocessing test suite 'test' for model-0.0.0...
[1 of 2] Compiling NotWorking       ( test/NotWorking.hs, dist/build/test/test-tmp/NotWorking.o )
/home/kristen/.cabal/lib/x86_64-linux-ghc-8.0.2/testing-0.1.0.0-H0eQZRk21rm7mNxzhdxSqb/Test/Ids.hi
Declaration for makeId
Unfolding of makeId:
  Can't find interface-file declaration for variable MyId
    Probable cause: bug in .hi-boot file, or inconsistent .hi file
    Use -ddump-if-trace to get an idea of which file caused the error
[2 of 2] Compiling Main             ( test/Spec.hs, dist/build/test/test-tmp/Main.o )
Linking dist/build/test/test ...
/home/kristen/.cabal/lib/x86_64-linux-ghc-8.0.2/testing-0.1.0.0-H0eQZRk21rm7mNxzhdxSqb/libHStesting-0.1.0.0-H0eQZRk21rm7mNxzhdxSqb.a(Ids.o):(.data+0x0): undefined reference to `modelzm0zi0zi0zm3PRplodnnvDBYrvoS4XINO_TypesziIds_MyId_static_info'
collect2: error: ld returned 1 exit status
`cc' failed in phase `Linker'. (Exit code: 1)

I think that the cyclic dependency in CouldntMatchTypeRepro is different from the most common use case in #1575, such as the containers or bytestring test suites, where the library's types aren't exposed through the API of the test helper. Packages like bytestring could use an older version of the package to build the test framework while testing the source version of the library. To support building CouldntMatchTypeRepro in one command, we would need to build the model library, build testing, and then build the model test suite.

I also tested the use case where the library types don't need to match by building the bytestring test suite with cabal HEAD, and it still worked.