build-tool-depends not in PATH for build-type: configure packages

[bgamari]

Describe the bug
build-tool-depends executables are not included in PATH when running executable/testsuite components of packages with build-type: configure.

To Reproduce
Reproducer available here.

$ git clone https://gitlab.haskell.org/bgamari/build-tool-repro
$ cd build-tool-repro
$ cabal test
...
Running 1 test suites...
Test suite build-tool-repro: RUNNING...
build-tool-repro: build-tool-repro-child: callProcess: posix_spawnp: does not exist (No such file or directory)

Test suite build-tool-repro: FAIL
Test suite logged to:
/mnt/data/exp/haskell-packages/process/hi/dist-newstyle/build/x86_64-linux/ghc-9.4.8/build-tool-repro-0.1.0.0/test/build-tool-repro-0.1.0.0-build-tool-repro.log
0 of 1 test suites (0 of 1 test cases) passed.
Error: cabal: Tests failed for build-tool-repro-0.1.0.0.

Expected behavior
The test passes.

System information

• Linux at very least
• cabal-install 3.10.2.1

[sheaf]

I can reproduce this issue. I investigated a bit and I think a good starting point would be to look at Distribution.Simple.Build.addInternalBuildTools. Does it get called when using autoconfUserHooks? Are some programs missing?

[alt-romes]

I can reproduce this issue. I investigated a bit and I think a good starting point would be to look at Distribution.Simple.Build.addInternalBuildTools. Does it get called when using autoconfUserHooks? Are some programs missing?

I checked that the programs are all there, and if you had access to the configured program database you would be able to runProgram the build-tool from the testsuite. I'm looking into it.

[alt-romes]

I managed to reproduce this with cabal run build-tool-repro (the test name), however, with cabal test I get:

$ cabal test
Warning: this is a debug build of cabal-install with assertions enabled.
Resolving dependencies...
Build profile: -w ghc-9.8.2 -O1
In order, the following will be built (use -v for more details):
 - build-tool-repro-0.1.0.0 (first run)
Warning: this is a debug build of cabal-install with assertions enabled.
Configuring build-tool-repro-0.1.0.0...
configured
Warning: this is a debug build of cabal-install with assertions enabled.
Preprocessing executable 'build-tool-repro-child' for build-tool-repro-0.1.0.0...
Building executable 'build-tool-repro-child' for build-tool-repro-0.1.0.0...
[1 of 1] Compiling Main             ( Main.hs, /private/var/folders/tv/35hlch6s3y15hfvndc71l6d40000gn/T/tmp.pLSkfl1LAW/build-tool-repro/dist-newstyle/build/aarch64-osx/ghc-9.8.2/build-tool-repro-0.1.0.0/build/build-tool-repro-child/build-tool-repro-child-tmp/Main.o )
[2 of 2] Linking /private/var/folders/tv/35hlch6s3y15hfvndc71l6d40000gn/T/tmp.pLSkfl1LAW/build-tool-repro/dist-newstyle/build/aarch64-osx/ghc-9.8.2/build-tool-repro-0.1.0.0/build/build-tool-repro-child/build-tool-repro-child
Preprocessing test suite 'build-tool-repro' for build-tool-repro-0.1.0.0...
Building test suite 'build-tool-repro' for build-tool-repro-0.1.0.0...
[1 of 1] Compiling Main             ( test/Main.hs, /private/var/folders/tv/35hlch6s3y15hfvndc71l6d40000gn/T/tmp.pLSkfl1LAW/build-tool-repro/dist-newstyle/build/aarch64-osx/ghc-9.8.2/build-tool-repro-0.1.0.0/build/build-tool-repro/build-tool-repro-tmp/Main.o )
[2 of 2] Linking /private/var/folders/tv/35hlch6s3y15hfvndc71l6d40000gn/T/tmp.pLSkfl1LAW/build-tool-repro/dist-newstyle/build/aarch64-osx/ghc-9.8.2/build-tool-repro-0.1.0.0/build/build-tool-repro/build-tool-repro
Warning: this is a debug build of cabal-install with assertions enabled.
Error: [Cabal-9341]
Failed to find the installed unit 'build-tool-repro-0.1.0.0-inplace' in package database stack.

Error: [Cabal-7125]
Tests failed for build-tool-repro-0.1.0.0.

Which hints at a different problem?

[alt-romes]

The root of the problem seems to be that with build-type: Configure, just like with build-type: Custom, we will configure the entire package (executable and test) all at once (with the ./Setup configure ... interface). This implies that to augment the PATH with build-tool-repro-child when running the testsuite executable, the PATH when building the executable build-tool-repro-child must already contain build-tool-repro-child. This may not necessarily be a problem in practice though.

To clarify, the PATH used at runtime is extended when configuring a package via the --extra-prog-path Cabal flag.

We can see this discrepancy if we have build-type: Custom and depend on an external build tool (i.e. in another package). The verbose output prints for some build-tool-depends: dep:dep :

Including the following directories in PATH:
- /private/var/folders/tv/35hlch6s3y15hfvndc71l6d40000gn/T/tmp.s9U5pxhe6c/build-tool-repro/dist-newstyle/build/aarch64-osx/ghc-9.8.2/dep-0.1.0.0/x/dep/build/dep
- /Users/romes/.cabal/bin

Whereas, if the build-tool-depends is from the same package where testsuite is, we will instead just see:

Including the following directories in PATH:
- /Users/romes/.cabal/bin

[andreabedini]

Is this related to the issue in #8421?