ctest -N does not work with custom cmake path from preset

[rettenbs]

Brief Issue Summary

We have a custom CMake installation. The PATH environment variable is not configured and does not contain cmake or ctest.

cmakeExecutable in CMakePreset.json is set correctly.

Everything work, except for ctest:

[proc] Executing command: ctest -T test --output-on-failure
[proc] The command: ctest -T test --output-on-failure failed with error: Error: spawn ctest ENOENT
[proc] The command: ctest -N failed with error: Error: spawn ctest ENOENT
[ctest] There was an error running ctest to determine available test executables

I could get the tests running by adding the CMake bin folder to environment.PATH in the preset:

[proc] Executing command: ctest -T test --output-on-failure
<...>
[ctest]     Start 1: HelloTest.BasicAssertions
[ctest] 1/1 Test #1: HelloTest.BasicAssertions ........   Passed    0.03 sec
[ctest] 
[ctest] 100% tests passed, 0 tests failed out of 1
[ctest] 
[ctest] Total Test time (real) =   0.06 sec

However, determining the available tests using ctest -N still fails.

I found a difference in code:

const child = driver.executeCommand(
            ctestpath,
            ctestArgs,
            ((customizedTask && consumer) ? consumer : new CTestOutputLogger()),
            { environment: await driver.getCTestCommandEnvironment(), cwd: driver.binaryDir });

const result = await driver.executeCommand(ctestpath, ['-N', ...buildConfigArgs], undefined, { cwd: driver.binaryDir, silent: true }).result;

It looks like ctestpath is not detected correctly in both cases but due to environment: await driver.getCTestCommandEnvironment() it still works if the preset environment contains cmake.

CMake Tools Diagnostics

{
  "os": "win32",
  "vscodeVersion": "1.72.2",
  "cmtVersion": "1.12.27",
  "configurations": [
    {
      "folder": "<...>",
      "cmakeVersion": "3.24.2",
      "configured": true,
      "generator": "Ninja",
      "usesPresets": true,
      "compilers": {
        "C": "<...>/gcc.exe",
        "CXX": "<...>/c++.exe"
      }
    }
  ],
  "cpptoolsIntegration": {
    "isReady": true,
    "hasCodeModel": true,
    "activeBuildType": "Release",
    "buildTypesSeen": [
      "Release"
    ],
    "requests": [],
    "responses": [],
    "partialMatches": [],
    "targetCount": 33,
    "executablesCount": 1,
    "librariesCount": 4,
    "targets": []
  },
  "settings": [
    {
      "communicationMode": "automatic",
      "useCMakePresets": "auto",
      "configureOnOpen": null
    }
  ]
}

Debug Log

No response

Additional Information

I shorten some paths in the diagnostics output. I do not think they are relevant for the issue.

[elahehrashedi]

Can you try and run the command in the terminal and see if it works?

[bobbrow]

In other words, are you able to run cmake with your test preset in the terminal without our extension? (e.g. cmake --preset mytestpreset). Or maybe that works because you are already in the directory where cmake.exe lives when you run that command? What about if the working directory is not the same as the directory where cmake.exe lives?

[rettenbs]

I cannot run cmake from the terminal (unless I manually adapt the PATH variable).

I use the cmakeExecutable attribute in the preset to tell Visual Studio Code where to find cmake:

{
    "version": 5,
    "cmakeMinimumRequired": {
      "major": 3,
      "minor": 23,
      "patch": 0
    },
    "configurePresets": [
        {
            "name": "Release",
            "generator": "Ninja",
            "binaryDir": "${sourceDir}/build/Release",
            "cmakeExecutable": "C:\\path\\to\\cmake\\cmake.exe",
            "cacheVariables": {
                "CMAKE_BUILD_TYPE": "Debug",
                "CMAKE_EXPORT_COMPILE_COMMANDS": "ON"
            },
            "environment": {
                "PATH": "C:\\path\\to\\ninja;$penv{PATH}"
            }
        }
    ],
   <...>
}

If I change the preset to the following, the execution of the tests from Visual Studio Code is possible but ctest -N still fails:

{
    "version": 5,
    "cmakeMinimumRequired": {
      "major": 3,
      "minor": 23,
      "patch": 0
    },
    "configurePresets": [
        {
            "name": "Release",
            "generator": "Ninja",
            "binaryDir": "${sourceDir}/build/Release",
            "cmakeExecutable": "C:\\path\\to\\cmake\\cmake.exe",
            "cacheVariables": {
                "CMAKE_BUILD_TYPE": "Debug",
                "CMAKE_EXPORT_COMPILE_COMMANDS": "ON"
            },
            "environment": {
                "PATH": "C:\\path\\to\\cmake;C:\\path\\to\\ninja;$penv{PATH}"
            }
        }
    ],
   <...>
}

[bobbrow]

Thank you for the extra information. I found the problem and will submit a fix soon.