[CMake] Meson not properly respecting CMAKE_PREFIX_PATH when searching for dependencies on windows #7323

p01arst0rm · 2020-06-14T21:15:19Z

Describe the bug
When searching for dependencies on windows, Meson is not capable of finding
an installed dependency with CMake. Native CMake is capable of finding the library
when -DCMAKE_PREFIX_PATH= is supplied at build time.

To Reproduce
The current issue includes LibKF5Attica but i suspect this isnt to do with
the library. The following can be found in build/meson-logs/meson-log.txt

CMake binary for MachineChoice.HOST is cached.
Using 'CMAKE_PREFIX_PATH' from environment with value: '"C:\\Program Files (x86)\\Attica"'
Preliminary CMake check failed. Aborting.
Run-time dependency libkf5attica found: NO (tried pkgconfig and cmake)


CMake binary for MachineChoice.HOST is cached.
Using 'CMAKE_PREFIX_PATH' from environment with value: '"C:\\Program Files (x86)\\Attica"'
Preliminary CMake check failed. Aborting.
Run-time dependency kf5attica found: NO (tried pkgconfig and cmake)

Path enumerations have been tried, but the following provided path,
-DCMAKE_PREFIX_PATH="C:\Program Files (x86)\Attica" allows the native CMake build to locate
the dependency.

Expected behavior
Find the cmake dependency using CMAKE_PREFIX_PATH

system parameters
native build
Windows 10 (Current patch KB4538461)
python version 3.7.1
meson version 0.54.1
ninja Not Applicable.

The text was updated successfully, but these errors were encountered:

mensinda · 2020-06-14T21:16:50Z

could you please retry this test with the current meson master? Some fixes for CMAKE_PREFIX_PATH went in recently.

p01arst0rm · 2020-06-14T21:17:01Z

ok c:

p01arst0rm · 2020-06-14T21:26:49Z

@mensinda no luck, still hasnt found it. looking into it a bit more atm.

p01arst0rm · 2020-06-14T21:32:40Z

not sure if this is relevant, but the output of the build doesn't seem to be respecting case in the library names;

Using 'CMAKE_PREFIX_PATH' from environment with value: '"C:\\Program Files (x86)\\attica"'
Run-time dependency libkf5attica found: NO (tried pkgconfig and cmake)
Using 'CMAKE_PREFIX_PATH' from environment with value: '"C:\\Program Files (x86)\\attica"'
Run-time dependency kf5attica found: NO (tried pkgconfig and cmake)

should be Lib[KF5Attica]

mensinda · 2020-06-14T21:35:01Z

What is the directory structure in C:\\Program Files (x86)\\attica?

p01arst0rm · 2020-06-14T21:46:27Z

@mensinda see attached

mensinda · 2020-06-14T21:58:05Z

could you please try applying

diff --git a/mesonbuild/dependencies/base.py b/mesonbuild/dependencies/base.py
index eed714a83..c08891dff 100644
--- a/mesonbuild/dependencies/base.py
+++ b/mesonbuild/dependencies/base.py
@@ -1203,10 +1203,13 @@ class CMakeDependency(ExternalDependency):
 
         # Checks <path>, <path>/cmake, <path>/CMake
         def find_module(path: str) -> bool:
+            mlog.debug('R: {}'.format(path))
             for i in [path, os.path.join(path, 'cmake'), os.path.join(path, 'CMake')]:
                 if not self._cached_isdir(i):
                     continue
 
+                mlog.debug('MOD: {}'.format(i))
+
                 # Check the directory case insensitve
                 content = self._cached_listdir(i)
                 candidates = ['Find{}.cmake', '{}Config.cmake', '{}-config.cmake']
@@ -1217,10 +1220,13 @@ class CMakeDependency(ExternalDependency):
 
         # Search in <path>/(lib/<arch>|lib*|share) for cmake files
         def search_lib_dirs(path: str) -> bool:
+            mlog.debug('CP: {}'.format(self.cmakeinfo['common_paths']))
             for i in [os.path.join(path, x) for x in self.cmakeinfo['common_paths']]:
                 if not self._cached_isdir(i):
                     continue
 
+                mlog.debug('LD: {}:'.format(i))
+
                 # Check <path>/(lib/<arch>|lib*|share)/cmake/<name>*/
                 cm_dir = os.path.join(i, 'cmake')
                 if self._cached_isdir(cm_dir):
@@ -1234,6 +1240,7 @@ class CMakeDependency(ExternalDependency):
                 # <path>/(lib/<arch>|lib*|share)/<name>*/(cmake|CMake)/
                 content = self._cached_listdir(i)
                 content = list(filter(lambda x: x[1].startswith(lname), content))
+                mlog.debug(content)
                 for k in content:
                     if find_module(os.path.join(i, k[0])):
                         return True

and uploading the resulting meson-log.txt ?

p01arst0rm · 2020-06-14T22:07:46Z

absolutely, on it c:

p01arst0rm · 2020-06-14T22:29:40Z

@mensinda see attached :)

terminal.txt
meson-log.txt

mensinda · 2020-06-14T22:33:20Z

OK, thanks. Looks like CMAKE_PREFIX_PATH is ignored for some reason. I will have to look into this tomorrow...

p01arst0rm · 2020-06-14T22:34:53Z

ok! awesome thanks :)

p01arst0rm · 2020-06-17T02:05:12Z

hey! any updates on this? ^^

mensinda · 2020-06-17T11:33:12Z

It's on my todo, but I have an exam Friday.

p01arst0rm · 2020-06-17T13:19:11Z

@mensinda mind if i give it a go? ;)

mensinda · 2020-06-17T13:21:01Z

Sure, feel free to.

p01arst0rm · 2020-06-18T17:55:32Z

@mensinda dumped the stack into the log file at time of dep search, looking through to see where it should be getting the path from :)

p01arst0rm · 2020-06-18T18:00:52Z

modification in mesonbuild/dependencies/base.py

2366|    for c in candidates:
2367|        mlog.debug('TRYC: {}'.format(c))

result:

TRYC: 
functools.partial(<class 'mesonbuild.dependencies.base.CMakeDependency'>,
'KF5Attica',
<mesonbuild.environment.Environment object at 0x0393B3D0>, 
{'required': False, 'cmake_module_path': 'CMakeModules/', 'version': '>= 5.6.0'})

p01arst0rm · 2020-06-18T18:02:11Z

looks like its not being fed the correct module path upstream from _preliminary_find_check()

p01arst0rm · 2020-06-18T18:09:46Z

modification in mesonbuild/dependencies/base.py

1090 |        
1093 +        cm_args.append('-DCMAKE_PREFIX_PATH="C:\Program Files (x86)\Attica"')
1094 |        if cm_path:
1095 |            cm_args.append('-DCMAKE_MODULE_PATH=' + ';'.join(cm_path))

hardcoded dependency path to candidate, resulting in:

Determining dependency 'KF5Attica' with CMake executable 'C:\\Program Files\\CMake\\bin\\cmake.EXE'
Try CMake generator: auto
None of 'CC' are defined in the environment, not changing global flags.
None of 'CFLAGS' are defined in the environment, not changing global flags.
None of 'LDFLAGS' are defined in the environment, not changing global flags.
None of 'CPPFLAGS' are defined in the environment, not changing global flags.
None of 'CC_LD' are defined in the environment, not changing global flags.
None of 'FC' are defined in the environment, not changing global flags.
None of 'FFLAGS' are defined in the environment, not changing global flags.
None of 'LDFLAGS' are defined in the environment, not changing global flags.
None of 'FC_LD' are defined in the environment, not changing global flags.
None of 'F_LD' are defined in the environment, not changing global flags.
Called `C:\Program Files\CMake\bin\cmake.EXE -DNAME=KF5Attica -DARCHS= -DCOMPS= -DCMAKE_PREFIX_PATH="C:\Program Files (x86)\Attica" -DCMAKE_MODULE_PATH=C:\Users\p01ar\git\tomahawk\CMakeModules/ --trace-expand --trace-format=json-v1 --no-warn-unused-cli --trace-redirect=cmake_trace.txt . -DCMAKE_PREFIX_PATH="C;\Program Files (x86)\attica"` in C:\Users\p01ar\git\tomahawk\build\meson-private\cmake_KF5Attica -> 0
Run-time dependency kf5attica found: NO (tried pkgconfig and cmake)

p01arst0rm · 2020-06-18T18:16:21Z

Called `C:\Program Files\CMake\bin\cmake.EXE -DNAME=libKF5Attica -DARCHS= -DCOMPS= -DCMAKE_PREFIX_PATH="C:\Program Files (x86)\Attica" -DCMAKE_MODULE_PATH=C:\Users\p01ar\git\tomahawk\CMakeModules/ --trace-expand --trace-format=json-v1 --no-warn-unused-cli --trace-redirect=cmake_trace.txt . -DCMAKE_PREFIX_PATH="C;\Program Files (x86)\attica"` in C:\Users\p01ar\git\tomahawk\build\meson-private\cmake_libKF5Attica -> 0

my first -DCMAKE_PREFIX_PATH="C:\Program Files (x86)\Attica" added perfectly fine butttt
HMMMM that -DCMAKE_PREFIX_PATH="C;\Program Files (x86)\attica" that i didn't add has a SUSPICIOUS LOOKING C;\ IN IT :p

p01arst0rm · 2020-06-18T18:16:52Z

it's also ignoring the capitalisation of the path...

p01arst0rm · 2020-06-18T19:04:31Z

when running the meson-private/cmake_KF5Attica project with the provided shell command in meson-log.txt

`C:\Program Files\CMake\bin\cmake.EXE -DNAME=KF5Attica -DARCHS= -DCOMPS= -DCMAKE_PREFIX_PATH="C:\Program Files (x86)\Attica" --trace-expand --trace-format=json-v1 --no-warn-unused-cli --trace-redirect=cmake_trace.txt . -DCMAKE_PREFIX_PATH="C;\Program Files (x86)\attica"` in C:\Users\p01ar\git\tomahawk\build\meson-private\cmake_KF5Attica -> 0

CMake successfully found the project; it also seems to work with C;\ as the drive name, which is strange but kind of cool. It is getting delivered a working CMAKE_PREFIX_PATH to the build check, though

@mensinda

p01arst0rm · 2020-06-18T19:07:09Z

seems this may be related to #7305

p01arst0rm · 2020-06-18T19:15:54Z

'traced the issue down to '_detect_dep()' in 'mesonbuild/dependencies/base.py'.

TRYC: functools.partial(<class 'mesonbuild.dependencies.base.CMakeDependency'>, 'KF5Attica', <mesonbuild.environment.Environment object at 0x0397B390>, {'required': False, 'cmake_module_path': 'CMakeModules/', 'version': '>= 5.6.0'})
CMake binary for MachineChoice.HOST is not cached
None of 'CMAKE' are defined in the environment, not changing global flags.
CMake binary missing from cross or native file, or env var undefined.
Trying a default CMake fallback at cmake
Found CMake: C:\Program Files\CMake\bin\cmake.EXE (3.17.3)
Using 'CMAKE_PREFIX_PATH' from environment with value: '"C:\\Program Files (x86)\\attica"'
Extracting basic cmake information
_get_cmake_info
Try CMake generator: auto
None of 'CC' are defined in the environment, not changing global flags.
None of 'CFLAGS' are defined in the environment, not changing global flags.
None of 'LDFLAGS' are defined in the environment, not changing global flags.
None of 'CPPFLAGS' are defined in the environment, not changing global flags.
None of 'CC_LD' are defined in the environment, not changing global flags.
None of 'FC' are defined in the environment, not changing global flags.
None of 'FFLAGS' are defined in the environment, not changing global flags.
None of 'LDFLAGS' are defined in the environment, not changing global flags.
None of 'FC_LD' are defined in the environment, not changing global flags.
None of 'F_LD' are defined in the environment, not changing global flags.

Called `C:\Program Files\CMake\bin\cmake.EXE --trace-expand --trace-format=json-v1 --no-warn-unused-cli --trace-redirect=cmake_trace.txt .` in C:\Users\p01ar\git\tomahawk\build\meson-private\cmake_KF5Attica -> 0
  -- Module search paths:    ['/Program Files', '/Program Files (x86)', '/Program Files/CMake']
  -- CMake root:             C:/Program Files/CMake/share/cmake-3.17
  -- CMake architectures:    []
  -- CMake lib search paths: ['lib', 'lib32', 'lib64', 'libx32', 'share']


CL: __init__
CP: ['lib', 'lib32', 'lib64', 'libx32', 'share']
R: C:\Users\p01ar\git\tomahawk\CMakeModules/
MOD: C:\Users\p01ar\git\tomahawk\CMakeModules/

Determining dependency 'KF5Attica' with CMake executable 'C:\\Program Files\\CMake\\bin\\cmake.EXE'
_detect_dep
Try CMake generator: auto
None of 'CC' are defined in the environment, not changing global flags.
None of 'CFLAGS' are defined in the environment, not changing global flags.
None of 'LDFLAGS' are defined in the environment, not changing global flags.
None of 'CPPFLAGS' are defined in the environment, not changing global flags.
None of 'CC_LD' are defined in the environment, not changing global flags.
None of 'FC' are defined in the environment, not changing global flags.
None of 'FFLAGS' are defined in the environment, not changing global flags.
None of 'LDFLAGS' are defined in the environment, not changing global flags.
None of 'FC_LD' are defined in the environment, not changing global flags.
None of 'F_LD' are defined in the environment, not changing global flags.
Called `C:\Program Files\CMake\bin\cmake.EXE -DNAME=KF5Attica -DARCHS= -DCOMPS= -DCMAKE_PREFIX_PATH="C:\Program Files (x86)\Attica" --trace-expand --trace-format=json-v1 --no-warn-unused-cli --trace-redirect=cmake_trace.txt .` in C:\Users\p01ar\git\tomahawk\build\meson-private\cmake_KF5Attica -> 0
Run-time dependency kf5attica found: NO (tried pkgconfig and cmake)

p01arst0rm · 2020-06-18T19:22:25Z

added the following to 'mesonbuild/dependencies/base.py`

1351  |          # Current generator was successful
1352  |          if ret1 == 0:
1353  +              mlog.debug('generator successful!')
1354  |              CMakeDependency.class_working_generator = i
1355  |              break

resulting log was

Try CMake generator: auto
None of 'CC' are defined in the environment, not changing global flags.
None of 'CFLAGS' are defined in the environment, not changing global flags.
None of 'LDFLAGS' are defined in the environment, not changing global flags.
None of 'CPPFLAGS' are defined in the environment, not changing global flags.
None of 'CC_LD' are defined in the environment, not changing global flags.
None of 'FC' are defined in the environment, not changing global flags.
None of 'FFLAGS' are defined in the environment, not changing global flags.
None of 'LDFLAGS' are defined in the environment, not changing global flags.
None of 'FC_LD' are defined in the environment, not changing global flags.
None of 'F_LD' are defined in the environment, not changing global flags.
Called `C:\Program Files\CMake\bin\cmake.EXE -DNAME=KF5Attica -DARCHS= -DCOMPS= -DCMAKE_PREFIX_PATH="C:\Program Files (x86)\Attica" --trace-expand --trace-format=json-v1 --no-warn-unused-cli --trace-redirect=cmake_trace.txt .` in C:\Users\p01ar\git\tomahawk\build\meson-private\cmake_KF5Attica -> 0
generator successful!

the cmake build isnt the reason the dependency isnt being found.

p01arst0rm · 2020-06-18T20:40:25Z

ok so its creating the temp build project passing it in the correct shell command; yet meson cannot find it, and native cmake can. intresting.

p01arst0rm · 2020-06-22T22:57:25Z

I think ive found the issue.. when you pass a single item to CMAKE_PREFIX_PATH on windows, it works perfectly fine. but, if you give it multiple paths (eg `CMAKE_PREFIX_PATH="C:\Program Files;C:\Program Files (x86)" its failing..

p01arst0rm · 2020-06-23T00:07:27Z

ok so i think this is just a huge rabbit hole that i've fallen down because it seems this might just be a repeat of #7305. it keeps giving very strange errors which have nothing to do with why its failing to find the dependency. again, modifying the installed cmake files seems to allow meson to find them. i dont know what i'm doing wrong here.........

p01arst0rm · 2020-06-23T00:30:15Z

i did a bunch more digging, and it looks like its nothing to do with the dependency at all?! for some reason, when specifying the language types in the initial project() decleration, it breaks searching for dependencies on windows. weird.

Jillinger · 2021-03-15T17:09:18Z

Hey @p01arst0rm I don't know if you got your issue resolved, but I think it's similar to a problem I was having, and was able to solve.
Try export CMAKE_PREFIX_PATH=/c/Program Files (x86)/Attica
Regards

p01arst0rm · 2021-03-16T00:52:54Z

Hey @p01arst0rm I don't know if you got your issue resolved, but I think it's similar to a problem I was having, and was able to solve.
Try export CMAKE_PREFIX_PATH=/c/Program Files (x86)/Attica
Regards

its nothing to do with CMAKE_PREFIX_PATH, its to do with the variable being ignored or broken internally somewhere; the prefix path was set.

Jillinger · 2021-03-18T10:41:36Z

Yes. It's being ignored, because it needs to "make itself visible". Hence export VAR=here-I-am
At least that's what solved it for me. The prefix was set. The system saw it. Meson didn't.
Regards

p01arst0rm · 2021-03-18T12:03:24Z

Yes. It's being ignored, because it needs to "make itself visible". Hence export VAR=here-I-am
At least that's what solved it for me. The prefix was set. The system saw it. Meson didn't.
Regards

again, no. This is an issue on windows, not *nix. on NT systems you use set, not export. furthermore, the setting of the cmake prefix path was done inside of a bootstrap file which also properly set the environment variables, and those variables were in fact detected by meson, as shown above in the thread. this is not a setup issue, it was a valid bug at time of opening, and it was never properly resolved

Jillinger · 2021-03-18T13:50:08Z

Okay Thanks.

Keisial · 2022-12-08T02:32:33Z

Hey @p01arst0rm I don't know if you got your issue resolved, but I think it's similar to a problem I was having, and was able to solve.
Try export CMAKE_PREFIX_PATH=/c/Program Files (x86)/Attica
Regards

its nothing to do with CMAKE_PREFIX_PATH, its to do with the variable being ignored or broken internally somewhere; the prefix path was set.

@Jillinger has a point here. The colon is used as path separator on *nix systems inside environment vars. I can see how -incorrectly applying the same logic in Windows- it could be getting confused by treatingC:\Program Files as multiple paths whereas it could be working with the unixy /c/Program files format (as long as it is finally converted back further down).

p01arst0rm · 2022-12-08T11:31:32Z

true, but that assumes youre running in a mingw like environment (/c/*) this is running native inside an nt shell

mensinda added dependency:cmake Issues related to `dependency` with the `cmake` method dependencies labels Jun 14, 2020

AcTiViSioN911 mentioned this issue Nov 21, 2020

Cant build it on windows... Genymobile/scrcpy#1928

Open

btgoodwin mentioned this issue May 25, 2021

Cannot build on Windows amzn/amazon-s3-gst-plugin#13

Closed

[CMake] Meson not properly respecting CMAKE_PREFIX_PATH when searching for dependencies on windows #7323

[CMake] Meson not properly respecting CMAKE_PREFIX_PATH when searching for dependencies on windows #7323

Comments

p01arst0rm commented Jun 14, 2020 • edited Loading

mensinda commented Jun 14, 2020

p01arst0rm commented Jun 14, 2020

p01arst0rm commented Jun 14, 2020

p01arst0rm commented Jun 14, 2020 • edited Loading

mensinda commented Jun 14, 2020

p01arst0rm commented Jun 14, 2020

mensinda commented Jun 14, 2020

p01arst0rm commented Jun 14, 2020

p01arst0rm commented Jun 14, 2020

mensinda commented Jun 14, 2020

p01arst0rm commented Jun 14, 2020

p01arst0rm commented Jun 17, 2020

mensinda commented Jun 17, 2020

p01arst0rm commented Jun 17, 2020

mensinda commented Jun 17, 2020

p01arst0rm commented Jun 18, 2020 • edited Loading

p01arst0rm commented Jun 18, 2020 • edited Loading

p01arst0rm commented Jun 18, 2020

p01arst0rm commented Jun 18, 2020

p01arst0rm commented Jun 18, 2020

p01arst0rm commented Jun 18, 2020

p01arst0rm commented Jun 18, 2020 • edited Loading

p01arst0rm commented Jun 18, 2020

p01arst0rm commented Jun 18, 2020

p01arst0rm commented Jun 18, 2020

p01arst0rm commented Jun 18, 2020

p01arst0rm commented Jun 22, 2020

p01arst0rm commented Jun 23, 2020

p01arst0rm commented Jun 23, 2020

Jillinger commented Mar 15, 2021

p01arst0rm commented Mar 16, 2021

Jillinger commented Mar 18, 2021 • edited Loading

p01arst0rm commented Mar 18, 2021

Jillinger commented Mar 18, 2021

Keisial commented Dec 8, 2022

p01arst0rm commented Dec 8, 2022

p01arst0rm commented Jun 14, 2020 •

edited

Loading

p01arst0rm commented Jun 14, 2020 •

edited

Loading

p01arst0rm commented Jun 18, 2020 •

edited

Loading

p01arst0rm commented Jun 18, 2020 •

edited

Loading

p01arst0rm commented Jun 18, 2020 •

edited

Loading

Jillinger commented Mar 18, 2021 •

edited

Loading