[hdf5] Build error for Windows Arm64 platforms

[niyas-sait]

Host Environment

• OS: Windows
• Compiler: VS 2022

To Reproduce
Steps to reproduce the behavior:
.\vcpkg.exe install hdf5 --triplet=arm64-windows

Failure logs

From CMake error log

CMake Error at config/cmake/ConfigureChecks.cmake:334 (file):
  file failed to open for reading (No such file or directory):

    C:/Users/niysai01/Workspace/vcpkg/buildtrees/hdf5/arm64-windows-dbg/pac_Cconftest.out
Call Stack (most recent call first):
  CMakeLists.txt:464 (include)

** Additional Context **

Static builds seem to be working

.\vcpkg.exe install hdf5 --triplet=arm64-windows-static

[LilyWangLL]

Thanks for posting this issue. Please provide the complete error log file. Because when I installing hdf5:arm64-windows locally, except this error, I also have the following error:

CMake Error: TRY_RUN() invoked in cross-compiling mode, please set the following cache variables appropriately:
   HAVE_IOEO_EXITCODE (advanced)
For details see E:/Lily/hdf5/vcpkg/buildtrees/hdf5/arm64-windows-dbg/TryRunResults.cmake
CMake Error: TRY_RUN() invoked in cross-compiling mode, please set the following cache variables appropriately:
   H5_PRINTF_LL_TEST_RUN (advanced)
   H5_PRINTF_LL_TEST_RUN__TRYRUN_OUTPUT (advanced)
For details see E:/Lily/hdf5/vcpkg/buildtrees/hdf5/arm64-windows-dbg/TryRunResults.cmake
CMake Error: TRY_RUN() invoked in cross-compiling mode, please set the following cache variables appropriately:
   RUN_RESULT_VAR (advanced)
   RUN_RESULT_VAR__TRYRUN_OUTPUT (advanced)
For details see E:/Lily/hdf5/vcpkg/buildtrees/hdf5/arm64-windows-dbg/TryRunResults.cmake
CMake Error: TRY_RUN() invoked in cross-compiling mode, please set the following cache variables appropriately:
   H5_LDOUBLE_TO_LONG_SPECIAL_RUN (advanced)
   H5_LDOUBLE_TO_LONG_SPECIAL_RUN__TRYRUN_OUTPUT (advanced)
For details see E:/Lily/hdf5/vcpkg/buildtrees/hdf5/arm64-windows-dbg/TryRunResults.cmake
CMake Error: TRY_RUN() invoked in cross-compiling mode, please set the following cache variables appropriately:
   H5_LONG_TO_LDOUBLE_SPECIAL_RUN (advanced)
   H5_LONG_TO_LDOUBLE_SPECIAL_RUN__TRYRUN_OUTPUT (advanced)
For details see E:/Lily/hdf5/vcpkg/buildtrees/hdf5/arm64-windows-dbg/TryRunResults.cmake
CMake Error: TRY_RUN() invoked in cross-compiling mode, please set the following cache variables appropriately:
   H5_LDOUBLE_TO_LLONG_ACCURATE_RUN (advanced)
   H5_LDOUBLE_TO_LLONG_ACCURATE_RUN__TRYRUN_OUTPUT (advanced)
For details see E:/Lily/hdf5/vcpkg/buildtrees/hdf5/arm64-windows-dbg/TryRunResults.cmake
CMake Error: TRY_RUN() invoked in cross-compiling mode, please set the following cache variables appropriately:
   H5_LLONG_TO_LDOUBLE_CORRECT_RUN (advanced)
   H5_LLONG_TO_LDOUBLE_CORRECT_RUN__TRYRUN_OUTPUT (advanced)
For details see E:/Lily/hdf5/vcpkg/buildtrees/hdf5/arm64-windows-dbg/TryRunResults.cmake
CMake Error: TRY_RUN() invoked in cross-compiling mode, please set the following cache variables appropriately:
   H5_NO_ALIGNMENT_RESTRICTIONS_RUN (advanced)
   H5_NO_ALIGNMENT_RESTRICTIONS_RUN__TRYRUN_OUTPUT (advanced)
For details see E:/Lily/hdf5/vcpkg/buildtrees/hdf5/arm64-windows-dbg/TryRunResults.cmake

And your error in this issue will be fixed by new version of hdf5, I saw PR #24758 is doing update hdf5.

[niyas-sait]

Attached console and CMake log.

hd5.log
dbg-error.log
dbg-out.log

For reference, I am building on windows/arm64 platform

[niyas-sait]

And I tried PR #24758 but unfortunately getting some error

> cat C:\Users\niysai01\Workspace\vcpkg\buildtrees\hdf5\config-arm64-windows-dbg-err.log
CMake Error at config/cmake/ConfigureChecks.cmake:320 (list):
  list index: 1 out of range (-1, 0)
Call Stack (most recent call first):
  CMakeLists.txt:464 (include)

[LilyWangLL]

Hi @nsait-linaro, PR #24758 has been merged, and it has new change before merged, could you update your vcpkg and install version 1.2.2 of hdf5:arm64-windows? If it still install failed, please also provide the error log. Thanks~

[niyas-sait]

Thanks, @LilyWangLL.

I am getting the exact same error (logs in #24898 (comment) ) even with the latest vcpkg.

[LilyWangLL]

Sorry, if your error still is

CMake Error at config/cmake/ConfigureChecks.cmake:334 (file):
  file failed to open for reading (No such file or directory):

    C:/Users/niysai01/Workspace/vcpkg/buildtrees/hdf5/arm64-windows-dbg/pac_Cconftest.out

This has been merged into new version 1.2.2, the related upstream issue: HDFGroup/hdf5#1558, fixed by HDFGroup/hdf5#1089.

[LilyWangLL]

Ping @nsait-linaro for response.

[niyas-sait]

Unfortunately, It is still failing but with a different error this time.

.\vcpkg.exe install hdf5 --triplet=arm64-windows
Computing installation plan...
The following packages will be built and installed:
    hdf5[core,szip,zlib]:arm64-windows -> 1.12.2
Detecting compiler hash for triplet arm64-windows...
Restored 0 packages from C:\Users\niysai01\AppData\Local\vcpkg\archives in 774.5 us. Use --debug to see more details.
Starting package 1/1: hdf5:arm64-windows
Building package hdf5[core,szip,zlib]:arm64-windows...
-- Using cached HDFGroup-hdf5-hdf5-1_12_2.tar.gz.
-- Cleaning sources at C:/Users/niysai01/Workspace/vcpkg/buildtrees/hdf5/src/df5-1_12_2-c0e67ca243.clean. Use --editable to skip cleaning for the packages you specify.
-- Extracting source C:/Users/niysai01/Workspace/vcpkg/downloads/HDFGroup-hdf5-hdf5-1_12_2.tar.gz
-- Applying patch hdf5_config.patch
-- Applying patch szip.patch
-- Applying patch pkgconfig-requires.patch
-- Applying patch pkgconfig-link-order.patch
-- Using source at C:/Users/niysai01/Workspace/vcpkg/buildtrees/hdf5/src/df5-1_12_2-c0e67ca243.clean
-- Found external ninja('1.10.2').
-- Configuring arm64-windows-dbg
-- Configuring arm64-windows-rel
-- Building arm64-windows-dbg
-- Building arm64-windows-rel
CMake Error at scripts/cmake/vcpkg_execute_build_process.cmake:158 (message):
    Command failed: C:/Users/niysai01/Workspace/vcpkg/downloads/tools/cmake-3.22.2-windows/cmake-3.22.2-windows-i386/bin/cmake.exe --build . --config Release --target install -- -v -j9
    Working Directory: C:/Users/niysai01/Workspace/vcpkg/buildtrees/hdf5/arm64-windows-rel
    See logs for more information:
      C:\Users\niysai01\Workspace\vcpkg\buildtrees\hdf5\install-arm64-windows-rel-out.log

Call Stack (most recent call first):
  installed/arm64-windows/share/vcpkg-cmake/vcpkg_cmake_build.cmake:111 (vcpkg_execute_build_process)
  installed/arm64-windows/share/vcpkg-cmake/vcpkg_cmake_install.cmake:41 (vcpkg_cmake_build)
  ports/hdf5/portfile.cmake:77 (vcpkg_cmake_install)
  scripts/ports.cmake:146 (include)


Error: Building package hdf5:arm64-windows failed with: BUILD_FAILED

install-arm64-windows-rel-out.log

[1/347] C:\PROGRA~2\MICROS~2\2019\COMMUN~1\VC\Tools\MSVC\1429~1.301\bin\Hostx86\arm64\cl.exe   -D_BIND_TO_CURRENT_VCLIBS_VERSION=1 -D_CONSOLE -D_CRT_NONSTDC_NO_DEPRECATE -D_CRT_SECURE_NO_DEPRECATE -D_CRT_SECURE_NO_WARNINGS -IC:\Users\niysai01\Workspace\vcpkg\installed\arm64-windows\include -IC:\Users\niysai01\Workspace\vcpkg\buildtrees\hdf5\src\df5-1_12_2-c0e67ca243.clean\src -IC:\Users\niysai01\Workspace\vcpkg\buildtrees\hdf5\arm64-windows-rel\src /nologo /DWIN32 /D_WINDOWS /utf-8 /MP  -wd5105 /MD /O2 /Oi /Gy /DNDEBUG /Z7 /showIncludes /Fosrc\CMakeFiles\H5make_libsettings.dir\H5make_libsettings.c.obj /Fdsrc\CMakeFiles\H5make_libsettings.dir\ /FS -c C:\Users\niysai01\Workspace\vcpkg\buildtrees\hdf5\src\df5-1_12_2-c0e67ca243.clean\src\H5make_libsettings.c
[2/347] cmd.exe /C "cd . && C:\Users\niysai01\Workspace\vcpkg\downloads\tools\cmake-3.22.2-windows\cmake-3.22.2-windows-i386\bin\cmake.exe -E vs_link_exe --intdir=src\CMakeFiles\H5make_libsettings.dir --rc=C:\PROGRA~2\WI3CF2~1\10\bin\100226~1.0\x86\rc.exe --mt=C:\PROGRA~2\WI3CF2~1\10\bin\100226~1.0\x86\mt.exe --manifests  -- C:\PROGRA~2\MICROS~2\2019\COMMUN~1\VC\Tools\MSVC\1429~1.301\bin\Hostx86\arm64\link.exe  src\CMakeFiles\H5make_libsettings.dir\H5make_libsettings.c.obj  /out:bin\H5make_libsettings.exe /implib:bin\H5make_libsettings.lib /pdb:bin\H5make_libsettings.pdb /version:0.0 /machine:ARM64 -stack:10000000 /nologo /DEBUG /INCREMENTAL:NO /OPT:REF /OPT:ICF   /subsystem:console  ws2_32.lib  kernel32.lib user32.lib
gdi32.lib winspool.lib shell32.lib ole32.lib oleaut32.lib uuid.lib comdlg32.lib advapi32.lib && cd ."
[3/347] C:\PROGRA~2\MICROS~2\2019\COMMUN~1\VC\Tools\MSVC\1429~1.301\bin\Hostx86\arm64\cl.exe   -D_BIND_TO_CURRENT_VCLIBS_VERSION=1 -D_CONSOLE -D_CRT_NONSTDC_NO_DEPRECATE -D_CRT_SECURE_NO_DEPRECATE -D_CRT_SECURE_NO_WARNINGS -IC:\Users\niysai01\Workspace\vcpkg\installed\arm64-windows\include -IC:\Users\niysai01\Workspace\vcpkg\buildtrees\hdf5\src\df5-1_12_2-c0e67ca243.clean\src -IC:\Users\niysai01\Workspace\vcpkg\buildtrees\hdf5\arm64-windows-rel\src /nologo /DWIN32 /D_WINDOWS /utf-8 /MP  -wd5105 /MD /O2 /Oi /Gy /DNDEBUG /Z7 /showIncludes /Fosrc\CMakeFiles\H5detect.dir\H5detect.c.obj /Fdsrc\CMakeFiles\H5detect.dir\ /FS -c C:\Users\niysai01\Workspace\vcpkg\buildtrees\hdf5\src\df5-1_12_2-c0e67ca243.clean\src\H5detect.c
[4/347] cmd.exe /C "cd . && C:\Users\niysai01\Workspace\vcpkg\downloads\tools\cmake-3.22.2-windows\cmake-3.22.2-windows-i386\bin\cmake.exe -E vs_link_exe --intdir=src\CMakeFiles\H5detect.dir --rc=C:\PROGRA~2\WI3CF2~1\10\bin\100226~1.0\x86\rc.exe --mt=C:\PROGRA~2\WI3CF2~1\10\bin\100226~1.0\x86\mt.exe --manifests  -- C:\PROGRA~2\MICROS~2\2019\COMMUN~1\VC\Tools\MSVC\1429~1.301\bin\Hostx86\arm64\link.exe  src\CMakeFiles\H5detect.dir\H5detect.c.obj  /out:bin\H5detect.exe /implib:bin\H5detect.lib /pdb:bin\H5detect.pdb /version:0.0 /machine:ARM64 -stack:10000000 /nologo /DEBUG /INCREMENTAL:NO /OPT:REF /OPT:ICF   /subsystem:console  ws2_32.lib  kernel32.lib user32.lib gdi32.lib winspool.lib shell32.lib ole32.lib oleaut32.lib uuid.lib comdlg32.lib advapi32.lib && cd ."
[5/347] cmd.exe /C "cd /D C:\Users\niysai01\Workspace\vcpkg\buildtrees\hdf5\arm64-windows-rel\src && C:\Users\niysai01\Workspace\vcpkg\buildtrees\hdf5\arm64-windows-rel\bin\H5detect.exe H5Tinit.c && C:\Users\niysai01\Workspace\vcpkg\downloads\tools\cmake-3.22.2-windows\cmake-3.22.2-windows-i386\bin\cmake.exe -E touch gen_SRCS.stamp1"
FAILED: src/gen_SRCS.stamp1 src/H5Tinit.c C:/Users/niysai01/Workspace/vcpkg/buildtrees/hdf5/arm64-windows-rel/src/gen_SRCS.stamp1 C:/Users/niysai01/Workspace/vcpkg/buildtrees/hdf5/arm64-windows-rel/src/H5Tinit.c
cmd.exe /C "cd /D C:\Users\niysai01\Workspace\vcpkg\buildtrees\hdf5\arm64-windows-rel\src && C:\Users\niysai01\Workspace\vcpkg\buildtrees\hdf5\arm64-windows-rel\bin\H5detect.exe H5Tinit.c && C:\Users\niysai01\Workspace\vcpkg\downloads\tools\cmake-3.22.2-windows\cmake-3.22.2-windows-i386\bin\cmake.exe -E touch gen_SRCS.stamp1"
[6/347] cmd.exe /C "cd /D C:\Users\niysai01\Workspace\vcpkg\buildtrees\hdf5\arm64-windows-rel\src && C:\Users\niysai01\Workspace\vcpkg\buildtrees\hdf5\arm64-windows-rel\bin\H5make_libsettings.exe H5lib_settings.c && C:\Users\niysai01\Workspace\vcpkg\downloads\tools\cmake-3.22.2-windows\cmake-3.22.2-windows-i386\bin\cmake.exe -E touch gen_SRCS.stamp2"
FAILED: src/gen_SRCS.stamp2 src/H5lib_settings.c C:/Users/niysai01/Workspace/vcpkg/buildtrees/hdf5/arm64-windows-rel/src/gen_SRCS.stamp2 C:/Users/niysai01/Workspace/vcpkg/buildtrees/hdf5/arm64-windows-rel/src/H5lib_settings.c
cmd.exe /C "cd /D C:\Users\niysai01\Workspace\vcpkg\buildtrees\hdf5\arm64-windows-rel\src && C:\Users\niysai01\Workspace\vcpkg\buildtrees\hdf5\arm64-windows-rel\bin\H5make_libsettings.exe H5lib_settings.c && C:\Users\niysai01\Workspace\vcpkg\downloads\tools\cmake-3.22.2-windows\cmake-3.22.2-windows-i386\bin\cmake.exe -E touch gen_SRCS.stamp2"
ninja: build stopped: subcommand failed.

[LilyWangLL]

This issue is similar with #16203, @JackBoosY could you help take a look this issue?

[dg0yt]

C:\Users\niysai01\Workspace\vcpkg\buildtrees\hdf5\arm64-windows-rel\bin\H5detect.exe

IIRC there is a general issue with cross builds of hdf5. It wants to run an executable (on the host, x64) that it builds for the target (arm64). This also prevents android and (cross) mingw builds.

[JackBoosY]

To add arm support, we should do the following steps:

1. add the following code to dependencies in vcpkg.json:

{
    "name": "hdf5",
    "default-features": false,
    "host": true,
    "features": [
        "tool"
    ]
}

2. find and pass the host tool path to vcpkg_cmake_configure in portfile.cmake if TARGET_TRIPLET is not equal HOST_TRIPLET.

[dg0yt]

To add arm support, we should do the following steps:

1. add the following code to dependencies in vcpkg.json:

{
    "name": "hdf5",
    "default-features": false,
    "host": true,
    "features": [
        "tool"
    ]
}

2. find and pass the host tool path to vcpkg_cmake_configure in portfile.cmake if TARGET_TRIPLET is not equal HOST_TRIPLET.

I don't think it is so easy. (And that's why I didn't add it yet.) IIRC the tool is to provide information about the target platform, and the host tool cannot provide this. You really need to run the tool at least once on the target platform, capture the required configuration information, and supply this information directly to the CMake cross build.

e.g. https://forum.hdfgroup.org/t/cross-compiling-for-windows/6735

[JackBoosY]

I don't think it is so easy. (And that's why I didn't add it yet.) IIRC the tool is to provide information about the target platform, and the host tool cannot provide this. You really need to run the tool at least once on the target platform, capture the required configuration information, and supply this information directly to the CMake cross build.

e.g. https://forum.hdfgroup.org/t/cross-compiling-for-windows/6735

I will have a try when I have spare time.

[MarkVabulas]

This also is a major limitation to compiling any other libraries which depend upon this one for arm64 builds.

[russelltg]

I have managed to hack together an hdf5 build for iOS here https://github.com/russelltg/vcpkg/tree/ios_changes_old/ports/hdf5
The files I generated are probably not 100% correct, but at least it builds. Idk if this is a direction that makes sense for this.

[MarkVabulas]

I have managed to hack together an hdf5 build for iOS here https://github.com/russelltg/vcpkg/tree/ios_changes_old/ports/hdf5 The files I generated are probably not 100% correct, but at least it builds. Idk if this is a direction that makes sense for this.

Will this also work for other arm64 builds, or only iOS ones? All of the cross-compiling is broken for arm64 builds, as far as I understand it.

[spinicist]

Hello,

I updated my vcpkg dependencies for the first time in a long time today, and I've hit this issue trying to cross-compile HDF5 for macos-x86 on an M1 (macos-arm64) box. This didn't happen before, so I will try rolling vcpkg back.

The problem also occurs in szip, which is one of HDF5's default dependencies. However, when I set default features to false, the problem occurs within hdf5 itself.

[spinicist]

PS - as @dg0yt noted above, this is a problem with cross-compilation, not with ARM support for HDF5.

I can compile the same project on my M1 box targeting macos-arm64 and everything is fine.

[spinicist]

Apologies for three messages in short succession, and for perhaps confusing matters.

I have rolled back my vcpkg repo to the tag 2022.07.25. HDF5 now builds fine. If I come up to the most recent tag, 2022.08.15, it stops building. I have no idea whether it is a change in the HDF5 port or vcpkg itself between these dates which causes the issue.

This doesn't make a great deal of sense to me because this issue was originally reported in May.

[RDCH106]

Do you know if there has been any progress here?

I have also experimented a build error using vcpkg 2023.06.20 and arm64-android => #34824

[github-actions]

This is an automated message. Per our repo policy, stale issues get closed if there has been no activity in the past 180 days. The issue will be automatically closed in 14 days. If you wish to keep this issue open, please add a new comment.