From ea728e2cc7ffe6489a017aa9d8afcf944e67f7a7 Mon Sep 17 00:00:00 2001 From: Allen Byrne Date: Sat, 28 Sep 2024 13:52:03 -0500 Subject: [PATCH] merge missing changes from 1.14.5 --- doc/threadsafety-warning.md | 16 +++++ release_docs/RELEASE.txt | 116 ++++++++++++++++-------------------- src/H5Ppublic.h | 6 +- 3 files changed, 70 insertions(+), 68 deletions(-) create mode 100644 doc/threadsafety-warning.md diff --git a/doc/threadsafety-warning.md b/doc/threadsafety-warning.md new file mode 100644 index 00000000000..49abcb00af9 --- /dev/null +++ b/doc/threadsafety-warning.md @@ -0,0 +1,16 @@ +## A Warning + +Any application that creates threads that use the HDF5 library must join those threads before either process exit or library close through H5close(). If all HDF5-using threads aren't joined, the threads may exhibit undefined behavior. + +## Discussion for Developers on Potential Improvements + +It would in principle be possible to make it safe to have threads continue using HDF5 resources after a call to H5close() by keeping a count of threads within the library. (There is probably no solution to an early process exit producing undefined behavior within threads.) This method would only be able to count (and presumably, only _need_ to count) threads that directly interact with the library. Because each thread would need to be counted exactly once, this would most likely be done by use of a thread-local key with e.g. a boolean value used to track whether the a global atomic thread counter has already counted this thread. Then, if H5close() is invoked while this thread counter is above one (because one thread must be doing the closing), the library would not close, and instead keep its resources valid to hopefully avoid bad behavior with the threads. + +The issues with this approach are as follows: + +1. The process of checking for the existence/value of the thread-local key is slow, or at least slow enough that it's probably not worth adding this to almost every single API call to prevent this particular edge case. +2. Even with this approach, bad behavior would still be possible if the application does something like expose HDF5 resources to threads indirectly via a global variable. +3. How to allow H5close() to fail is nonobvious. H5close() could be allowed to return an error indicating a failure to close, but the number of applications which could usefully respond to such an error by joining threads is small. If an application were able/willing to join its created threads, presumably it would have done so before calling H5close(). Alternatively, H5close() could succeed but silently leave the library open. This creates the potential for confusing, unexpected behavior when the user thinks they are closing and re-opening the library, e.g. if environment variables are modified between close and re-open, or if resources such as default property lists are modified. +4. Applications should join threads before closing libraries that those threads are using, so all of this work would constitute an above-and-beyond effort to maintain safe and defined behavior in the face of an unsafe application. + +Despite these issues, if a more performant method was found to perform threadcounting like this, it might still constitute a worthwhile change. \ No newline at end of file diff --git a/release_docs/RELEASE.txt b/release_docs/RELEASE.txt index 91331af9217..f61cea30843 100644 --- a/release_docs/RELEASE.txt +++ b/release_docs/RELEASE.txt @@ -171,64 +171,68 @@ Bug Fixes since HDF5-1.14.4 release Platforms Tested =================== - - HDF5 supports the latest macOS versions, including the current and two - preceding releases. As new major macOS versions become available, HDF5 - will discontinue support for the oldest version and add the latest - version to its list of compatible systems, along with the previous two - releases. - - Linux 5.16.14-200.fc35 GNU gcc (GCC) 11.2.1 20220127 (Red Hat 11.2.1-9) - #1 SMP x86_64 GNU/Linux GNU Fortran (GCC) 11.2.1 20220127 (Red Hat 11.2.1-9) - Fedora35 clang version 13.0.0 (Fedora 13.0.0-3.fc35) + - HDF5 is tested with the two latest macOS versions that are available + on github runners. As new major macOS versions become available, HDF5 + will discontinue support for the older version and add the new latest + version to its list of compatible systems, along with the previous + version. + + Linux 6.8.0-1010-aws GNU gcc, gfortran, g++ + #10-Ubuntu SMP 2024 x86_64 (Ubuntu 13.2.0-23ubuntu4) 13.2.0 + GNU/Linux Ubuntu 24.04 Ubuntu clang version 18.1.3 (1ubuntu1) + Intel(R) oneAPI DPC++/C++ Compiler 2024.2.0 + ifx (IFX) 2024.2.0 20240602 (cmake and autotools) - Linux 5.19.0-1023-aws GNU gcc, gfortran, g++ - #24-Ubuntu SMP x86_64 GNU/Linux (Ubuntu 11.3.0-1ubuntu1~22.04) 11.3.0 + Linux 6.5.0-1018-aws GNU gcc, gfortran, g++ + #18-Ubuntu SMP x86_64 GNU/Linux (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0 Ubuntu 22.04 Ubuntu clang version 14.0.0-1ubuntu1 - Intel(R) oneAPI DPC++/C++ Compiler 2023.1.0 - ifort (IFORT) 2021.9.0 20230302 + Intel(R) oneAPI DPC++/C++ Compiler 2024.0.2 + ifx (IFX) 2024.0.2 20231213 (cmake and autotools) - Linux 5.14.21-cray_shasta_c cray-mpich/8.1.23 + Linux 5.14.21-cray_shasta_c cray-mpich/8.1.28 #1 SMP x86_64 GNU/Linux cce/15.0.0 - (frontier) gcc/12.2.0 + (frontier) gcc/13.2 (cmake) - Linux 5.11.0-34-generic GNU gcc (GCC) 9.4.0-1ubuntu1 - #36-Ubuntu SMP x86_64 GNU/Linux GNU Fortran (GCC) 9.4.0-1ubuntu1 - Ubuntu 20.04 Ubuntu clang version 10.0.0-4ubuntu1 - Intel(R) oneAPI DPC++/C++ Compiler 2023.1.0 - ifort (IFORT) 2021.9.0 20230302 + Linux 5.14.0-427.24.1.el9_4 GNU gcc, gfortran, g++ (Red Hat 11.4.1-3) + #1 SMP x86_64 GNU/Linux clang version 17.0.6 + Rocky 9 Intel(R) oneAPI DPC++/C++ Compiler 2024.2.0 + ifx (IFX) 2024.2.0 (cmake and autotools) - Linux 4.14.0-115.35.1.1chaos aue/openmpi/4.1.4-arm-22.1.0.12 - #1 SMP aarch64 GNU/Linux Arm C/C++/Fortran Compiler version 22.1 - (stria) (based on LLVM 13.0.1) + Linux-4.18.0-553.16.1.1toss.t4 openmpi/4.1.2 + #1 SMP x86_64 GNU/Linux clang 14.0.6 + (corona, dane) GCC 12.1.1 + Intel(R) oneAPI DPC++/C++ Compiler 2023.2.1 + ifx (IFX) 2023.2.1 + + Linux-4.18.0-553.5.1.1toss.t4 openmpi/4.1/4.1.6 + #1 SMP x86_64 GNU/Linux clang 16.0.6 + (eclipse) GCC 12.3.0 + Intel(R) oneAPI DPC++/C++ Compiler 2024.0.2 + ifx (IFX) 2024.0.2 (cmake) Linux 4.14.0-115.35.1.3chaos spectrum-mpi/rolling-release - #1 SMP ppc64le GNU/Linux clang 12.0.1 - (vortex) GCC 8.3.1 - XL 2021.09.22 + #1 SMP ppc64le GNU/Linux clang 17.0.6 + (vortex) GCC 12.2.1 + nvhpc 24.1 + XL 2023.06.28 (cmake) - Linux-4.14.0-115.21.2 spectrum-mpi/rolling-release - #1 SMP ppc64le GNU/Linux clang 12.0.1, 14.0.5 + Linux-4.14.0-115.35.1 spectrum-mpi/rolling-release + #1 SMP ppc64le GNU/Linux clang 14.0.5, 15.0.6 (lassen) GCC 8.3.1 - XL 16.1.1.2, 2021.09.22, 2022.08.05 + XL 2021.09.22, 2022.08.05 (cmake) - Linux-4.12.14-197.99-default cray-mpich/7.7.14 - #1 SMP x86_64 GNU/Linux cce 12.0.3 - (theta) GCC 11.2.0 - llvm 9.0 - Intel 19.1.2 - Linux 3.10.0-1160.36.2.el7.ppc64 gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-39) #1 SMP ppc64be GNU/Linux g++ (GCC) 4.8.5 20150623 (Red Hat 4.8.5-39) Power8 (echidna) GNU Fortran (GCC) 4.8.5 20150623 (Red Hat 4.8.5-39) - Linux 3.10.0-1160.24.1.el7 GNU C (gcc), Fortran (gfortran), C++ (g++) + Linux 3.10.0-1160.80.1.el7 GNU C (gcc), Fortran (gfortran), C++ (g++) #1 SMP x86_64 GNU/Linux compilers: Centos7 Version 4.8.5 20150623 (Red Hat 4.8.5-4) (jelly/kituo/moohan) Version 4.9.3, Version 7.2.0, Version 8.3.0, @@ -251,37 +255,19 @@ Platforms Tested (autotools and cmake) - Linux-3.10.0-1160.0.0.1chaos openmpi-4.1.2 - #1 SMP x86_64 GNU/Linux clang 6.0.0, 11.0.1 - (quartz) GCC 7.3.0, 8.1.0 - Intel 19.0.4, 2022.2, oneapi.2022.2 - - Linux-3.10.0-1160.90.1.1chaos openmpi/4.1 - #1 SMP x86_64 GNU/Linux GCC 7.2.0 - (skybridge) Intel/19.1 + Linux-3.10.0-1160.119.1.1chaos openmpi/4.1.4 + #1 SMP x86_64 GNU/Linux clang 16.0.6 + (skybridge) Intel(R) oneAPI DPC++/C++ Compiler 2023.2.0 + ifx (IFX) 2023.2.0 (cmake) Linux-3.10.0-1160.90.1.1chaos openmpi/4.1 - #1 SMP x86_64 GNU/Linux GCC 7.2.0 - (attaway) Intel/19.1 + #1 SMP x86_64 GNU/Linux clang 16.0.6 + (attaway) GCC 12.1.0 + Intel(R) oneAPI DPC++/C++ Compiler 2024.0.2 + ifx (IFX) 2024.0.2 (cmake) - Linux-3.10.0-1160.90.1.1chaos openmpi-intel/4.1 - #1 SMP x86_64 GNU/Linux Intel/19.1.2, 21.3.0 and 22.2.0 - (chama) (cmake) - - macOS Apple M1 11.6 Apple clang version 12.0.5 (clang-1205.0.22.11) - Darwin 20.6.0 arm64 gfortran GNU Fortran (Homebrew GCC 11.2.0) 11.1.0 - (macmini-m1) Intel icc/icpc/ifort version 2021.3.0 202106092021.3.0 20210609 - - macOS Big Sur 11.3.1 Apple clang version 12.0.5 (clang-1205.0.22.9) - Darwin 20.4.0 x86_64 gfortran GNU Fortran (Homebrew GCC 10.2.0_3) 10.2.0 - (bigsur-1) Intel icc/icpc/ifort version 2021.2.0 20210228 - - Mac OS X El Capitan 10.11.6 Apple clang version 7.3.0 from Xcode 7.3 - 64-bit gfortran GNU Fortran (GCC) 5.2.0 - (osx1011test) Intel icc/icpc/ifort version 16.0.2 - Linux 2.6.32-573.22.1.el6 GNU C (gcc), Fortran (gfortran), C++ (g++) #1 SMP x86_64 GNU/Linux compilers: Centos6 Version 4.4.7 20120313 @@ -294,9 +280,9 @@ Platforms Tested Windows 10 x64 Visual Studio 2019 w/ clang 12.0.0 with MSVC-like command-line (C/C++ only - cmake) Visual Studio 2019 w/ Intel (C/C++ only - cmake) - Visual Studio 2022 w/ clang 15.0.1 + Visual Studio 2022 w/ clang 17.0.3 with MSVC-like command-line (C/C++ only - cmake) - Visual Studio 2022 w/ Intel C/C++/Fortran oneAPI 2023 (cmake) + Visual Studio 2022 w/ Intel C/C++ oneAPI 2023 (cmake) Visual Studio 2019 w/ MSMPI 10.1 (C only - cmake) @@ -350,7 +336,7 @@ Known Problems results in very complex and fragile build files. - At present, metadata cache images may not be generated by parallel - applications. Parallel applications can read files with metadata cache + applications. Parallel applications can read files with metadata cache images, but since this is a collective operation, a deadlock is possible if one or more processes do not participate. diff --git a/src/H5Ppublic.h b/src/H5Ppublic.h index 6ee93441586..0ebe6c54a7a 100644 --- a/src/H5Ppublic.h +++ b/src/H5Ppublic.h @@ -2499,8 +2499,8 @@ H5_DLL herr_t H5Pset_deflate(hid_t plist_id, unsigned level); * pipeline * \param[in] flags Bit vector specifying certain general properties of * the filter - * \param[in] cd_nelmts Number of elements in \p c_values - * \param[in] c_values Auxiliary data for the filter + * \param[in] cd_nelmts Number of elements in \p cd_values + * \param[in] cd_values Auxiliary data for the filter * * \return \herr_t * @@ -2756,7 +2756,7 @@ H5_DLL herr_t H5Pset_deflate(hid_t plist_id, unsigned level); * */ H5_DLL herr_t H5Pset_filter(hid_t plist_id, H5Z_filter_t filter, unsigned int flags, size_t cd_nelmts, - const unsigned int c_values[]); + const unsigned int cd_values[]); /** * \ingroup OCPL *