forked from HDFGroup/hdf5
-
Notifications
You must be signed in to change notification settings - Fork 0
/
HISTORY-1_14_0-1_16_0.txt
6387 lines (4625 loc) · 263 KB
/
HISTORY-1_14_0-1_16_0.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
HDF5 History
============
This file contains development history of the HDF5 1.14 branch
06. Release Information for hdf5-1.14.5
05. Release Information for hdf5-1.14.4
04. Release Information for hdf5-1.14.3
03. Release Information for hdf5-1.14.2
02. Release Information for hdf5-1.14.1
01. Release Information for hdf5-1.14.0
[Search on the string '%%%%' for section breaks of each release.]
%%%%1.14.5%%%%
HDF5 version 1.14.5 released on 2024-09-30
================================================================================
INTRODUCTION
============
This document describes the differences between this release and the previous
HDF5 release. It contains information on the platforms tested and known
problems in this release. For more details check the HISTORY*.txt files in the
HDF5 source.
Note that documentation in the links below will be updated at the time of each
final release.
Links to HDF5 documentation can be found on:
https://support.hdfgroup.org/releases/hdf5/latest-docs.html
The official HDF5 releases can be obtained from:
https://support.hdfgroup.org/downloads/index.html
Changes from Release to Release and New Features in the HDF5-1.14.x release series
can be found at:
https://support.hdfgroup.org/releases/hdf5/documentation/release_specific_info.md
If you have any questions or comments, please send them to the HDF Help Desk:
help@hdfgroup.org
CONTENTS
========
- New Features
- Support for new platforms and languages
- Bug Fixes since HDF5-1.14.4
- Platforms Tested
- Known Problems
- CMake vs. Autotools installations
New Features
============
Configuration:
-------------
- Added signed Windows msi binary and signed Apple dmg binary files.
The release process now provides signed Windows and Mac installation
binaries in addition to the Debian and rpm installation binaries. The Mac
binaries are built as universal binaries on an ARM-based Mac. Installer
files are no longer compressed into packaged archives.
- Moved examples to the HDF5Examples folder in the source tree.
Moved the C++ and Fortran examples from the examples folder to the HDF5Examples
folder and renamed to TUTR, tutorial. This is referenced from the LearnBasics
doxygen page.
- Added support for using zlib-ng package as the zlib library:
CMake: HDF5_USE_ZLIB_NG
Autotools: --enable-zlibng
Added the option HDF5_USE_ZLIB_NG to allow the replacement of the
default ZLib package by the zlib-ng package as a built-in compression library.
- Disable CMake UNITY_BUILD for hdf5
CMake added a target property, UNITY_BUILD, that when set to true, the target
source files will be combined into batches for faster compilation. By default,
the setting is OFF, but could be enabled by a project that includes HDF5 as a subproject.
HDF5 has disabled this feature by setting the property to OFF in the HDFMacros.cmake file.
- Removed "function/code stack" debugging configuration option:
CMake: HDF5_ENABLE_CODESTACK
Autotools: --enable-codestack
This was used to debug memory leaks internal to the library, but has been
broken for >1.5 years and is now easily replaced with third-party tools
(e.g. libbacktrace: https://github.com/ianlancetaylor/libbacktrace) on an
as-needed basis when debugging an issue.
- Added configure options for enabling/disabling non-standard programming
language features
- Added the CMake variable HDF5_ENABLE_ROS3_VFD to the HDF5 CMake config
file hdf5-config.cmake. This allows it to easily detect if the library
has been built with or without read-only S3 functionality.
Library:
--------
- Added new routines for interacting with error stacks: H5Epause_stack,
H5Eresume_stack, and H5Eis_paused. These routines can be used to
indicate that errors from a call to an HDF5 routine should not be
pushed on to an error stack. Primarily targeted toward third-party
developers of Virtual File Drivers (VFDs) and Virtual Object Layer (VOL)
connectors, these routines allow developers to perform "speculative"
operations (such as trying to open a file or object) without requiring
that the error stack be cleared after a speculative operation fails.
Parallel Library:
-----------------
-
Fortran Library:
----------------
- Add Fortran H5R APIs:
h5rcreate_attr_f, h5rcreate_object_f, h5rcreate_region_f,
h5ropen_attr_f, h5ropen_object_f, h5ropen_region_f,
h5rget_file_name_f, h5rget_attr_name_f, h5rget_obj_name_f,
h5rcopy_f, h5requal_f, h5rdestroy_f, h5rget_type_f
C++ Library:
------------
-
Java Library:
-------------
-
Tools:
------
- Added doxygen files for the tools
Implement the tools usage text as pages in doxygen.
- Added option to adjust the page buffer size in tools
The page buffer cache size for a file can now be adjusted using the
--page-buffer-size=N
option in the h5repack, h5diff, h5dump, h5ls, and h5stat tools. This
will call the H5Pset_page_buffer_size() API function with the specified
size in bytes.
- Allowed h5repack to reserve space for a user block without a file
This is useful for users who want to reserve space in the file for
future use without requiring a file to copy.
High-Level APIs:
----------------
-
C Packet Table API:
-------------------
-
Internal header file:
---------------------
-
Documentation:
--------------
- Documented that leaving HDF5 threads running at termination is unsafe
Added doc/threadsafety-warning.md as a warning that threads which use HDF5
resources must be closed before either process exit or library close.
If HDF5 threads are alive during either of these operations, their resources
will not be cleaned up properly and undefined behavior is possible.
This document also includes a discussion on potential ways to mitigate this issue.
Support for new platforms, languages and compilers
==================================================
-
Bug Fixes since HDF5-1.14.4 release
===================================
Library
-------
- Fixed a memory leak in H5F__accum_write()
The memory was allocated in H5F__accum_write() and was to be freed in
H5F__accum_reset() during the closing process but a failure occurred just
before the deallocation, leaving the memory un-freed. The problem is
now fixed.
Fixes GitHub #4585
- Fixed an incorrect returned value by H5LTfind_dataset()
H5LTfind_dataset() returned true for non-existing datasets because it only
compared up to the length of the searched string, such as "Day" vs "DayNight".
Applied the user's patch to correct this behavior.
Fixes GitHub #4780
- Fixed a segfault by H5Gmove2, extended to fix H5Lcopy and H5Lmove
A user's application segfaulted when it passed in an invalid location ID
to H5Gmove2. The src and dst location IDs must be either a file or a group
ID. The fix was also applied to H5Lcopy and H5Lmove. Now, all these
three functions will fail if either the src or dst location ID is not a file
or a group ID.
Fixes GitHub #4737
- Fixed a segfault by H5Lget_info()
A user's program generated a segfault when the ID passed into H5Lget_info()
was a datatype ID. This was caused by non-VOL functions being used internally
where VOL functions should have been. This correction was extended to many
other functions to prevent potential issue in the future.
Fixes GitHub #4730
- Fixed a segfault by H5Fget_intent(), extended to fix several other functions
A user's program generated a segfault when the ID passed into H5Fget_intent()
was not a file ID. In addition to H5Fget_intent(), a number of APIs also failed
to detect an incorrect ID being passed in, which can potentially cause various
failures, including segfault. The affected functions are listed below and now
properly detect incorrect ID parameters:
H5Fget_intent()
H5Fget_fileno()
H5Fget_freespace()
H5Fget_create_plist()
H5Fget_access_plist()
H5Fget_vfd_handle()
H5Dvlen_get_buf_size()
H5Fget_mdc_config()
H5Fset_mdc_config()
H5Freset_mdc_hit_rate_stats()
Fixes GitHub #4656 and GitHub #4662
- Fixed a bug with large external datasets
When performing a large I/O on an external dataset, the library would only
issue a single read or write system call. This could cause errors or cause
the data to be incorrect. These calls do not guarantee that they will
process the entire I/O request, and may need to be called multiple times
to complete the I/O, advancing the buffer and reducing the size by the
amount actually processed by read or write each time. Implemented this
algorithm for external datasets in both the read and write cases.
Fixes GitHub #4216
Fixes h5py GitHub #2394
- Fixed a bug in the Subfiling VFD that could cause a buffer over-read
and memory allocation failures
When performing vector I/O with the Subfiling VFD, making use of the
vector I/O size extension functionality could cause the VFD to read
past the end of the "I/O sizes" array that is passed in. When an entry
in the "I/O sizes" array has the value 0 and that entry is at an array
index greater than 0, this signifies that the value in the preceding
array entry should be used for the rest of the I/O vectors, effectively
extending the last valid I/O size across the remaining entries. This
allows an application to save a bit on memory by passing in a smaller
"I/O sizes" array. The Subfiling VFD didn't implement a check for this
functionality in the portion of the code that generates I/O vectors,
causing it to read past the end of the "I/O sizes" array when it was
shorter than expected. This could also result in memory allocation
failures, as the nearby memory allocations are based off the values
read from that array, which could be uninitialized.
- Fixed H5Rget_attr_name to return the length of the attribute's name
without the null terminator
H5Rget_file_name and H5Rget_obj_name both return the name's length
without the null terminator. H5Rget_attr_name now behaves consistently
with the other two APIs. Going forward, all the get character string
APIs in HDF5 will be modified/written in this manner, regarding the
length of a character string.
Fixes GitHub #4447
- Fixed heap-buffer-overflow in h5dump
h5dump aborted when provided with a malformed input file. The was because
the buffer size for checksum was smaller than H5_SIZEOF_CHKSUM, causing
an overflow while calculating the offset to the checksum in the buffer.
A check was added so H5F_get_checksums would fail appropriately in all
of its occurrences.
Fixes GitHub #4434
- Fixed library to allow usage of page buffering feature for serial file
access with parallel builds of HDF5
When HDF5 is built with parallel support enabled, previously the library would
disallow any usage of page buffering, even if a file was not opened with
parallel access. The library now allows usage of page buffering for serial
file access with parallel builds of HDF5. Usage of page buffering is still
disabled for any form of parallel file access, even if only 1 MPI process
is used.
- Fixed function H5Requal to actually compare the reference pointers
Fixed an issue with H5Requal always returning true because the
function was only comparing the ref2_ptr to itself.
- Fixed infinite loop closing library issue when h5dump with a user provided test file
The library's metadata cache calls the "get_final_load_size" client callback
to find out the actual size of the object header. As the size obtained
exceeds the file's EOA, it throws an error but the object header structure
allocated through the client callback is not freed, causing the issue
described.
(1) Free the structure allocated in the object header client callback after
saving the needed information in udata. (2) Deserialize the object header
prefix in the object header's "deserialize" callback regardless.
Fixes GitHub #3790
Java Library
------------
-
Configuration
-------------
- Fixed usage issue with FindZLIB.cmake module
When building HDF5 with CMake and relying on the FindZLIB.cmake module,
the Find module would correctly find the ZLIB library but not set an OUTPUT_NAME
on the target. Also, the target returned, ZLIB::ZLIB, was not in the ZLIB_LIBRARIES
variable. This caused issues when requesting the OUTPUT_NAME of the target in
the pkg-config settings.
Similar to HDF5_USE_LIBAEC_STATIC, "Find static AEC library", option, we added
a new option, HDF5_USE_ZLIB_STATIC, "Find static zlib library". These options
allow a user to specify whether to use a static or shared version of the compression
library in a find_package call.
- Corrected usage of FetchContent in the HDFLibMacros.cmake file.
CMake version 3.30 changed the behavior of the FetchContent module to deprecate
the use of FetchContent_Populate() in favor of FetchContent_MakeAvailable(). Therefore,
the copying of HDF specialized CMakeLists.txt files to the dependent project's source
was implemented in the FetchContent_Declare() call.
- Fixed/reverted an Autotools configure hack that causes problems on MacOS
A sed line in configure.ac was added in the past to paper over some
problems with older versions of the Autotools that would add incorrect
linker flags. This used the -i option in a way that caused silent
errors on MacOS that did not break the build.
The original fix for this problem (in 1.14.4) removed the sed line
entirely, but it turns out that the sed cleanup is still necessary
on some systems, where empty -l options will be added to the libtool
script.
This sed line has been restored and reworked to not use -i.
Fixes GitHub issues #3843 and #4448
- Fixed a list index out of range issue in the runTest.cmake file
Fixed an issue in config/cmake/runTest.cmake where the CMake logic
would try to access an invalid list index if the number of lines in
a test's output and reference files don't match.
- Fix Autotools -Werror cleanup
The Autotools temporarily scrub -Werror(=whatever) from CFLAGS, etc.
so configure checks don't trip over warnings generated by configure
check programs. The sed line originally only scrubbed -Werror but not
-Werror=something, which would cause errors when the '=something' was
left behind in CFLAGS.
The sed line has been updated to handle -Werror=something lines.
Fixes one issue raised in #3872
- Changed default of 'Error on HDF5 doxygen warnings' DOXYGEN_WARN_AS_ERROR option.
The default setting of DOXYGEN_WARN_AS_ERROR to 'FAIL_ON_WARNINGS' has been changed
to 'NO'. It was decided that the setting was too aggressive and should be a user choice.
The github actions and scripts have been updated to reflect this.
* HDF5_ENABLE_DOXY_WARNINGS: ON/OFF (Default: OFF)
* --enable-doxygen-errors: enable/disable (Default: disable)
Tools
-----
- Fixed several issues in ph5diff
The parallel logic for the ph5diff tool inside the shared h5diff code was
refactored and cleaned up to fix several issues with the ph5diff tool. This
fixed:
- several concurrency issues in ph5diff that can result in interleaved
output,
- an issue where output can sometimes be dropped when it ends up in
ph5diff's output overflow file, and
- an issue where MPI_Init was called after HDF5 had been initialized,
preventing the library from setting up an MPI communicator attribute
to perform library cleanup on MPI_Finalize.
Performance
-------------
-
Fortran API
-----------
-
High-Level Library
------------------
-
Fortran High-Level APIs
-----------------------
-
Documentation
-------------
-
F90 APIs
--------
-
C++ APIs
--------
-
Testing
-------
-
Platforms Tested
===================
- HDF5 is tested with the two latest macOS versions that are available
on github runners. As new major macOS versions become available, HDF5
will discontinue support for the older version and add the new latest
version to its list of compatible systems, along with the previous
version.
Linux 6.8.0-1010-aws GNU gcc, gfortran, g++
#10-Ubuntu SMP 2024 x86_64 (Ubuntu 13.2.0-23ubuntu4) 13.2.0
GNU/Linux Ubuntu 24.04 Ubuntu clang version 18.1.3 (1ubuntu1)
Intel(R) oneAPI DPC++/C++ Compiler 2024.2.0
ifx (IFX) 2024.2.0 20240602
(cmake and autotools)
Linux 6.5.0-1018-aws GNU gcc, gfortran, g++
#18-Ubuntu SMP x86_64 GNU/Linux (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0
Ubuntu 22.04 Ubuntu clang version 14.0.0-1ubuntu1
Intel(R) oneAPI DPC++/C++ Compiler 2024.0.2
ifx (IFX) 2024.0.2 20231213
(cmake and autotools)
Linux 5.14.21-cray_shasta_c cray-mpich/8.1.28
#1 SMP x86_64 GNU/Linux cce/15.0.0
(frontier) gcc/13.2
(cmake)
Linux 5.14.0-427.24.1.el9_4 GNU gcc, gfortran, g++ (Red Hat 11.4.1-3)
#1 SMP x86_64 GNU/Linux clang version 17.0.6
Rocky 9 Intel(R) oneAPI DPC++/C++ Compiler 2024.2.0
ifx (IFX) 2024.2.0
(cmake and autotools)
Linux-4.18.0-553.16.1.1toss.t4 openmpi/4.1.2
#1 SMP x86_64 GNU/Linux clang 14.0.6
(corona, dane) GCC 12.1.1
Intel(R) oneAPI DPC++/C++ Compiler 2023.2.1
ifx (IFX) 2023.2.1
Linux-4.18.0-553.5.1.1toss.t4 openmpi/4.1/4.1.6
#1 SMP x86_64 GNU/Linux clang 16.0.6
(eclipse) GCC 12.3.0
Intel(R) oneAPI DPC++/C++ Compiler 2024.0.2
ifx (IFX) 2024.0.2
(cmake)
Linux 4.14.0-115.35.1.3chaos spectrum-mpi/rolling-release
#1 SMP ppc64le GNU/Linux clang 17.0.6
(vortex) GCC 12.2.1
nvhpc 24.1
XL 2023.06.28
(cmake)
Linux-4.14.0-115.35.1 spectrum-mpi/rolling-release
#1 SMP ppc64le GNU/Linux clang 14.0.5, 15.0.6
(lassen) GCC 8.3.1
XL 2021.09.22, 2022.08.05
(cmake)
Linux 3.10.0-1160.36.2.el7.ppc64 gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-39)
#1 SMP ppc64be GNU/Linux g++ (GCC) 4.8.5 20150623 (Red Hat 4.8.5-39)
Power8 (echidna) GNU Fortran (GCC) 4.8.5 20150623 (Red Hat 4.8.5-39)
Linux 3.10.0-1160.80.1.el7 GNU C (gcc), Fortran (gfortran), C++ (g++)
#1 SMP x86_64 GNU/Linux compilers:
Centos7 Version 4.8.5 20150623 (Red Hat 4.8.5-4)
(jelly/kituo/moohan) Version 4.9.3, Version 7.2.0, Version 8.3.0,
Version 9.1.0, Version 10.2.0
Intel(R) C (icc), C++ (icpc), Fortran (icc)
compilers:
Version 17.0.0.098 Build 20160721
GNU C (gcc) and C++ (g++) 4.8.5 compilers
with NAG Fortran Compiler Release 7.1(Hanzomon)
Intel(R) C (icc) and C++ (icpc) 17.0.0.098 compilers
with NAG Fortran Compiler Release 7.1(Hanzomon)
MPICH 3.1.4 compiled with GCC 4.9.3
MPICH 3.3 compiled with GCC 7.2.0
OpenMPI 3.1.3 compiled with GCC 7.2.0 and 4.1.2
compiled with GCC 9.1.0
PGI C, Fortran, C++ for 64-bit target on
x86_64;
Versions 18.4.0 and 19.10-0
NVIDIA nvc, nvfortran and nvc++ version 22.5-0
(autotools and cmake)
Linux-3.10.0-1160.119.1.1chaos openmpi/4.1.4
#1 SMP x86_64 GNU/Linux clang 16.0.6
(skybridge) Intel(R) oneAPI DPC++/C++ Compiler 2023.2.0
ifx (IFX) 2023.2.0
(cmake)
Linux-3.10.0-1160.90.1.1chaos openmpi/4.1
#1 SMP x86_64 GNU/Linux clang 16.0.6
(attaway) GCC 12.1.0
Intel(R) oneAPI DPC++/C++ Compiler 2024.0.2
ifx (IFX) 2024.0.2
(cmake)
Linux 2.6.32-573.22.1.el6 GNU C (gcc), Fortran (gfortran), C++ (g++)
#1 SMP x86_64 GNU/Linux compilers:
Centos6 Version 4.4.7 20120313
(platypus) Version 4.9.3, 5.3.0, 6.2.0
MPICH 3.1.4 compiled with GCC 4.9.3
PGI C, Fortran, C++ for 64-bit target on
x86_64;
Version 19.10-0
Windows 10 x64 Visual Studio 2019 w/ clang 12.0.0
with MSVC-like command-line (C/C++ only - cmake)
Visual Studio 2019 w/ Intel (C/C++ only - cmake)
Visual Studio 2022 w/ clang 17.0.3
with MSVC-like command-line (C/C++ only - cmake)
Visual Studio 2022 w/ Intel C/C++ oneAPI 2023 (cmake)
Visual Studio 2019 w/ MSMPI 10.1 (C only - cmake)
Known Problems
==============
- When building with the NAG Fortran compiler using the Autotools and libtool
2.4.2 or earlier, the -shared flag will be missing '-Wl,', which will cause
compilation to fail. This is due to a bug in libtool that was fixed in 2012
and released in 2.4.4 in 2014.
- When the library detects and builds in support for the _Float16 datatype, an
issue has been observed on at least one MacOS 14 system where the library
fails to initialize due to not being able to detect the byte order of the
_Float16 type (https://github.com/HDFGroup/hdf5/issues/4310):
#5: H5Tinit_float.c line 308 in H5T__fix_order(): failed to detect byte order
major: Datatype
minor: Unable to initialize object
If this issue is encountered, support for the _Float16 type can be disabled
with a configuration option:
CMake: HDF5_ENABLE_NONSTANDARD_FEATURE_FLOAT16=OFF
Autotools: --disable-nonstandard-feature-float16
- When HDF5 is compiled with NVHPC versions 23.5 - 23.9 (additional versions may
also be applicable) and with -O2 (or higher) and -DNDEBUG, test failures occur
in the following tests:
H5PLUGIN-filter_plugin
H5TEST-flush2
H5TEST-testhdf5-base
MPI_TEST_t_filters_parallel
Sporadic failures (even with lower -O levels):
Java JUnit-TestH5Pfapl
Java JUnit-TestH5D
Also, NVHPC will fail to compile the test/tselect.c test file with a compiler
error of 'use of undefined value' when the optimization level is -O2 or higher.
This is confirmed to be a bug in the nvc compiler that has been fixed as of
23.11. If you are using an affected version of the NVidia compiler, the
work-around is to set the optimization level to -O1.
https://forums.developer.nvidia.com/t/hdf5-no-longer-compiles-with-nv-23-9/269045
- CMake files do not behave correctly with paths containing spaces.
Do not use spaces in paths because the required escaping for handling spaces
results in very complex and fragile build files.
- At present, metadata cache images may not be generated by parallel
applications. Parallel applications can read files with metadata cache
images, but since this is a collective operation, a deadlock is possible
if one or more processes do not participate.
- The subsetting option in ph5diff currently will fail and should be avoided.
The subsetting option works correctly in serial h5diff.
- Flang Fortran compilation will fail (last check version 17) due to not yet
implemented: (1) derived type argument passed by value (H5VLff.F90),
and (2) support for REAL with KIND = 2 in intrinsic SPACING used in testing.
- Fortran tests HDF5_1_8.F90 and HDF5_F03.F90 will fail with Cray compilers
greater than version 16.0 due to a compiler bug. The latest version verified
as failing was version 17.0.
- Several tests currently fail on certain platforms:
MPI_TEST-t_bigio fails with spectrum-mpi on ppc64le platforms.
MPI_TEST-t_subfiling_vfd and MPI_TEST_EXAMPLES-ph5_subfiling fail with
cray-mpich on theta and with XL compilers on ppc64le platforms.
MPI_TEST_testphdf5_tldsc fails with cray-mpich 7.7 on cori and theta.
- File space may not be released when overwriting or deleting certain nested
variable length or reference types.
- Known problems in previous releases can be found in the HISTORY*.txt files
in the HDF5 source. Please report any new problems found to
help@hdfgroup.org.
CMake vs. Autotools installations
=================================
While both build systems produce similar results, there are differences.
Each system produces the same set of folders on Linux (only CMake works
on standard Windows); bin, include, lib and share. Autotools places the
COPYING and RELEASE.txt file in the root folder, CMake places them in
the share folder.
The bin folder contains the tools and the build scripts. Additionally, CMake
creates dynamic versions of the tools with the suffix "-shared". Autotools
installs one set of tools depending on the "--enable-shared" configuration
option.
build scripts
-------------
Autotools: h5c++, h5cc, h5fc
CMake: h5c++, h5cc, h5hlc++, h5hlcc
The include folder holds the header files and the fortran mod files. CMake
places the fortran mod files into separate shared and static subfolders,
while Autotools places one set of mod files into the include folder. Because
CMake produces a tools library, the header files for tools will appear in
the include folder.
The lib folder contains the library files, and CMake adds the pkgconfig
subfolder with the hdf5*.pc files used by the bin/build scripts created by
the CMake build. CMake separates the C interface code from the fortran code by
creating C-stub libraries for each Fortran library. In addition, only CMake
installs the tools library. The names of the szip libraries are different
between the build systems.
The share folder will have the most differences because CMake builds include
a number of CMake specific files for support of CMake's find_package and support
for the HDF5 Examples CMake project.
The issues with the gif tool are:
HDFFV-10592 CVE-2018-17433
HDFFV-10593 CVE-2018-17436
HDFFV-11048 CVE-2020-10809
These CVE issues have not yet been addressed and are avoided by not building
the gif tool by default. Enable building the High-Level tools with these options:
autotools: --enable-hlgiftools
cmake: HDF5_BUILD_HL_GIF_TOOLS=ON
%%%%1.14.4%%%%
HDF5 version 1.14.4-2 released on 2024-04-15
================================================================================
INTRODUCTION
============
This document describes the differences between this release and the previous
HDF5 release. It contains information on the platforms tested and known
problems in this release. For more details check the HISTORY*.txt files in the
HDF5 source.
Note that documentation in the links below will be updated at the time of each
final release.
Links to HDF5 documentation can be found on:
https://portal.hdfgroup.org/documentation/
The official HDF5 releases can be obtained from:
https://www.hdfgroup.org/downloads/hdf5/
Changes from release to release and new features in the HDF5-1.14.x release series
can be found at:
https://portal.hdfgroup.org/documentation/hdf5-docs/release_specific_info.html
If you have any questions or comments, please send them to the HDF Help Desk:
help@hdfgroup.org
CONTENTS
========
- New Features
- Support for new platforms and languages
- Bug Fixes since HDF5-1.14.3
- Platforms Tested
- Known Problems
- CMake vs. Autotools installations
New Features
============
Configuration:
-------------
- Added configure options for enabling/disabling non-standard programming
language features
* Added a new configuration option that allows enabling or disabling of
support for features that are extensions to programming languages, such
as support for the _Float16 datatype:
CMake: HDF5_ENABLE_NONSTANDARD_FEATURES (ON/OFF) (Default: ON)
Autotools: --enable-nonstandard-features (yes/no) (Default: yes)
When this option is enabled, configure time checks are still performed
to ensure that a feature can be used properly, but these checks may not
be sufficient when compiler support for a feature is incomplete or broken,
resulting in library build failures. When set to OFF/no, this option
provides a way to disable support for all non-standard features to avoid
these issues. Individual features can still be re-enabled with their
respective configuration options.
* Added a new configuration option that allows enabling or disabling of
support for the _Float16 C datatype:
CMake: HDF5_ENABLE_NONSTANDARD_FEATURE_FLOAT16 (ON/OFF) (Default: ON)
Autotools: --enable-nonstandard-feature-float16 (yes/no) (Default: yes)
While support for the _Float16 C datatype can generally be detected and
used properly, some compilers have incomplete support for the datatype
and will pass configure time checks while still failing to build HDF5.
This option provides a way to disable support for the _Float16 datatype
when the compiler doesn't have the proper support for it.
- Deprecate bin/cmakehdf5 script
With the improvements made in CMake since version 3.23 and the addition
of CMake preset files, this script is no longer necessary.
See INSTALL_CMake.txt file, Section X: Using CMakePresets.json for compiling
- Overhauled LFS support checks
In 2024, we can assume that Large File Support (LFS) exists on all
systems we support, though it may require flags to enable it,
particularly when building 32-bit binaries. The HDF5 source does
not use any of the 64-bit specific API calls (e.g., ftello64)
or explicit 64-bit offsets via off64_t.
Autotools
* We now use AC_SYS_LARGEFILE to determine how to support LFS. We
previously used a custom m4 script for this.
CMake
* The HDF_ENABLE_LARGE_FILE option (advanced) has been removed
* We no longer run a test program to determine if LFS works, which
will help with cross-compiling
* On Linux we now unilaterally set -D_LARGEFILE_SOURCE and
-D_FILE_OFFSET_BITS=64, regardless of 32/64 bit system. CMake
doesn't offer a nice equivalent to AC_SYS_LARGEFILE and since
those options do nothing on 64-bit systems, this seems safe and
covers all our bases. We don't set -D_LARGEFILE64_SOURCE since
we don't use any of the POSIX 64-bit specific API calls like
ftello64, as noted above.
* We didn't test for LFS support on non-Linux platforms. We've added
comments for how LFS should probably be supported on AIX and Solaris,
which seem to be alive, though uncommon. PRs would be appreciated if
anyone wishes to test this.
This overhaul also fixes GitHub #2395, which points out that the LFS flags
used when building with CMake differ based on whether CMake has been
run before. The LFS check program that caused this problem no longer exists.
- The CMake HDF5_ENABLE_DEBUG_H5B option has been removed
This enabled some additional version-1 B-tree checks. These have been
removed so the option is no longer necessary.
This option was CMake-only and marked as advanced.
- New option for building with static CRT in Windows
The following option has been added:
HDF5_BUILD_STATIC_CRT_LIBS "Build With Static Windows CRT Libraries" OFF
Because our minimum CMake is 3.18, the macro to change runtime flags no longer
works as CMake changed the default behavior in CMake 3.15.
Fixes GitHub issue #3984
- Added support for the new MSVC preprocessor
Microsoft added support for a new, standards-conformant preprocessor
to MSVC, which can be enabled with the /Zc:preprocessor option. This
preprocessor would trip over our HDopen() variadic function-like
macro, which uses a feature that only works with the legacy preprocessor.
ifdefs have been added that select the correct HDopen() form and
allow building HDF5 with the /Zc:preprocessor option.
The HDopen() macro is located in an internal header file and only
affects building the HDF5 library from source.
Fixes GitHub #2515
- Renamed HDF5_ENABLE_USING_MEMCHECKER to HDF5_USING_ANALYSIS_TOOL
The HDF5_USING_ANALYSIS_TOOL is used to indicate to test macros that
an analysis tool is being used and that the tests should not use
the runTest.cmake macros and it's variations. The analysis tools,
like valgrind, test the macro code instead of the program under test.
HDF5_ENABLE_USING_MEMCHECKER is still used for controlling the HDF5
define, H5_USING_MEMCHECKER.
- New option for building and naming tools in CMake
The following option has been added:
HDF5_BUILD_STATIC_TOOLS "Build Static Tools Not Shared Tools" OFF
The default will build shared tools unless BUILD_SHARED_LIBS = OFF.
Tools will no longer have "-shared" as only one set of tools will be created.
- Incorporated HDF5 examples repository into HDF5 library.
The HDF5Examples folder is equivalent to the hdf5-examples repository.
This enables building and testing the examples
during the library build process or after the library has been installed.
Previously, the hdf5-examples archives were downloaded
for packaging with the library. Now the examples can be built
and tested without a packaged install of the library.
However, to maintain the ability to use the HDF5Examples with an installed
library, it is necessary to map the option names used by the library
to those used by the examples. The typical pattern is:
<example option> = <library option>
HDF_BUILD_FORTRAN = ${HDF5_BUILD_FORTRAN}
- Added new option for CMake to mark tests as SKIPPED.
HDF5_DISABLE_TESTS_REGEX is a REGEX string that will be checked with
test names and if there is a match then that test's property will be
set to DISABLED. HDF5_DISABLE_TESTS_REGEX can be initialized on the
command line: "-DHDF5_DISABLE_TESTS_REGEX:STRING=<regex>"
See CMake documentation for regex-specification.
- Added defaults to CMake for long double conversion checks
HDF5 performs a couple of checks at build time to see if long double
values can be converted correctly (IBM's Power architecture uses a
special format for long doubles). These checks were performed using
TRY_RUN, which is a problem when cross-compiling.
These checks now use default values appropriate for most non-Power
systems when cross-compiling. The cache values can be pre-set if
necessary, which will preempt both the TRY_RUN and the default.
Affected values:
H5_LDOUBLE_TO_LONG_SPECIAL (default no)
H5_LONG_TO_LDOUBLE_SPECIAL (default no)
H5_LDOUBLE_TO_LLONG_ACCURATE (default yes)
H5_LLONG_TO_LDOUBLE_CORRECT (default yes)
H5_DISABLE_SOME_LDOUBLE_CONV (default no)
Fixes GitHub #3585
Library:
--------
- Relaxed behavior of H5Pset_page_buffer_size() when opening files
This API call sets the size of a file's page buffer cache. This call
was extremely strict about matching its parameters to the file strategy
and page size used to create the file, requiring a separate open of the
file to obtain these parameters.
These requirements have been relaxed when using the fapl to open
a previously-created file:
* When opening a file that does not use the H5F_FSPACE_STRATEGY_PAGE
strategy, the setting is ignored and the file will be opened, but
without a page buffer cache. This was previously an error.
* When opening a file that has a page size larger than the desired
page buffer cache size, the page buffer cache size will be increased
to the file's page size. This was previously an error.
The behavior when creating a file using H5Pset_page_buffer_size() is
unchanged.
Fixes GitHub issue #3382
- Added support for _Float16 16-bit half-precision floating-point datatype
Support for the _Float16 C datatype has been added on platforms where:
- The _Float16 datatype and its associated macros (FLT16_MIN, FLT16_MAX,
FLT16_EPSILON, etc.) are available
- A simple test program that converts between the _Float16 datatype and
other datatypes with casts can be successfully compiled and run at
configure time. Some compilers appear to be buggy or feature-incomplete
in this regard and will generate calls to compiler-internal functions
for converting between the _Float16 datatype and other datatypes, but
will not link these functions into the build, resulting in build
failures.
The following new macros have been added:
H5_HAVE__FLOAT16 - This macro is defined in H5pubconf.h and will have
the value 1 if support for the _Float16 datatype is
available. It will not be defined otherwise.
H5_SIZEOF__FLOAT16 - This macro is defined in H5pubconf.h and will have
a value corresponding to the size of the _Float16
datatype, as computed by sizeof(). It will have the
value 0 if support for the _Float16 datatype is not
available.
H5_HAVE_FABSF16 - This macro is defined in H5pubconf.h and will have the
value 1 if the fabsf16 function is available for use.
H5_LDOUBLE_TO_FLOAT16_CORRECT - This macro is defined in H5pubconf.h and
will have the value 1 if the platform can
correctly convert long double values to
_Float16. Some compilers have issues with
this.
H5T_NATIVE_FLOAT16 - This macro maps to the ID of an HDF5 datatype representing
the native C _Float16 datatype for the platform. If
support for the _Float16 datatype is not available, the
macro will map to H5I_INVALID_HID and should not be used.
H5T_IEEE_F16BE - This macro maps to the ID of an HDF5 datatype representing
a big-endian IEEE 754 16-bit floating-point datatype. This
datatype is available regardless of whether _Float16 support
is available or not.