Merge branch 'HDFGroup:develop' into develop

.clang-format

@@ -68,12 +68,14 @@ StatementMacros:
   - CATCH
   - END_MEMBERS
   - FUNC_ENTER_API
+  - FUNC_ENTER_API_NAMECHECK_ONLY
   - FUNC_ENTER_NOAPI
   - FUNC_ENTER_NOAPI_NOERR
   - FUNC_ENTER_NOAPI_NOINIT
   - FUNC_ENTER_NOAPI_NOINIT_NOERR
   - FUNC_ENTER_PACKAGE
   - FUNC_LEAVE_API
+  - FUNC_LEAVE_API_NAMECHECK_ONLY
   - FUNC_LEAVE_NOAPI
   - FUNC_LEAVE_NOAPI_NOFS
   - H5E_END_TRY

doxygen/aliases

@@ -243,7 +243,7 @@ ALIASES += ref_mdc_in_hdf5="<a href=\"https://portal.hdfgroup.org/display/HDF5/M
 ALIASES += ref_mdc_logging="<a href=\"https://portal.hdfgroup.org/display/HDF5/H5F_START_MDC_LOGGING\">Metadata Cache Logging</a>"
 ALIASES += ref_news_112="<a href=\"https://portal.hdfgroup.org/documentation/hdf5-docs/release_specifics/new_features_1_12.html\">New Features in HDF5 Release 1.12</a>"
 ALIASES += ref_h5ocopy="<a href=\"https://portal.hdfgroup.org/display/HDF5/Copying+Committed+Datatypes+with+H5Ocopy\">Copying Committed Datatypes with H5Ocopy()</a>"
-ALIASES += ref_sencode_fmt_change="<a href=\"https://portal.hdfgroup.org/pages/viewpage.action?pageId=58100093&preview=/58100093/58100094/encode_format_RFC.pdf\">RFC H5Secnode() / H5Sdecode() Format Change</a>"
+ALIASES += ref_sencode_fmt_change="<a href=\"https://docs.hdfgroup.org/hdf5/rfc/H5Sencode_format.docx.pdf\">RFC H5Sencode() / H5Sdecode() Format Change</a>"
 ALIASES += ref_vlen_strings="\Emph{Creating variable-length string datatypes}"
 ALIASES += ref_vol_doc="VOL documentation"
 

doxygen/examples/intro_SWMR.html

@@ -4,10 +4,13 @@
 
 <h2 id="introduction-to-swmr">Introduction to SWMR</h2>
 <p>The Single-Writer / Multiple-Reader (SWMR) feature enables multiple processes to read an HDF5 file while it is being written to (by a single process) without using locks or requiring communication between processes.</p>
+<p><img src=tutr-swmr1.png alt="tutr-swmr1.png" width=500>
 <p>All communication between processes must be performed via the HDF5 file. The HDF5 file under SWMR access must reside on a system that complies with POSIX write() semantics.</p>
 <p>The basic engineering challenge for this to work was to ensure that the readers of an HDF5 file always see a coherent (though possibly not up to date) HDF5 file.</p>
 <p>The issue is that when writing data there is information in the metadata cache in addition to the physical file on disk:</p>
+<p><img src=tutr-swmr2.png alt="tutr-swmr2.png" width=500>
 <p>However, the readers can only see the state contained in the physical file:</p>
+<p><img src=tutr-swmr3.png alt="tutr-swmr3.png" width=500>
 <p>The SWMR solution implements dependencies on when the metadata can be flushed to the file. This ensures that metadata cache flush operations occur in the proper order, so that there will never be internal file pointers in the physical file that point to invalid (unflushed) file addresses.</p>
 <p>A beneficial side effect of using SWMR access is better fault tolerance. It is more difficult to corrupt a file when using SWMR.</p>
 <h2 id="documentation">Documentation</h2>
@@ -46,7 +49,8 @@ <h3 id="swmr-writer">SWMR Writer:</h3>
 Periodically flush data.</p>
 <h4 id="example-code">Example Code:</h4>
 <p>Create the file using the latest file format property:</p>
-<p>   fapl = H5Pcreate (H5P_FILE_ACCESS);
+<p>
+   fapl = H5Pcreate (H5P_FILE_ACCESS);
    status = H5Pset_libver_bounds (fapl, H5F_LIBVER_LATEST, H5F_LIBVER_LATEST);
    fid = H5Fcreate (filename, H5F_ACC_TRUNC, H5P_DEFAULT, fapl);
 [Create objects (files, datasets, ...). Close any attributes and named datatype objects. Groups and datasets may remain open before starting SWMR access to them.]</p>

doxygen/examples/intro_VDS.html

@@ -5,11 +5,11 @@
 <p>The HDF5 Virtual Dataset (VDS) feature enables users to access data in a collection of HDF5 files as a single HDF5 dataset and to use the HDF5 APIs to work with that dataset.</p>
 <p>For example, your data may be collected into four files:</p>
 
-<p><img src="tutrvds-multimgs.png" ></center>
+<p><img src="tutrvds-multimgs.png" alt="tutrvds-multimgs.png" width=750></center>
 
 <p>You can map the datasets in the four files into a single VDS that can be accessed just like any other dataset:</p>
 
-<p><img src="tutrvds-snglimg.png" ></center>
+<p><img src="tutrvds-snglimg.png" alt="tutrvds-snglimg.png" width=500></center>
 
 <p>The mapping between a VDS and the HDF5 source datasets is persistent and transparent to an application. If a source file is missing the fill value will be displayed.</p>
 <p>See the Virtual (VDS) Documentation for complete details regarding the VDS feature.</p>
@@ -42,6 +42,7 @@
 <p>Programming Examples
 Example 1
 This example creates three HDF5 files, each with a one-dimensional dataset of 6 elements. The datasets in these files are the source datasets that are then used to create a 4 x 6 Virtual Dataset with a fill value of -1. The first three rows of the VDS are mapped to the data from the three source datasets as shown below:</p>
+<p><img src="tutrvds-ex.png" alt="tutrvds-ex.png" width=500></p>
 <p>In this example the three source datasets are mapped to the VDS with this code:</p>
 <pre><code>src\_space = H5Screate\_simple (RANK1, dims, NULL);
 for (i = 0; i &lt; 3; i++) {
@@ -67,4 +68,5 @@
 The h5dump utility can be used to view a VDS. The h5dump output for a VDS looks exactly like that for any other dataset. If h5dump cannot find a source dataset then the fill value will be displayed.</p>
 <p>You can determine that a dataset is a VDS by looking at its properties with h5dump -p. It will display each source dataset mapping, beginning with Mapping 0. Below is an excerpt of the output of h5dump -p on the vds.h5 file created in Example 1.You can see that the entire source file a.h5 is mapped to the first row of the /VDS dataset:</p>
 
+<p><img src="tutrvds-map.png" alt="tutrvds-map.png" width=650></p>
 </body></html>

doxygen/img/images_to_copy.dox

@@ -0,0 +1,11 @@
+/** \page HTML_IMGS Images for html files
+
+<img src=tutrvds-map.png>
+<img src=tutrvds-ex.png>
+<img src=tutr-swmr3.png>
+<img src=tutr-swmr2.png>
+<img src=tutr-swmr1.png>
+<img src=tutrvds-snglimg.png>
+<img src=tutrvds-multimgs.png>
+
+*/

hl/src/H5DO.c

@@ -80,20 +80,6 @@ H5DOread_chunk(hid_t dset_id, hid_t dxpl_id, const hsize_t *offset, uint32_t *fi
  *
  * Return:      Non-negative on success/Negative on failure
  *
- * Note:
- * 	This routine is copied from the fast forward feature branch: features/hdf5_ff
- *	src/H5FF.c:H5DOappend() with the following modifications:
- * 	1) Remove and replace macro calls such as
- *		FUNC_ENTER_API, H5TRACE, HGOTO_ERROR
- * 	   accordingly because hl does not have these macros
- *	2) Replace H5I_get_type() by H5Iget_type()
- *	3) Replace H5P_isa_class() by H5Pisa_class()
- *	4) Fix a bug in the following: replace extension by size[axis]
- *		if(extension < old_size) {
- *		  ret_value = FAIL;
- *		  goto done;
- *   		}
- *
  *-------------------------------------------------------------------------
  */
 herr_t

hl/tools/h5watch/h5watch.c

@@ -785,13 +785,13 @@ main(int argc, char *argv[])
     h5tools_init();
 
     /* To exit from h5watch for SIGTERM signal */
-    if (HDsignal(SIGTERM, catch_signal) == SIG_ERR) {
+    if (signal(SIGTERM, catch_signal) == SIG_ERR) {
         error_msg("An error occurred while setting a signal handler.\n");
         leave(EXIT_FAILURE);
     }
 
     /* To exit from h5watch for SIGINT signal */
-    if (HDsignal(SIGINT, catch_signal) == SIG_ERR) {
+    if (signal(SIGINT, catch_signal) == SIG_ERR) {
         error_msg("An error occurred while setting a signal handler.\n");
         leave(EXIT_FAILURE);
     }

src/H5Clog_json.c

@@ -245,7 +245,7 @@ H5C__log_json_set_up(H5C_log_info_t *log_info, const char log_location[], int mp
     /* Open log file and set it to be unbuffered */
     if (NULL == (json_udata->outfile = fopen(file_name, "w")))
         HGOTO_ERROR(H5E_CACHE, H5E_LOGGING, FAIL, "can't create mdc log file");
-    HDsetbuf(json_udata->outfile, NULL);
+    setbuf(json_udata->outfile, NULL);
 
 done:
     if (file_name)

src/H5Clog_trace.c

@@ -240,7 +240,7 @@ H5C__log_trace_set_up(H5C_log_info_t *log_info, const char log_location[], int m
     /* Open log file and set it to be unbuffered */
     if (NULL == (trace_udata->outfile = fopen(file_name, "w")))
         HGOTO_ERROR(H5E_CACHE, H5E_LOGGING, FAIL, "can't create mdc log file");
-    HDsetbuf(trace_udata->outfile, NULL);
+    setbuf(trace_udata->outfile, NULL);
 
     /* Write the header */
     fprintf(trace_udata->outfile, "### HDF5 metadata cache trace file version 1 ###\n");

src/H5Fmodule.h

@@ -891,7 +891,8 @@
  * @see <a href="https://portal.hdfgroup.org/documentation/hdf5-docs/advanced_topics/file_image_ops.html">
  * HDF5 File Image Operations</a>
  * section for information on more advanced usage of the Memory file driver, and
- * @see <a href="http://www.hdfgroup.org/HDF5/doc/Advanced/ModifiedRegionWrites/ModifiedRegionWrites.pdf">
+ * @see <a
+ * href="https://docs.hdfgroup.org/archive/support/HDF5/doc/Advanced/ModifiedRegionWrites/ModifiedRegionWrites.pdf">
  * Modified Region Writes</a>
  * section for information on how to set write operations so that only modified regions are written
  * to storage.

src/H5Ipublic.h

@@ -399,7 +399,7 @@ H5_DLL int H5Iget_ref(hid_t id);
  * \brief Creates and returns a new ID type
  *
  * \param[in] hash_size Minimum hash table size (in entries) used to store IDs
- *                      for the new type
+ *                      for the new type (unused in 1.8.13 and later)
  * \param[in] reserved Number of reserved IDs for the new type
  * \param[in] free_func Function used to deallocate space for a single ID
  *
@@ -409,7 +409,8 @@ H5_DLL int H5Iget_ref(hid_t id);
  *          identifier for it.
  *
  *          The \p hash_size parameter indicates the minimum size of the hash
- *          table used to store IDs in the new type.
+ *          table used to store IDs in the new type. This parameter is unused
+ *          in 1.8.13 and later, when the implementation of ID storage changed.
  *
  *          The \p reserved parameter indicates the number of IDs in this new
  *          type to be reserved. Reserved IDs are valid IDs which are not

src/H5TS.c

@@ -414,10 +414,12 @@ H5TS__mutex_acquire(H5TS_mutex_t *mutex, unsigned int lock_count, bool *acquired
  *--------------------------------------------------------------------------
  */
 herr_t
-H5TSmutex_acquire(unsigned int lock_count, bool *acquired){
+H5TSmutex_acquire(unsigned int lock_count, bool *acquired)
+{
     FUNC_ENTER_API_NAMECHECK_ONLY
 
-        FUNC_LEAVE_API_NAMECHECK_ONLY(H5TS__mutex_acquire(&H5_g.init_lock, lock_count, acquired))}
+    FUNC_LEAVE_API_NAMECHECK_ONLY(H5TS__mutex_acquire(&H5_g.init_lock, lock_count, acquired))
+}
 /* end H5TSmutex_acquire() */
 
 /*--------------------------------------------------------------------------
@@ -440,7 +442,8 @@ H5TSmutex_acquire(unsigned int lock_count, bool *acquired){
  *
  *--------------------------------------------------------------------------
  */
-herr_t H5TS_mutex_lock(H5TS_mutex_t *mutex)
+herr_t
+H5TS_mutex_lock(H5TS_mutex_t *mutex)
 {
     herr_t ret_value = SUCCEED;
 

src/H5private.h

@@ -288,14 +288,6 @@
 #define SUCCEED 0
 #define FAIL    (-1)
 
-/* The HDF5 library uses the symbol `ERR` frequently.  So do
- * header files for libraries such as curses(3), terminfo(3), etc.
- * Remove its definition here to avoid clashes with HDF5.
- */
-#ifdef ERR
-#undef ERR
-#endif
-
 /* number of members in an array */
 #ifndef NELMTS
 #define NELMTS(X) (sizeof(X) / sizeof(X[0]))
@@ -798,18 +790,9 @@ H5_DLL H5_ATTR_CONST int Nflock(int fd, int operation);
 #ifndef HDremove
 #define HDremove(S) remove(S)
 #endif
-#ifndef HDrename
-#define HDrename(OLD, NEW) rename(OLD, NEW)
-#endif
-#ifndef HDrewind
-#define HDrewind(F) rewind(F)
-#endif
 #ifndef HDrmdir
 #define HDrmdir(S) rmdir(S)
 #endif
-#ifndef HDsetbuf
-#define HDsetbuf(F, S) setbuf(F, S)
-#endif
 #ifndef HDsetenv
 #define HDsetenv(N, V, O) setenv(N, V, O)
 #endif
@@ -825,9 +808,6 @@ H5_DLL H5_ATTR_CONST int Nflock(int fd, int operation);
 #ifndef HDsigemptyset
 #define HDsigemptyset(S) sigemptyset(S)
 #endif
-#ifndef HDsignal
-#define HDsignal(N, F) signal(N, F)
-#endif
 #ifndef HDsleep
 #define HDsleep(N) sleep(N)
 #endif
@@ -850,9 +830,6 @@ H5_DLL H5_ATTR_CONST int Nflock(int fd, int operation);
 #ifndef HDstrtok_r
 #define HDstrtok_r(X, Y, Z) strtok_r(X, Y, Z)
 #endif
-#ifndef HDtmpfile
-#define HDtmpfile() tmpfile()
-#endif
 #ifndef HDunlink
 #define HDunlink(S) unlink(S)
 #endif

test/app_ref.c

@@ -172,7 +172,7 @@ main(void)
 
     RAND_INC(T_ESTACK);
 
-    HDsignal(SIGABRT, &Abrt_Handler);
+    signal(SIGABRT, &Abrt_Handler);
 
     if (H5close() < 0)
         TEST_ERROR;

test/del_many_dense_attrs.c

@@ -69,7 +69,7 @@ main(void)
     h5_reset();
 
     /* To exit from the file for SIGABRT signal */
-    if (HDsignal(SIGABRT, catch_signal) == SIG_ERR)
+    if (signal(SIGABRT, catch_signal) == SIG_ERR)
         TEST_ERROR;
 
     fapl = h5_fileaccess();

test/dt_arith.c

@@ -3261,7 +3261,7 @@ test_conv_flt_1(const char *name, int run_test, hid_t src, hid_t dst)
      * The remainder of this function is executed only by the child if
      * HANDLE_SIGFPE is defined.
      */
-    HDsignal(SIGFPE, fpe_handler);
+    signal(SIGFPE, fpe_handler);
 
     /* What are the names of the source and destination types */
     if (H5Tequal(src, H5T_NATIVE_FLOAT)) {

test/filenotclosed.c

@@ -80,7 +80,7 @@ main(void)
     h5_reset();
 
     /* To exit from the file for SIGABRT signal */
-    if (HDsignal(SIGABRT, catch_signal) == SIG_ERR)
+    if (signal(SIGABRT, catch_signal) == SIG_ERR)
         TEST_ERROR;
 
     fapl = h5_fileaccess();

test/h5test.c

@@ -1784,7 +1784,7 @@ h5_send_message(const char *send, const char *arg1, const char *arg2)
 
     fclose(signalfile);
 
-    HDrename(TMP_SIGNAL_FILE, send);
+    rename(TMP_SIGNAL_FILE, send);
 } /* h5_send_message() */
 
 /*-------------------------------------------------------------------------
@@ -2099,8 +2099,8 @@ h5_compare_file_bytes(char *f1name, char *f2name)
     }
 
     /* Compare each byte and fail if a difference is found */
-    HDrewind(f1ptr);
-    HDrewind(f2ptr);
+    rewind(f1ptr);
+    rewind(f2ptr);
     for (ii = 0; ii < f1size; ii++) {
         if (fread(&f1char, 1, 1, f1ptr) != 1) {
             ret_value = -1;
@@ -2216,7 +2216,7 @@ h5_duplicate_file_by_bytes(const char *orig, const char *dest)
 
     HDfseek(orig_ptr, 0, SEEK_END);
     fsize = (size_t)HDftell(orig_ptr);
-    HDrewind(orig_ptr);
+    rewind(orig_ptr);
 
     dest_ptr = fopen(dest, "wb");
     if (NULL == dest_ptr) {

test/tcheck_version.c

@@ -126,9 +126,9 @@ main(int ac, char **av)
     (void)_CrtSetReportHook2(_CRT_RPTHOOK_INSTALL, handle_crt_abort);
 #endif
     parse(ac, av);
-    HDsignal(SIGABRT, &abort_intercept);
+    signal(SIGABRT, &abort_intercept);
     H5check_version(major, minor, release);
-    HDsignal(SIGABRT, SIG_DFL);
+    signal(SIGABRT, SIG_DFL);
 #ifdef H5_HAVE_WIN32_API
     (void)_CrtSetReportHook2(_CRT_RPTHOOK_REMOVE, handle_crt_abort);
 #endif

testpar/t_2Gio.c

@@ -4270,8 +4270,8 @@ main(int argc, char **argv)
 
 #ifndef H5_HAVE_WIN32_API
     /* Un-buffer the stdout and stderr */
-    HDsetbuf(stderr, NULL);
-    HDsetbuf(stdout, NULL);
+    setbuf(stderr, NULL);
+    setbuf(stdout, NULL);
 #endif
 
     MPI_Init(&argc, &argv);

testpar/t_shapesame.c

@@ -4283,8 +4283,8 @@ main(int argc, char **argv)
 
 #ifndef H5_HAVE_WIN32_API
     /* Un-buffer the stdout and stderr */
-    HDsetbuf(stderr, NULL);
-    HDsetbuf(stdout, NULL);
+    setbuf(stderr, NULL);
+    setbuf(stdout, NULL);
 #endif
 
 #ifdef H5_HAVE_TEST_API

testpar/testphdf5.c

@@ -317,8 +317,8 @@ main(int argc, char **argv)
 
 #ifndef H5_HAVE_WIN32_API
     /* Un-buffer the stdout and stderr */
-    HDsetbuf(stderr, NULL);
-    HDsetbuf(stdout, NULL);
+    setbuf(stderr, NULL);
+    setbuf(stdout, NULL);
 #endif
 
 #ifdef H5_HAVE_TEST_API

tools/lib/h5tools_utils.c

@@ -79,7 +79,7 @@ parallel_print(const char *format, ...)
                 /* Terminate the outbuff at the end of the previous output */
                 outBuff[outBuffOffset] = '\0';
 
-                overflow_file = HDtmpfile();
+                overflow_file = tmpfile();
                 if (overflow_file == NULL)
                     fprintf(rawerrorstream,
                             "warning: could not create overflow file.  Output may be truncated.\n");