Merge PRs #143, 144 to hdf5_1_12 (#630)

* Snapshot version 1.12 release 1-3. Update version to 1.12.1-4. * First cut of the H5 public API documentation. (#80) * First cut of the H5 public API documentation. * Added H5Z "bonus track." * Applied Quincey's patch. * Added the missing patches from Quincey's original patch. * H5PL (complete) and basic H5VL API documentation. * Added H5I API docs. * Added H5L API docs. * First installment from Elena's H5T batch. * Second installment of Elena's H5T batch. * Final installment of Elena's H5T batch. * Full set of current H5F documentation. (#105) * First cut of the H5 public API documentation. * Added H5Z "bonus track." * Applied Quincey's patch. * Added the missing patches from Quincey's original patch. * H5PL (complete) and basic H5VL API documentation. * Added H5I API docs. * Added H5L API docs. * First installment from Elena's H5T batch. * Second installment of Elena's H5T batch. * Final installment of Elena's H5T batch. * Migrated documentation for SWMR functions. * Catching up on MDC functions. * Integrated the H5F MDC function documentation. * Added MDC and parallel H5F functions. * Slightly updated main page. * Added doxygen/dox/H5AC_cache_config_t.dox to MANIFEST. * Doxygen - added (mostly) beginner functions (#112) * Doxygen - added (mostly) beginner functions * Removed duplicate H5Pset_szip function * Add src/H5module.h to MANIFEST. * close #195. (#196) * Update HDF5PluginMacros.cmake * Update HDF5PluginMacros.cmake * Avoid aligned access for references by decoding into temporary buffer and then copying the result into the actual buffer. Update test to be more thorough with using compound datatype fields everywhere. (#206) * Modify temporary rpath for testing in java example scripts. (#230) * Fix undefined left shifting of negative numbers (#338) Undefined Bahavior Sanitizer errored here about left shifting negative numbers. * Fixes various warnings noticed on Windows (#425) * Fixes various warnings noticed on Windows - Adds a prototype for our implementation of vasprintf - Return type of H5_get_utf16_str() is now non-const - Fixes possible uninitialized return type in Wremove_utf8 - Better isolation of fork() code in accum.c:test_swmr_write_big() - Better isolation of non-zlib code in dsets.c:test_filter_delete() - Removed unused variable in trefer.c:test_reference_cmpnd_obj() * Fixes clang-format issues * Applied clang-tidy readability-non-const-parameter warning fixes auto… (#429) * Automatically applied clang-tidy readability-avoid-const-params-in-decls fixes Removes useless const declarations. * Fixed most readability-non-const-parameter warnings These changes were made automatically by clang-tidy, but I manually reverted the changes related to the H5Z_func_t signature. * Reformat source with clang v10.0.1. Co-authored-by: Larry Knox <lrknox@hdfgroup.org> * Added C++11 override keyword where appropriate (#433) Added H5_OVERRIDE macro for compatibility with both C++11 and older. * Various clang tidy warning fixes (#448) * Fixed clang-tidy bugprone-reserved-identifier warnings * Fixed clang-tidy bugprone-assert-side-effect warnings * Fixed clang-tidy bugprone-copy-constructor-init warning * Fixed clang-tidy readability-redundant-preprocessor warning For error_test.c the removed code was already dead, because it was in the else of an `#if H5_USE_16_API` block. Based on H5Location.h, I think p_get_ref_obj_type was meant to be in `#ifndef DOXYGEN_SHOULD_SKIP_THIS` and an `#endif` was missing. Similarly, in the header, getObjTypeByIdx is only in H5_NO_DEPRECATED_SYMBOLS, not DOXYGEN_SHOULD_SKIP_THIS. * Fixed clang-tidy readability-redundant-string-init warnings * Fixed some clang-tidy performance-type-promotion-in-math-fn warnings * Fixed clang-tidy performance-unnecessary-value-param warnings * Reformat source with clang v10.0.1. Co-authored-by: Larry Knox <lrknox@hdfgroup.org> * Removed checks/workarounds for pre-C++89 compatibility (#449) After 30+ years, just assume that the following exist: - extension-less includes - namespaces - std:: - static_cast - bool * Fixed all clang-tidy bugprone-suspicious-string-compare warnings (#451) * Fixed all clang-tidy bugprone-suspicious-string-compare warnings This change was generated entirely by clang-tidy itself. * Reformat code with clang v10.0.1. Co-authored-by: Larry Knox <lrknox@hdfgroup.org> * Remove 2 functions incorrectly merged from develop in a cherry-pick merge of PR #451. * Minor parallel improvements (#519) * Improve MPI error reporting, handled failed operations in parallel tests more nicely, and clean up MPI_Allreduce for determining whether to break collective I/O * Committing clang-format changes Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com> * Clean up MPI-IO VFD tracing support (#520) * Clean up tracing support * Committing clang-format changes Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com> * Brings the native implementation of H5Fdelete() from Bitbucket (#524) * Committing clang-format changes * Brings the native VFD H5Fdelete() implementation from Bitbucket Only brings the 'del' callbacks, not the 'open/close' scheme. * Formatter changes * Committing clang-format changes * Fixes direct VFD callback name * Removes UNUSED macro from family API call * Adds barrier and rank 0 check to MPI-I/O VFD delete * Revert "Adds barrier and rank 0 check to MPI-I/O VFD delete" This reverts commit 909765f. * Revert "Revert "Adds barrier and rank 0 check to MPI-I/O VFD delete"" This reverts commit 9b04bef. * Adds a second barrier after the delete in MPI-I/O VFD * Only delete files in the core VFD when the backing store flag is set * Fixes string issues in multi VFD Also, h5test.c cleanup code now uses H5Fdelete(). * Formatted source * Rework fapl checks for MPI-I/O VFD delete callback Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com> * Minor warning fixes in develop (#526) * Committing clang-format changes * Minor warning fixes Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com> * Removes implementation of my_strdup() from the multi VFD (#527) * Committing clang-format changes * Removes my_strdup() from the multi VFD * Use strdup directly when memory sanity checks are off Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com> * Removes dead H5ST package from the library (#528) * Committing clang-format changes * Removes the unused H5ST package from the library Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com> * Fix HDFFV-11232 (#530) * fixed missed closing of a dataset * fixed missed closing of a dataset * fixed typo in error return * Committing clang-format changes * minor edits * code format * Committing clang-format changes * code format * minor edit * added H5fortkit dependency for H5VLff.F90, HDFFV-11232 Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com> * Fixes incorrect usage of H5I_BADID (#554) * Committing clang-format changes * Fixes incorrect use of H5I_BADID Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com> * Fixes a segfault when H5Pset_mdc_log_options is called multiple times on a fapl (#601) * Committing clang-format changes * Fixes a segfault when H5Pset_mdc_log_options() is called multiple times An internal string is incorrectly freed when the API call is invoked multiple times on a property list, which will usually cause a segfault to occur. On the first call the log location is NULL so the problem doesn't occur. Fixes HDFFV-11239 * Fixes typos Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com> * Fix for a segfault when H5Pset_fapl_log is passed an invalid fapl ID (#607) * Committing clang-format changes * Fixes an issue where H5Pset_fapl_log sefaults when passed an invalid fapl ID This was due to a pointer-containing struct being memset after the first internal API call. If the first call failed, the error condition would check if the pointer was not NULL and then attempt to free it if not. This would lead to the freeing of a wild pointer if an invalid fapl ID were passed in. This was fixed by reordering the memset and adding a test to ensure the problem stays fixed. Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com> * Fixes crashes when size_hint > UINT32_MAX is passed to H5Gcreate1 (#611) * Committing clang-format changes * Fixes incorrect size_hint handling in H5Gcreate1 * Updates the size hint type for group creation * Updates the RELEASE.txt note * Revert "Updates the RELEASE.txt note" This reverts commit 3df386a. * Reverts previous behavior to use a uint32_t struct field * Updates RELEASE.txt Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com> * Revert "Brings the native implementation of H5Fdelete() from Bitbucket (#524)" This reverts commit 38d1b12. * Removed mentions of Wdeclaration-after-statement now that C99 is requ… (#447) * Removed mentions of Wdeclaration-after-statement now that C99 is required * Remove -Werror=declaration-after-statement from error-general file. Co-authored-by: Larry Knox <lrknox@hdfgroup.org> * Made private my_yyinput function static (#618) This prevents it being exported as a public symbol. * Adds const to a few global variables (#623) * Committing clang-format changes * Adds consts to a few global variables Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com> * (fix) Segmentation fault when using a compound type. (#143) * (fix) Segmentation fault when using a compound type. In the case when a compounded attribute is written to dataset followed by writing the data with a data transform function to the dataset will result in a segmentation fault. It turns out the data is classified as compounded while it is not. Now, the state is always reset first to not compounded followed by the existing check if the variable is compounded. * (fix) Removed undesired comment lines. * (fix) Segmentation fault when using a compound type: added test. * (fix) Added the missing cmpd_transform.c file to MANIFEST. * (fix) cmpd_dtransform test: autotools and source header. Added the cmp_dtransform test to the autotools configuration and updated the HDF Group copyright header. Co-authored-by: Jan-Willem Blokland <Jan-Willem.Blokland@Shell.com> * (fix) H5Z_xform_create function and scientific notation (#144) * (fix) H5Z_xform_create function and scientific notation Implemented a more sophisticated check to support scientific notation in the expression of the H5Zset_data_transform function. * (fix) H5Z_xform_create and scientific notation: Added test. Added a test to demonstrate that the parsing of expression which includes scientific notation works correctly. Improved inline comment. Co-authored-by: Jan-Willem Blokland <Jan-Willem.Blokland@Shell.com> * Committing clang-format changes Co-authored-by: Gerd Heber <gheber@hdfgroup.org> Co-authored-by: bljhdf <58825073+bljhdf@users.noreply.github.com> Co-authored-by: H. Joe Lee <hyoklee@hdfgroup.org> Co-authored-by: Quincey Koziol <quincey@koziol.cc> Co-authored-by: Sean McBride <sean@rogue-research.com> Co-authored-by: Dana Robinson <43805+derobins@users.noreply.github.com> Co-authored-by: Quincey Koziol <koziol@lbl.gov> Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Scot Breitenfeld <brtnfld@hdfgroup.org> Co-authored-by: Jan-Willem Blokland <J.W.S.Blokland@XS4All.nl> Co-authored-by: Jan-Willem Blokland <Jan-Willem.Blokland@Shell.com>
HDFGroup · May 6, 2021 · 6ff6504 · 6ff6504
1 parent f573889
commit 6ff6504
Show file tree

Hide file tree

Showing 7 changed files with 189 additions and 14 deletions.
diff --git a/MANIFEST b/MANIFEST
@@ -1139,6 +1139,7 @@
 ./test/cache_tagging.c
 ./test/chunk_info.c
 ./test/cmpd_dset.c
+./test/cmpd_dtransform.c
 ./test/cork.c
 ./test/corrupt_stab_msg.h5
 ./test/cross_read.c

diff --git a/src/H5T.c b/src/H5T.c
@@ -3408,8 +3408,8 @@ H5T__create(H5T_class_t type, size_t size)
  * Note:      Common code for both H5T_copy and H5T_copy_reopen, as part of
  *            the const-correct datatype copying routines.
  *
- * Programmer:	David Young
- *	        January 18, 2020
+ * Programmer:  David Young
+ *              January 18, 2020
  *
  *-------------------------------------------------------------------------
  */
@@ -3462,8 +3462,8 @@ H5T__initiate_copy(const H5T_t *old_dt)
  * Return:    Success:    Pointer to a new copy of the OLD_DT argument.
  *            Failure:    NULL
  *
- * Programmer:	David Young
- *	        January 18, 2020
+ * Programmer:  David Young
+ *              January 18, 2020
  *
  *-------------------------------------------------------------------------
  */
@@ -3490,8 +3490,8 @@ H5T__copy_transient(H5T_t *old_dt)
  * Return:    Success:    Pointer to a new copy of the OLD_DT argument.
  *            Failure:    NULL
  *
- * Programmer:	David Young
- *	        January 18, 2020
+ * Programmer:  David Young
+ *              January 18, 2020
  *
  *-------------------------------------------------------------------------
  */
@@ -3521,8 +3521,8 @@ H5T__copy_all(H5T_t *old_dt)
  *
  * Note:      Common code for both H5T_copy and H5T_copy_reopen.
  *
- * Programmer:	David Young
- *	        January 18, 2020
+ * Programmer:  David Young
+ *              January 18, 2020
  *
  *-------------------------------------------------------------------------
  */
@@ -3794,8 +3794,8 @@ H5T_copy(const H5T_t *old_dt, H5T_copy_t method)
  * Return:    Success:    Pointer to a new copy of the OLD_DT argument.
  *            Failure:    NULL
  *
- * Programmer:	David Young
- *	        January 18, 2020
+ * Programmer:  David Young
+ *              January 18, 2020
  *
  *-------------------------------------------------------------------------
  */
@@ -5270,7 +5270,10 @@ H5T__path_find_real(const H5T_t *src, const H5T_t *dst, const char *name, H5T_co
     } /* end else-if */
 
     /* Set the flag to indicate both source and destination types are compound types
-     * for the optimization of data reading (in H5Dio.c). */
+     * for the optimization of data reading (in H5Dio.c).
+     * Make sure that path->are_compounds is only TRUE for compound types.
+     */
+    path->are_compounds = FALSE;
     if (H5T_COMPOUND == H5T_get_class(src, TRUE) && H5T_COMPOUND == H5T_get_class(dst, TRUE))
         path->are_compounds = TRUE;
 

diff --git a/src/H5Ztrans.c b/src/H5Ztrans.c
@@ -1538,10 +1538,20 @@ H5Z_xform_create(const char *expr)
                     "unable to allocate memory for data transform expression")
 
     /* Find the number of times "x" is used in this equation, and allocate room for storing that many points
+     * A more sophisticated check is needed to support scientific notation.
      */
-    for (i = 0; i < HDstrlen(expr); i++)
-        if (HDisalpha(expr[i]))
+    for (i = 0; i < HDstrlen(expr); i++) {
+        if (HDisalpha(expr[i])) {
+            if ((i > 0) && (i < (HDstrlen(expr) - 1))) {
+                if (((expr[i] == 'E') || (expr[i] == 'e')) &&
+                    (HDisdigit(expr[i - 1]) || (expr[i - 1] == '.')) &&
+                    (HDisdigit(expr[i + 1]) || (expr[i + 1] == '-') || (expr[i + 1] == '+')))
+                    continue;
+            }
+
             count++;
+        }
+    }
 
     /* When there are no "x"'s in the equation (ie, simple transform case),
      * we don't need to allocate any space since no array will have to be

diff --git a/test/CMakeLists.txt b/test/CMakeLists.txt
@@ -331,6 +331,7 @@ set (H5_TESTS
     thread_id # special link
     vol
     timer
+    cmpd_dtransform
 )
 
 macro (ADD_H5_EXE file)

diff --git a/test/Makefile.am b/test/Makefile.am
@@ -60,7 +60,7 @@ TEST_PROG= testhdf5 \
            cache cache_api cache_image cache_tagging lheap ohdr \
            stab gheap evict_on_close farray earray btree2 fheap \
            pool accum hyperslab istore bittests dt_arith page_buffer \
-           dtypes dsets chunk_info cmpd_dset filter_fail extend direct_chunk \
+           dtypes dsets chunk_info cmpd_dset cmpd_dtransform filter_fail extend direct_chunk \
            external efc objcopy objcopy_ref links unlink twriteorder big mtime \
            fillval mount \
            flush1 flush2 app_ref enum set_extent ttsafe enc_dec_plist \

diff --git a/test/cmpd_dtransform.c b/test/cmpd_dtransform.c
@@ -0,0 +1,136 @@
+/* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
+ * Copyright by The HDF Group.                                               *
+ * All rights reserved.                                                      *
+ *                                                                           *
+ * This file is part of HDF5.  The full HDF5 copyright notice, including     *
+ * terms governing use, modification, and redistribution, is contained in    *
+ * the COPYING file, which can be found at the root of the source code       *
+ * distribution tree, or in https://www.hdfgroup.org/licenses.               *
+ * If you do not have access to either file, you may request a copy from     *
+ * help@hdfgroup.org.                                                        *
+ * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * */
+
+/*
+ * Programmer:  Jan-Willem Blokland
+ *              December 1, 2020
+ *
+ * Purpose:     Test writing compounded attribute followed by
+ *              writing data with a data transform function.
+ */
+
+#include "h5test.h"
+
+#define FILENAME "cmpd_dtransform.h5"
+#define LENGTH   11
+
+typedef struct {
+    char name[64];
+    char unit[64];
+} att_t;
+
+int
+main(void)
+{
+    hsize_t dima[] = {1};
+    hsize_t dims[] = {LENGTH};
+    hid_t   str_dtyp_id, att_dtyp_id, file_id, fspace_id, dset_id, att_dspc_id, att_attr_id, dxpl_id;
+
+    /* Compound datatype */
+    att_t *atts = HDmalloc(sizeof(att_t));
+    HDstrcpy(atts[0].name, "Name");
+    HDstrcpy(atts[0].unit, "Unit");
+
+    /* String type */
+    if ((str_dtyp_id = H5Tcopy(H5T_C_S1)) < 0)
+        TEST_ERROR;
+    H5Tset_size(str_dtyp_id, 64);
+
+    /* Attribute type */
+    if ((att_dtyp_id = H5Tcreate(H5T_COMPOUND, sizeof(att_t))) < 0)
+        TEST_ERROR;
+    H5Tinsert(att_dtyp_id, "NAME", HOFFSET(att_t, name), str_dtyp_id);
+    H5Tinsert(att_dtyp_id, "UNIT", HOFFSET(att_t, unit), str_dtyp_id);
+
+    /* Create file. */
+    if ((file_id = H5Fcreate(FILENAME, H5F_ACC_TRUNC, H5P_DEFAULT, H5P_DEFAULT)) < 0)
+        TEST_ERROR;
+
+    /* Create file dataspace. */
+    if ((fspace_id = H5Screate_simple(1, dims, NULL)) < 0)
+        TEST_ERROR;
+
+    /* Create dataset. */
+    if ((dset_id = H5Dcreate2(file_id, "test_dset", H5T_NATIVE_INT, fspace_id, H5P_DEFAULT, H5P_DEFAULT,
+                              H5P_DEFAULT)) < 0)
+        TEST_ERROR;
+
+    /* Write the attribute (compound) to the dataset */
+    if ((att_dspc_id = H5Screate_simple(1, dima, NULL)) < 0)
+        TEST_ERROR;
+    if ((att_attr_id =
+             H5Acreate2(dset_id, "ATTRIBUTES", att_dtyp_id, att_dspc_id, H5P_DEFAULT, H5P_DEFAULT)) < 0)
+        TEST_ERROR;
+    if (H5Awrite(att_attr_id, att_dtyp_id, atts) < 0)
+        TEST_ERROR;
+
+    /* Create dataset transfer property list */
+    const char *expr = "2*x";
+    if ((dxpl_id = H5Pcreate(H5P_DATASET_XFER)) < 0)
+        TEST_ERROR;
+    if (H5Pset_data_transform(dxpl_id, expr) < 0) {
+        HDprintf("**** ERROR: H5Pset_data_transform (expression: %s) ****\n", expr);
+        TEST_ERROR;
+    }
+
+    int *data     = HDmalloc(LENGTH * sizeof(int));
+    int *data_res = HDmalloc(LENGTH * sizeof(int));
+    for (unsigned i = 0; i < LENGTH; i++) {
+        data[i]     = 10;
+        data_res[i] = 2 * data[i];
+    }
+
+    /* Write the data */
+    if (H5Dwrite(dset_id, H5T_NATIVE_INT, H5S_ALL, H5S_ALL, dxpl_id, data) < 0)
+        TEST_ERROR;
+
+    /* Read attribute */
+    att_t *atts_res = HDmalloc(sizeof(att_t));
+    if (H5Aread(att_attr_id, att_dtyp_id, atts_res) < 0)
+        TEST_ERROR;
+
+    /* Verify attribute */
+    if (HDstrcmp(atts_res[0].name, atts[0].name) != 0)
+        TEST_ERROR;
+    if (HDstrcmp(atts_res[0].unit, atts[0].unit) != 0)
+        TEST_ERROR;
+
+    /* Read the data */
+    if (H5Dread(dset_id, H5T_NATIVE_INT, H5S_ALL, H5S_ALL, H5P_DEFAULT, data) < 0)
+        TEST_ERROR;
+
+    /* Verify data */
+    for (unsigned idx = 0; idx < LENGTH; idx++) {
+        if (data[idx] != data_res[idx])
+            TEST_ERROR;
+    }
+
+    HDfree(atts);
+    HDfree(atts_res);
+    HDfree(data);
+    HDfree(data_res);
+
+    /* Close all identifiers. */
+    H5Pclose(dxpl_id);
+    H5Aclose(att_attr_id);
+    H5Sclose(att_dspc_id);
+    H5Dclose(dset_id);
+    H5Sclose(fspace_id);
+    H5Fclose(file_id);
+    H5Tclose(att_dtyp_id);
+    H5Tclose(str_dtyp_id);
+
+    return 0;
+
+error:
+    return 1;
+}
diff --git a/test/dtransform.c b/test/dtransform.c
@@ -581,6 +581,7 @@ test_specials(hid_t file)
     const char *special3 = "1000/x";
     const char *special4 = "-x";
     const char *special5 = "+x";
+    const char *special6 = "2e+1*x";
 
     TESTING("data transform of some special cases")
 
@@ -702,6 +703,29 @@ test_specials(hid_t file)
 
     COMPARE_INT(read_buf, data_res)
 
+    if (H5Dclose(dset_id) < 0)
+        TEST_ERROR
+
+    /*-----------------------------
+     * Operation 6: 2e+1*x
+     *----------------------------*/
+    if (H5Pset_data_transform(dxpl_id, special6) < 0)
+        TEST_ERROR;
+
+    for (row = 0; row < ROWS; row++)
+        for (col = 0; col < COLS; col++)
+            data_res[row][col] = transformData[row][col] * 20;
+
+    if ((dset_id = H5Dcreate2(file, "/special6", H5T_NATIVE_INT, dataspace, H5P_DEFAULT, H5P_DEFAULT,
+                              H5P_DEFAULT)) < 0)
+        TEST_ERROR
+    if (H5Dwrite(dset_id, H5T_NATIVE_INT, H5S_ALL, H5S_ALL, dxpl_id, transformData) < 0)
+        TEST_ERROR
+    if (H5Dread(dset_id, H5T_NATIVE_INT, H5S_ALL, H5S_ALL, H5P_DEFAULT, read_buf) < 0)
+        TEST_ERROR
+
+    COMPARE_INT(read_buf, data_res)
+
     if (H5Dclose(dset_id) < 0)
         TEST_ERROR