Selection io with subfiling vfd #1217
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* OESS-98 convert plugin option to FetchContent, add tests * Fixes for pkcfg files because of plugin option * OESS-98 fix tools test for plugins * Keep doxygen comments under 100 chars long - format hint * Whitespace * HDFFV-11144 - Reclassify CMake messages * HDFFV-11099/11100 added help text * Reworked switch statement to compare string instead * Fix typo * Update CDash mode * Correct name of threadsafe * Correct option name * Undo accidental commit * Note LLVM 10 to 11 format default changes * Update format plugin * Undo clang-format version 11 changes * One more correction * Update supported platforms * Revert whitespace changes * Correct whitespace * Changes from PR#3 * HDFFV-11213 added option to control gcc10 warnings diagnostics * HDFFV-11212 Use the new references correctly in JNI utility and tests * format source * Fix typo * Add new test file * HDFFV-11212 - update test and remove unused arg * Minor non-space formatting changes * Use H5I_INVALID_ID instead of "-1" * source formatting * add missing testfile, update jni function * Undo commit of debug code * remove mislocated file * Fix h5repack test for handling of fapls and id close * Update h5diff test files usage text * HDFFV-11212 add new ref tests for JNI export dataset * src format update * Remove blank line typo * src format typo * long double requires %Lg * Another long double foramt specifer S.B. %Lg * issue with t128bit test * Windows issue with h5dump and type. * Fix review issues * refactor function nesting and fix error checks * format fixes * Remove untested functions and javadoc quiet comments * Restore TRY block. * Change string append errors to memory exception * revert to H5_JNI_FATAL_ERROR - support functions need work * Add assertion error for h5util functions * remove duplicate function * format fix * Revert HD function error handling * Update copyright comments * GH #386 java folder copyright corrections * Whitespace * GH #359 implement and fix tools 1.6 API usage * remove excessive comments * Flip inits to correct ifdef section * rework ifdef to be simpler * format issue * Reformat ifdef inits * remove static attribute * format compliance * Update names * Revert because logic relies on float not being int
* OESS-98 convert plugin option to FetchContent, add tests * Fixes for pkcfg files because of plugin option * OESS-98 fix tools test for plugins * Keep doxygen comments under 100 chars long - format hint * Whitespace * HDFFV-11144 - Reclassify CMake messages * HDFFV-11099/11100 added help text * Reworked switch statement to compare string instead * Fix typo * Update CDash mode * Correct name of threadsafe * Correct option name * Undo accidental commit * Note LLVM 10 to 11 format default changes * Update format plugin * Undo clang-format version 11 changes * One more correction * Update supported platforms * Revert whitespace changes * Correct whitespace * Changes from PR#3 * HDFFV-11213 added option to control gcc10 warnings diagnostics * HDFFV-11212 Use the new references correctly in JNI utility and tests * format source * Fix typo * Add new test file * HDFFV-11212 - update test and remove unused arg * Minor non-space formatting changes * Use H5I_INVALID_ID instead of "-1" * source formatting * add missing testfile, update jni function * Undo commit of debug code * remove mislocated file * Fix h5repack test for handling of fapls and id close * Update h5diff test files usage text * HDFFV-11212 add new ref tests for JNI export dataset * src format update * Remove blank line typo * src format typo * long double requires %Lg * Another long double foramt specifer S.B. %Lg * issue with t128bit test * Windows issue with h5dump and type. * Fix review issues * refactor function nesting and fix error checks * format fixes * Remove untested functions and javadoc quiet comments * Restore TRY block. * Change string append errors to memory exception * revert to H5_JNI_FATAL_ERROR - support functions need work * Add assertion error for h5util functions * remove duplicate function * format fix * Revert HD function error handling * Update copyright comments * GH #386 java folder copyright corrections * Whitespace * GH #359 implement and fix tools 1.6 API usage * remove excessive comments * Flip inits to correct ifdef section * rework ifdef to be simpler * format issue * Reformat ifdef inits * remove static attribute * format compliance * Update names * Revert because logic relies on float not being int * Changes noticed from creating merge of #412 * Double underscore change * Correct compiler version variable used * Remove header guard underscores * Whitespace cleanup
) * Use do-while trick to force H5Epush_ret() to require trailing semi Fixed ensuing compiler errors. * Commit format changes from running clang-format with clang version 10.0.1. * Use do-while trick to force H5Epush_ret() to require trailing semi Fixed ensuing compiler errors. Updated RELEASE.txt. This change was made to allow clang-format to correctly format the code. Co-authored-by: Larry Knox <lrknox@hdfgroup.org>
* Fixed -Wunused-local-typedef warning * Fixed -Wformat warnings In one case also removed a `z` character. There was a `z%d` that I think was supposed to be `%zd` * Fixed -Wshorten-64-to-32 warnings * Fixed -Wself-assign warnings * Fixed -Wreserved-id-macro warnings * Commit format changes from clang-format, clang version 10.0.1. * Fixed -Wself-assign warnings * Fixed -Wunused-local-typedef warning * Fixed -Wformat warnings In two cases also removed a `z` character. There was a `z%d` that was supposed to be `%zd`. * Fixed -Wshorten-64-to-32 warnings * Fixed -Wreserved-id-macro warnings * Fixed -Wself-assign warnings * Format source. * Remove blank lines to pass format check. Co-authored-by: Larry Knox <lrknox@hdfgroup.org>
* Fixes various warnings noticed on Windows - Adds a prototype for our implementation of vasprintf - Return type of H5_get_utf16_str() is now non-const - Fixes possible uninitialized return type in Wremove_utf8 - Better isolation of fork() code in accum.c:test_swmr_write_big() - Better isolation of non-zlib code in dsets.c:test_filter_delete() - Removed unused variable in trefer.c:test_reference_cmpnd_obj() * Fixes clang-format issues
* Fixed clang-tidy readability-misleading-indentation warnings * Reformatted src/H5Zscaleoffset.c with clang v10.0.1. Co-authored-by: Larry Knox <lrknox@hdfgroup.org>
* Fixed clang-tidy readability-redundant-control-flow warnings Just removed useless trailing return statements. * Removed blank lines from h5diffgentest.c with clang-format v10.0.1. Co-authored-by: Larry Knox <lrknox@hdfgroup.org>
* Removed bad function pointer casts In one case fixed the actual function signature to be correct. * Reformat source with clang v10.0.1. Co-authored-by: Larry Knox <lrknox@hdfgroup.org>
#429) * Automatically applied clang-tidy readability-avoid-const-params-in-decls fixes Removes useless const declarations. * Fixed most readability-non-const-parameter warnings These changes were made automatically by clang-tidy, but I manually reverted the changes related to the H5Z_func_t signature. * Reformat source with clang v10.0.1. Co-authored-by: Larry Knox <lrknox@hdfgroup.org>
After 30+ years, just assume that the following exist: - extension-less includes - namespaces - std:: - static_cast - bool
Added H5_OVERRIDE macro for compatibility with both C++11 and older.
* Fixed clang-tidy bugprone-reserved-identifier warnings * Fixed clang-tidy bugprone-assert-side-effect warnings * Fixed clang-tidy bugprone-copy-constructor-init warning * Fixed clang-tidy readability-redundant-preprocessor warning For error_test.c the removed code was already dead, because it was in the else of an `#if H5_USE_16_API` block. Based on H5Location.h, I think p_get_ref_obj_type was meant to be in `#ifndef DOXYGEN_SHOULD_SKIP_THIS` and an `#endif` was missing. Similarly, in the header, getObjTypeByIdx is only in H5_NO_DEPRECATED_SYMBOLS, not DOXYGEN_SHOULD_SKIP_THIS. * Fixed clang-tidy readability-redundant-string-init warnings * Fixed some clang-tidy performance-type-promotion-in-math-fn warnings * Fixed clang-tidy performance-unnecessary-value-param warnings * Reformat source with clang v10.0.1. Co-authored-by: Larry Knox <lrknox@hdfgroup.org>
* Fixed all clang-tidy bugprone-suspicious-string-compare warnings This change was generated entirely by clang-tidy itself. * Reformat code with clang v10.0.1. Co-authored-by: Larry Knox <lrknox@hdfgroup.org>
* OESS-98 convert plugin option to FetchContent, add tests * Fixes for pkcfg files because of plugin option * OESS-98 fix tools test for plugins * Keep doxygen comments under 100 chars long - format hint * Whitespace * HDFFV-11144 - Reclassify CMake messages * HDFFV-11099/11100 added help text * Reworked switch statement to compare string instead * Fix typo * Update CDash mode * Correct name of threadsafe * Correct option name * Undo accidental commit * Note LLVM 10 to 11 format default changes * Update format plugin * Undo clang-format version 11 changes * One more correction * Update supported platforms * Revert whitespace changes * Correct whitespace * Changes from PR#3 * HDFFV-11213 added option to control gcc10 warnings diagnostics * HDFFV-11212 Use the new references correctly in JNI utility and tests * format source * Fix typo * Add new test file * HDFFV-11212 - update test and remove unused arg * Minor non-space formatting changes * Use H5I_INVALID_ID instead of "-1" * source formatting * add missing testfile, update jni function * Undo commit of debug code * remove mislocated file * Fix h5repack test for handling of fapls and id close * Update h5diff test files usage text * HDFFV-11212 add new ref tests for JNI export dataset * src format update * Remove blank line typo * src format typo * long double requires %Lg * Another long double foramt specifer S.B. %Lg * issue with t128bit test * Windows issue with h5dump and type. * Fix review issues * refactor function nesting and fix error checks * format fixes * Remove untested functions and javadoc quiet comments * Restore TRY block. * Change string append errors to memory exception * revert to H5_JNI_FATAL_ERROR - support functions need work * Add assertion error for h5util functions * remove duplicate function * format fix * Revert HD function error handling * Update copyright comments * GH #386 java folder copyright corrections * Whitespace * GH #359 implement and fix tools 1.6 API usage * remove excessive comments * Flip inits to correct ifdef section * rework ifdef to be simpler * format issue * Reformat ifdef inits * remove static attribute * format compliance * Update names * Revert because logic relies on float not being int * Changes noticed from creating merge of #412 * Double underscore change * Correct compiler version variable used * Remove header guard underscores * Whitespace cleanup * Split format source and commit changes on repo push * remove pre-split setting * Change windows TS to use older VS. * correct window os name
* Replaces checks for fork, etc. with H5_HAVE_UNISTD_H Code previously checked for individual POSIX API calls using H5_HAVE_FORK, etc. The calls we use have been standardized for decades and available via unistd.h. Some test messages that were missing when tests are skipped due to a lack of unistd.h were also added. The configure checks for individual POSIX API calls will be removed in a later commit. * Stupid formatter
* OESS-98 convert plugin option to FetchContent, add tests * Fixes for pkcfg files because of plugin option * OESS-98 fix tools test for plugins * Keep doxygen comments under 100 chars long - format hint * Whitespace * HDFFV-11144 - Reclassify CMake messages * HDFFV-11099/11100 added help text * Reworked switch statement to compare string instead * Fix typo * Update CDash mode * Correct name of threadsafe * Correct option name * Undo accidental commit * Note LLVM 10 to 11 format default changes * Update format plugin * Undo clang-format version 11 changes * One more correction * Update supported platforms * Revert whitespace changes * Correct whitespace * Changes from PR#3 * HDFFV-11213 added option to control gcc10 warnings diagnostics * HDFFV-11212 Use the new references correctly in JNI utility and tests * format source * Fix typo * Add new test file * HDFFV-11212 - update test and remove unused arg * Minor non-space formatting changes * Use H5I_INVALID_ID instead of "-1" * source formatting * add missing testfile, update jni function * Undo commit of debug code * remove mislocated file * Fix h5repack test for handling of fapls and id close * Update h5diff test files usage text * HDFFV-11212 add new ref tests for JNI export dataset * src format update * Remove blank line typo * src format typo * long double requires %Lg * Another long double foramt specifer S.B. %Lg * issue with t128bit test * Windows issue with h5dump and type. * Fix review issues * refactor function nesting and fix error checks * format fixes * Remove untested functions and javadoc quiet comments * Restore TRY block. * Change string append errors to memory exception * revert to H5_JNI_FATAL_ERROR - support functions need work * Add assertion error for h5util functions * remove duplicate function * format fix * Revert HD function error handling * Update copyright comments * GH #386 java folder copyright corrections * Whitespace * GH #359 implement and fix tools 1.6 API usage * remove excessive comments * Flip inits to correct ifdef section * rework ifdef to be simpler * format issue * Reformat ifdef inits * remove static attribute * format compliance * Update names * Revert because logic relies on float not being int * Changes noticed from creating merge of #412 * Double underscore change * Correct compiler version variable used * Remove header guard underscores * Whitespace cleanup * Split format source and commit changes on repo push * remove pre-split setting * Change windows TS to use older VS. * correct window os name * HDFFV-11212 JNI export util and Javadoc * Suggested review changes * Another change found * Committing clang-format changes Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
Fixes -Wreserved-identifier warnings due to multiple underscores in H5_HAVE___INLINE.
* Stop using global variables to share parse context with the lexer. The lexer uses an unconventional strategy for parsing lexical categories NUMBER (decimal numbers) and STRING (double-quoted strings) that involves sharing the parse context with the lexer using global variables. There are a couple of problems with that. First, the lexer is too complicated for the simple tokenization it performs—it's hard to tell if it is correct. Second, as @seanm points out, the shared global variables spill into the namespace shared by other libraries and application programs—e.g., VTK. * Regenerate source files from *.[yl]. * Replace strndup, which isn't available on Windows, with a custom routine, `trim_quotes`, that produces a copy of its `const char *` argument with leading and trailing double quotes ('"') removed. While I am here, remove the unnecessary statement `BEGIN INITIAL;`, which I should have deleted in a previous commit. * Regenerate .c from .l. * You haven't programmed in C until you have programmed in High-Definition (HD) C. * \#include "H5private.h" for HD* definitions. * Regenerate *.[ch] from *.[yl].
* Update clang config to put H5E_BEGIN_TRY / H5E_END_TRY on separate lines, empty C++ methods on separate lines, understand that ALL_MEMBERS / UNIQUE_MEMBERS are foreach macros, and properly skip the 'config' directory in the find command without emiting a warning * Committing clang-format changes Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
a Fortran compile -> a Fortran compiler
* OESS-98 convert plugin option to FetchContent, add tests * Fixes for pkcfg files because of plugin option * OESS-98 fix tools test for plugins * Keep doxygen comments under 100 chars long - format hint * Whitespace * HDFFV-11144 - Reclassify CMake messages * HDFFV-11099/11100 added help text * Reworked switch statement to compare string instead * Fix typo * Update CDash mode * Correct name of threadsafe * Correct option name * Undo accidental commit * Note LLVM 10 to 11 format default changes * Update format plugin * Undo clang-format version 11 changes * One more correction * Update supported platforms * Revert whitespace changes * Correct whitespace * Changes from PR#3 * HDFFV-11213 added option to control gcc10 warnings diagnostics * HDFFV-11212 Use the new references correctly in JNI utility and tests * format source * Fix typo * Add new test file * HDFFV-11212 - update test and remove unused arg * Minor non-space formatting changes * Use H5I_INVALID_ID instead of "-1" * source formatting * add missing testfile, update jni function * Undo commit of debug code * remove mislocated file * Fix h5repack test for handling of fapls and id close * Update h5diff test files usage text * HDFFV-11212 add new ref tests for JNI export dataset * src format update * Remove blank line typo * src format typo * long double requires %Lg * Another long double foramt specifer S.B. %Lg * issue with t128bit test * Windows issue with h5dump and type. * Fix review issues * refactor function nesting and fix error checks * format fixes * Remove untested functions and javadoc quiet comments * Restore TRY block. * Change string append errors to memory exception * revert to H5_JNI_FATAL_ERROR - support functions need work * Add assertion error for h5util functions * remove duplicate function * format fix * Revert HD function error handling * Update copyright comments * GH #386 java folder copyright corrections * Whitespace * GH #359 implement and fix tools 1.6 API usage * remove excessive comments * Flip inits to correct ifdef section * rework ifdef to be simpler * format issue * Reformat ifdef inits * remove static attribute * format compliance * Update names * Revert because logic relies on float not being int * Changes noticed from creating merge of #412 * Double underscore change * Correct compiler version variable used * Remove header guard underscores * Whitespace cleanup * Split format source and commit changes on repo push * remove pre-split setting * Change windows TS to use older VS. * correct window os name * HDFFV-11212 JNI export util and Javadoc * Suggested review changes * Another change found * Committing clang-format changes * HDFFV-11113 long double in tools * HDFFV-11113 add note Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
* Fixes a few testhdf5 warnings raised in Visual Studio Visual Studio is grumpier about treating pointers like integers than gcc. * Committing clang-format changes Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
* Suppresses the tcheck_version test's abort dialog on Windows Windows raises a modal abort/retry/ignore dialog box when CRT calls abort(). This change installs a report hook that suppresses the dialog so that the CMake tests don't time out waiting for a nonexistent user to click a dialog box. * Committing clang-format changes * Removes __cdecl from callback Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
…ribution policy. (#445) * Added description of the current HDF5 branches. * Removed capitalization in from Develop, Release, Feature to reflect the real naming schema * Added a draft of contributions guidance document. * Fixed typos. * Fixed section title. * Fixed typo. * Fixed typos and formatting. * Fixed many typos and simplified the text (e.g., removed testing instructions and left pointers to where to find them, etc.) * Fixed a typo. Please enter the commit message for your changes. Lines starting * Added contributing.md file and rearranged doc entries in alphabetical order. * ddressed Gerd's review comments; found and fixed more typos. * Addressed comments from Larry and Scot.
…405) * Fixed HDFFV-10480 (CVE-2018-11206) and HDFFV-11159 (CVE-2018-14033) Description Checked against buffer size to prevent segfault, in case of data corruption. + HDFFV-11159 CVE-2018-14033 Buffer over-read in H5O_layout_decode + HDFFV-10480 CVE-2018-11206 Buffer over-read in H5O_fill_new[/old]_decode Platforms tested: Linux/64 (jelly) * Accidentally left in another occurrence of the previous patch from user after a more correct fix was applied, that is the check now accounted for the previous advance of the buffer pointer. Removed it. * Typo * Fixed format issues. * Added test. * Changed arguments to ADD_H5_TEST * Fixing arguments to ADD_H5_TEST again. * Fixing arguments again. * Took out the CMake changes until Allen can help. * Added files: tCVE_2018_11206_fill_old.h5 tCVE_2018_11206_fill_new.h5 * Revert "Took out the CMake changes until Allen can help." This reverts commit c21324d. * Revert "Fixing arguments again." This reverts commit 5832a70. * Revert "Fixing arguments to ADD_H5_TEST again." This reverts commit b45de82. * Revert "Changed arguments to ADD_H5_TEST" This reverts commit 1671982. * Added first argument to ADD_H5_TEST for HDFFV-10480 fix. * Changed argument 0 to 1 * Revert "Changed argument 0 to 1" This reverts commit b343d66. * Revert "Added first argument to ADD_H5_TEST for HDFFV-10480 fix." This reverts commit b8a0f9a. * Added first argument and corrected the second. * Updated fixes for HDFFV-10480 and HDFFV-11159/HDFFV-11049 * Improved error messages.
Some of the uniniitialized bits in the buffer may get carried through all the way to disk, creating a risk for leaks. We observed an msan error during the floating point output conversion. Due to the encoding certain bits could remain untouched during the conversion. In this draft we zero initialize the dbuf used by every convertor.
Print error message if Perl is not found.
* Adds an implementation for nanosleep on Windows * Committing clang-format changes * Fix insane formatter issue Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
* Fix ASAN issue in h5dump error path * Rework error allocation free.
…ompiler flags files (#1033) * Avoid maintenance headaches: delete the `noerror-` variants of the compiler flags files, since they essentially duplicate the `error-` files modulo the replacement of `-Werror=` with `-W` and any changes in comments. (I verified the duplication with a script.) For autoconf, reinstate the use of the `demote_errors` shell function to derive the `noerror-` content from the `error-` content. `demote_errors` replaces `-Werror=` with `-W` when `WARNINGS_AS_ERRORS` is `no`. Slightly reorder `configure.ac` so that the setting of `WARNINGS_AS_ERRORS` takes effect before the `error-` files are sourced. * Take a stab at updating the CMake files to match the changes I made to the autoconf files to remove `noerror-` files. I'm not much of a CMake user so these changes are quite rough. Looks like the duplication can be reduced with the introduction of a new macro. * Delete `noerror-` files from the MANIFEST. * Reduce duplication in the CMake files: perform the HDF5_ENABLE_WARNINGS_AS_ERRORS test once in the ADD_H5_FLAGS macro. * Add a release note.
* OESS-168: Remove clang warnings. * Address review by @jehndersonHDF.
* OESS-168: Remove clang warnings. * Committing clang-format changes Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
* Add missing dataset reference text * Only print name if not printing data
* OESS-168: Remove clang warnings. * Committing clang-format changes * Address review by @jehndersonHDF. * Address review by @jehndersonHDF. * OESS-168: Fix CI failure. Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
…with_subfiling_vfd and resolved conflicts. Note that H5S_get_validated_dataspace() in H5S.c was removed. Re-inserted this function for now. Update testpar/t_subfiling_vfd.c to set HD_enable_selection_io_g. Uses MPI flag on the sub-filing VFD is still not set, as the VFD seg faults if it is. Tested parallel / debug on Charis and Jelly.
jrmainzer
requested review from
bljhdf,
bmribler,
brtnfld,
byrnHDF,
derobins,
epourmal,
fortnern,
gnuoyd,
jhendersonHDF,
lrknox,
qkoziol,
rawarren,
soumagne and
vchoi-hdfgroup
as code owners
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.