Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix problems with background buffers and array datatypes #4218

Merged
merged 12 commits into from
Apr 1, 2024
Merged

Fix problems with background buffers and array datatypes #4218

merged 12 commits into from
Apr 1, 2024

Conversation

fortnern
Copy link
Member

@fortnern fortnern commented Mar 22, 2024 •
edited
Loading

This patch addresses two problems with array datatype conversions:

  1. The conversion code did not advance in the background buffer when converting each constituent element. Addressed this.
  2. The conversion code always reported, in the INIT phase, that it did not require a background buffer, even if the parent did require one. While in the CONV phase the converison routine would initialize the parent conversion path and allocate a background buffer if necessary, it did not fill the background buffer with the destination contents. Changed this code to initialize the parent conversion in the INIT phase, report the background buffer necessity value from that path, and save a pointer to the conversion path for use in the CONV phase.

No testing yet, will add Monday

@fortnern fortnern marked this pull request as draft March 22, 2024 22:51
@fortnern fortnern mentioned this pull request Mar 22, 2024
Closed
@fortnern
Copy link
Member Author

I'm able to reproduce the h5dumpgentest failure and I'll look into it Monday

@fortnern fortnern marked this pull request as ready for review March 29, 2024 16:32
@fortnern fortnern added the Merge - To 1.14 This needs to be merged to HDF5 1.14 label Mar 29, 2024
@fortnern fortnern added this to the 1.14.4 milestone Mar 29, 2024
@lrknox lrknox merged commit 589f523 into HDFGroup:develop Apr 1, 2024
54 checks passed
lrknox pushed a commit to lrknox/hdf5 that referenced this pull request Apr 1, 2024
@fortnern @lrknox
Fix problems with background buffers and array datatypes (HDFGroup#4218)
209162a 
* Fix bug in array conversion with strided background buffer. Convert some
memmove calls to non-overlapping buffers to memcpy.

* Revert inappropriate use of mempy to memmove in H5T__conv_array

* Add testing

* Add RELEASE.txt note and overwrite test case.
derobins added a commit that referenced this pull request Apr 1, 2024
@lrknox @derobins
@hyoklee @brtnfld @byrnHDF @jhendersonHDF @bmribler @fortnern
Sync develop changes March 28 - April 1 to 1 14 (#4297)
9f7b987 
* Relaxed behavior of H5Pset_page_buffer_size() when opening files (#4280)

This API call sets the size of a file's page buffer cache. This call
was extremely strict about matching its parameters to the file strategy
and page size used to create the file, requiring a separate open of the
file to obtain these parameters.

These requirements have been relaxed when using the fapl to open
a previously-created file:

* When opening a file that does not use the H5F_FSPACE_STRATEGY_PAGE
  strategy, the setting is ignored and the file will be opened, but
  without a page buffer cache. This was previously an error.

* When opening a file that has a page size larger than the desired
  page buffer cache size, the page buffer cache size will be increased
  to the file's page size. This was previously an error.

The behavior when creating a file using H5Pset_page_buffer_size() is
unchanged.

Fixes GitHub issue #3382

* chore: improve error message (#4287)

close #192

* changed to if string contains instead (#4286)

* Fix off_t straggers (#4291)

Convert off_t to HDoff_t (mainly for Windows):

* h5jam
* h5unjam
* chunk_info test

* Add missing foreach for VFD and Pasthrough runs (#4292)

* Fixes for file format security issues (#4283)

* Add configure options for disabling extension features (#4277)

Add configure option to enable or disable extension features in general

Add configure option to enable or disable _Float16 support

Add new config options to various settings files

* Add documentation (H5FD) (#4269)

* Use AOCC 4.2 and OpenMPI 4.1.6 (#4290)

* Fix problems with background buffers and array datatypes (#4218)

* Fix bug in array conversion with strided background buffer. Convert some
memmove calls to non-overlapping buffers to memcpy.

* Revert inappropriate use of mempy to memmove in H5T__conv_array

* Add testing

* Add RELEASE.txt note and overwrite test case.

* Synch workflows and require apt-get update (#4294)

* Fix Figure 9. table format in HDF5 Groups User Guide (#4295)

---------

Co-authored-by: Dana Robinson <43805+derobins@users.noreply.github.com>
Co-authored-by: H. Joe Lee <hyoklee@hdfgroup.org>
Co-authored-by: Scot Breitenfeld <brtnfld@hdfgroup.org>
Co-authored-by: Allen Byrne <50328838+byrnHDF@users.noreply.github.com>
Co-authored-by: jhendersonHDF <jhenderson@hdfgroup.org>
Co-authored-by: bmribler <39579120+bmribler@users.noreply.github.com>
Co-authored-by: Neil Fortner <fortnern@gmail.com>
qkoziol pushed a commit to qkoziol/hdf5 that referenced this pull request Apr 2, 2024
@fortnern @qkoziol
Fix problems with background buffers and array datatypes (HDFGroup#4218)
13cd597 
* Fix bug in array conversion with strided background buffer. Convert some
memmove calls to non-overlapping buffers to memcpy.

* Revert inappropriate use of mempy to memmove in H5T__conv_array

* Add testing

* Add RELEASE.txt note and overwrite test case.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@qkoziol qkoziol qkoziol approved these changes

@jhendersonHDF jhendersonHDF jhendersonHDF approved these changes

@lrknox lrknox Awaiting requested review from lrknox lrknox is a code owner

@derobins derobins Awaiting requested review from derobins derobins is a code owner

@byrnHDF byrnHDF Awaiting requested review from byrnHDF byrnHDF is a code owner

@vchoi-hdfgroup vchoi-hdfgroup Awaiting requested review from vchoi-hdfgroup vchoi-hdfgroup is a code owner

@bmribler bmribler Awaiting requested review from bmribler bmribler is a code owner

@glennsong09 glennsong09 Awaiting requested review from glennsong09 glennsong09 is a code owner

@mattjala mattjala Awaiting requested review from mattjala mattjala is a code owner

@brtnfld brtnfld Awaiting requested review from brtnfld brtnfld is a code owner

Assignees
No one assigned
Labels
Merge - To 1.14 This needs to be merged to HDF5 1.14
Projects
Downstream HDF5 Merges
Status: Needs Merged
Milestone
1.14.4
Development

Successfully merging this pull request may close these issues.
4 participants
@fortnern @qkoziol @jhendersonHDF @lrknox