Update compact dataset I/O routines to handle driver-level memory copy #1054
Conversation
jhendersonHDF
requested review from
derobins,
soumagne,
brtnfld,
fortnern and
jrmainzer
jhendersonHDF
requested review from
qkoziol and
vchoi-hdfgroup
as code owners
src/H5Dcompact.c
Outdated
| FUNC_ENTER_STATIC | ||
|
|
||
| /* Retrieve lower-level file handle for I/O */ | ||
| if (H5F_shared_get_file_handle(udata->f_sh, &file_handle) < 0) |
There was a problem hiding this comment.
Here, we need a handle to the shared file structure's H5FD_t pointer. However, the H5F package hides this behind an incomplete typedef, so we can't just directly access it through udata->f_sh->lf. We have to create a routine in H5Fquery.c to give this back to us.
| @@ -185,6 +193,9 @@ | |||
| #define H5FD_CTL__GET_MPI_COMMUNICATOR_OPCODE 2 | |||
| #define H5FD_CTL__GET_MPI_RANK_OPCODE 3 | |||
| #define H5FD_CTL__GET_MPI_SIZE_OPCODE 4 | |||
| #define H5FD_CTL__MEM_ALLOC 5 | |||
| #define H5FD_CTL__MEM_FREE 6 | |||
| #define H5FD_CTL__MEM_COPY 7 | |||
There was a problem hiding this comment.
Here, I only reserved the op codes for driver-level memory allocations and frees. I haven't added any argument structs for them since there isn't a use case yet, but can do so if it's useful.
There was a problem hiding this comment.
Agree -- wait until we have use cases.
| /* Private headers needed by this file */ | ||
| #include "H5MMprivate.h" /* Memory management */ | ||
| #include "H5FDprivate.h" /* File drivers */ |
There was a problem hiding this comment.
Since H5F_shared_get_file_handles needs the definition for the H5FD_t type, and since H5FDprivate.h includes H5FDpublic.h, just include the private header rather than both here.
jhendersonHDF
force-pushed
the
gpu_vfd_updates
branch
2 times, most recently
from
c56b30f
to
1b2fc54
Compare
jhendersonHDF
force-pushed
the
gpu_vfd_updates
branch
from
1b2fc54
to
00dfa62
Compare
Rename CTL memory copy flag and H5Fquery routine to get file driver structure
jhendersonHDF
force-pushed
the
gpu_vfd_updates
branch
from
f1e268d
to
626d651
Compare
|
Should I assume all these changes to develop have had someone add release.txt entries? |
It looks like the VFD plugins PR didn't have a RELEASE.txt entry associated with it. Let me create that in a separate PR and then add a RELEASE.txt entry for this PR here. |
jhendersonHDF
requested review from
bmribler,
byrnHDF and
lrknox
as code owners
These changes are needed to be compatible with the GPU VFD. Since the VFD uses buffers that are allocated via cudaMalloc, which aren't compatible with memcpy from the standard library, HDF5 needs to be able to request that a driver handle a memory copy if necessary.