Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add new API function H5Pget_actual_select_io_mode() #2974

Merged
merged 44 commits into from
Oct 20, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
0ba0f1c
Add H5Pget_actual_selection_io_mode() API.
vchoi-hdfgroup May 2, 2023
4c8570d
Committing clang-format changes
github-actions[bot] May 2, 2023
98bcb50
Revert back to the original.
vchoi-hdfgroup May 2, 2023
d1ae893
Changes made based on PR review comments.
vchoi-hdfgroup May 14, 2023
bf23267
Committing clang-format changes
github-actions[bot] May 14, 2023
0f32c8c
Changes made based on review comments.
vchoi-hdfgroup May 16, 2023
4dd0901
Committing clang-format changes
github-actions[bot] May 16, 2023
2f352f4
Changes to doxygen for H5Pget_actual_selection_io().
vchoi-hdfgroup May 17, 2023
e18e1c3
Committing clang-format changes
github-actions[bot] May 17, 2023
cea5cf1
Modify H5Pget_actual_selection_io_mode to only count raw data
fortnern Jun 9, 2023
a99afc0
Committing clang-format changes
github-actions[bot] Jun 9, 2023
fc0fe26
Fix spelling
fortnern Jun 9, 2023
6c56c25
Merge branch 'actual_select_io_mode' of github.com:fortnern/hdf5 into…
fortnern Jun 9, 2023
7329597
Merge branch 'develop' into actual_select_io_mode
fortnern Sep 12, 2023
fc83b1e
Committing clang-format changes
github-actions[bot] Sep 12, 2023
648991e
Fix compile failures from latest merge
fortnern Sep 12, 2023
726d866
Merge branch 'actual_select_io_mode' of github.com:fortnern/hdf5 into…
fortnern Sep 12, 2023
e0af25d
Address minor review comments.
fortnern Sep 12, 2023
36e49e5
Minor spelling/ scope changes
fortnern Sep 12, 2023
2ec2d48
Improve documentation for H5Pget_actual_selection_io_mode() and remove
fortnern Sep 12, 2023
45f53d4
Add doxygen table entry for H5Pget_actual_selection_io_mode and
fortnern Sep 13, 2023
0ccd1f4
Committing clang-format changes
github-actions[bot] Sep 13, 2023
af62452
Add a line
vchoi-hdfgroup Sep 21, 2023
cf0d2ae
Committing clang-format changes
github-actions[bot] Sep 21, 2023
8092b0f
Remove the line added earlier that shouldn't be there.
vchoi-hdfgroup Sep 21, 2023
78f1a62
Modify H5CX code to always write the actual selection I/O mode to the
fortnern Oct 6, 2023
9570c2b
Merge branch 'actual_select_io_mode' of github.com:fortnern/hdf5 into…
fortnern Oct 6, 2023
a509107
Committing clang-format changes
github-actions[bot] Oct 6, 2023
973c9b2
Merge branch 'develop' into actual_select_io_mode
fortnern Oct 12, 2023
1e7588f
Work-in-progress for tests on H5Pget_actual_selection_io_mode().
vchoi-hdfgroup Oct 11, 2023
09aa92d
Fix test failures, fix function description
fortnern Oct 12, 2023
4518711
Committing clang-format changes
github-actions[bot] Oct 13, 2023
74aa0ca
Add tests to t_select_io_dset and t_filters_parallel, only report I/O
fortnern Oct 19, 2023
2514a4a
Merge branch 'actual_select_io_mode' of github.com:fortnern/hdf5 into…
fortnern Oct 19, 2023
840acf5
Committing clang-format changes
github-actions[bot] Oct 19, 2023
f27ffa5
Add RELEASE.txt entry
fortnern Oct 19, 2023
93efdd7
Remove conflict resoltuion text
fortnern Oct 19, 2023
7660d9e
Run bin/format_source
fortnern Oct 19, 2023
2f93970
Address review comments, add more comments, revert behaviour of scalar
fortnern Oct 20, 2023
dbc4510
Committing clang-format changes
github-actions[bot] Oct 20, 2023
06782e3
spelling
fortnern Oct 20, 2023
1f7dd3e
Merge branch 'develop' into actual_select_io_mode
fortnern Oct 20, 2023
f6e448d
Fix formatting
fortnern Oct 20, 2023
1fcab12
Revert behavior of selection I/O callbacks to report having done
fortnern Oct 20, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions doxygen/examples/tables/propertyLists.dox
Original file line number Diff line number Diff line change
Expand Up @@ -711,6 +711,10 @@ of the library for reading or writing the actual data.</td>
<td>Gets the cause for not performing selection or vector I/O on the last parallel I/O call.</td>
</tr>
<tr>
<td>#H5Pget_actual_selection_io_mode</td>
<td>Gets the type(s) (scalar, vector, selection) of raw data I/O performed on the last I/O call.</td>
</tr>
<tr>
<td>#H5Pset_modify_write_buf/#H5Pget_modify_write_buf</td>
<td>Sets/gets a flag allowing the library to modify the contents of the write buffer.</td>
</tr>
Expand Down
6 changes: 6 additions & 0 deletions release_docs/RELEASE.txt
Original file line number Diff line number Diff line change
Expand Up @@ -228,6 +228,12 @@ New Features

Library:
--------
- Added new API function H5Pget_actual_selection_io_mode()

This function allows the user to determine if the library performed
selection I/O, vector I/O, or scalar (legacy) I/O during the last HDF5
operation performed with the provided DXPL.

- Added support for in-place type conversion in most cases

In-place type conversion allows the library to perform type conversion
Expand Down
101 changes: 98 additions & 3 deletions src/H5CX.c
Original file line number Diff line number Diff line change
Expand Up @@ -299,6 +299,11 @@ typedef struct H5CX_t {
bool no_selection_io_cause_set; /* Whether reason for not performing selection I/O is set */
bool no_selection_io_cause_valid; /* Whether reason for not performing selection I/O is valid */

uint32_t
actual_selection_io_mode; /* Actual selection I/O mode used (H5D_ACTUAL_SELECTION_IO_MODE_NAME) */
hbool_t actual_selection_io_mode_set; /* Whether actual selection I/O mode is set */
hbool_t actual_selection_io_mode_valid; /* Whether actual selection I/O mode is valid */

/* Cached LCPL properties */
H5T_cset_t encoding; /* Link name character encoding */
bool encoding_valid; /* Whether link name character encoding is valid */
Expand Down Expand Up @@ -380,6 +385,8 @@ typedef struct H5CX_dxpl_cache_t {
H5D_selection_io_mode_t selection_io_mode; /* Selection I/O mode (H5D_XFER_SELECTION_IO_MODE_NAME) */
uint32_t no_selection_io_cause; /* Reasons for not performing selection I/O
(H5D_XFER_NO_SELECTION_IO_CAUSE_NAME) */
uint32_t actual_selection_io_mode; /* Actual selection I/O mode
(H5D_XFER_ACTUAL_SELECTION_IO_MODE_NAME) */
bool modify_write_buf; /* Whether the library can modify write buffers */
} H5CX_dxpl_cache_t;

Expand Down Expand Up @@ -571,13 +578,18 @@ H5CX_init(void)

/* Get the selection I/O mode */
if (H5P_get(dx_plist, H5D_XFER_SELECTION_IO_MODE_NAME, &H5CX_def_dxpl_cache.selection_io_mode) < 0)
HGOTO_ERROR(H5E_CONTEXT, H5E_CANTGET, FAIL, "Can't retrieve parallel transfer method");
HGOTO_ERROR(H5E_CONTEXT, H5E_CANTGET, FAIL, "Can't retrieve selection I/O mode");

/* Get the local & global reasons for breaking selection I/O values */
if (H5P_get(dx_plist, H5D_XFER_NO_SELECTION_IO_CAUSE_NAME, &H5CX_def_dxpl_cache.no_selection_io_cause) <
0)
HGOTO_ERROR(H5E_CONTEXT, H5E_CANTGET, FAIL, "Can't retrieve cause for no selection I/O");

/* Get the actual selection I/O mode */
if (H5P_get(dx_plist, H5D_XFER_ACTUAL_SELECTION_IO_MODE_NAME,
&H5CX_def_dxpl_cache.actual_selection_io_mode) < 0)
HGOTO_ERROR(H5E_CONTEXT, H5E_CANTGET, FAIL, "Can't retrieve actual selection I/O mode");

/* Get the modify write buffer property */
if (H5P_get(dx_plist, H5D_XFER_MODIFY_WRITE_BUF_NAME, &H5CX_def_dxpl_cache.modify_write_buf) < 0)
HGOTO_ERROR(H5E_CONTEXT, H5E_CANTGET, FAIL, "Can't retrieve modify write buffer property");
Expand Down Expand Up @@ -2514,6 +2526,47 @@ H5CX_get_no_selection_io_cause(uint32_t *no_selection_io_cause)
FUNC_LEAVE_NOAPI(ret_value)
} /* end H5CX_get_no_selection_io_cause() */

/*-------------------------------------------------------------------------
* Function: H5CX_get_actual_selection_io_mode
*
* Purpose: Retrieves the actual I/O mode (scalar, vector, and/or selection) for the current API call
*context.
*
* Return: Non-negative on success / Negative on failure
*
*-------------------------------------------------------------------------
*/
herr_t
H5CX_get_actual_selection_io_mode(uint32_t *actual_selection_io_mode)
{
H5CX_node_t **head = NULL; /* Pointer to head of API context list */
herr_t ret_value = SUCCEED; /* Return value */

FUNC_ENTER_NOAPI(FAIL)

/* Sanity check */
assert(actual_selection_io_mode);
head = H5CX_get_my_context(); /* Get the pointer to the head of the API context, for this thread */
assert(head && *head);
assert(H5P_DEFAULT != (*head)->ctx.dxpl_id);

/* This property is a special case - we want to wipe out any previous setting. Copy the default setting
* if it has not been set yet. */
if ((*head)->ctx.dxpl_id != H5P_DATASET_XFER_DEFAULT && !(*head)->ctx.actual_selection_io_mode_set &&
!(*head)->ctx.actual_selection_io_mode_valid) {
(*head)->ctx.actual_selection_io_mode = H5CX_def_dxpl_cache.actual_selection_io_mode;
(*head)->ctx.actual_selection_io_mode_set = true;
}
H5CX_RETRIEVE_PROP_VALID_SET(dxpl, H5P_DATASET_XFER_DEFAULT, H5D_XFER_ACTUAL_SELECTION_IO_MODE_NAME,
actual_selection_io_mode)

/* Get the value */
*actual_selection_io_mode = (*head)->ctx.actual_selection_io_mode;

done:
FUNC_LEAVE_NOAPI(ret_value)
} /* end H5CX_get_actual_selection_io_mode() */

/*-------------------------------------------------------------------------
* Function: H5CX_get_modify_write_buf
*
Expand Down Expand Up @@ -3443,7 +3496,7 @@ H5CX_test_set_mpio_coll_rank0_bcast(bool mpio_coll_rank0_bcast)
#endif /* H5_HAVE_PARALLEL */

/*-------------------------------------------------------------------------
* Function: H5CX_set_no_selecction_io_cause
* Function: H5CX_set_no_selection_io_cause
*
* Purpose: Sets the reason for not performing selection I/O for
* the current API call context.
Expand Down Expand Up @@ -3472,7 +3525,39 @@ H5CX_set_no_selection_io_cause(uint32_t no_selection_io_cause)
} /* end if */

FUNC_LEAVE_NOAPI_VOID
} /* end H5CX_set_no_selectiion_io_cause() */
} /* end H5CX_set_no_selection_io_cause() */

/*-------------------------------------------------------------------------
* Function: H5CX_set_actual_selection_io_mode
*
* Purpose: Sets the actual selection I/O mode for the current API
* call context.
*
* Return: <none>
*
*-------------------------------------------------------------------------
*/
void
H5CX_set_actual_selection_io_mode(uint32_t actual_selection_io_mode)
{
H5CX_node_t **head = NULL; /* Pointer to head of API context list */

FUNC_ENTER_NOAPI_NOINIT_NOERR

/* Sanity checks */
head = H5CX_get_my_context(); /* Get the pointer to the head of the API context, for this thread */
assert(head && *head);
assert((*head)->ctx.dxpl_id != H5P_DEFAULT);

/* If we're using the default DXPL, don't modify it */
if ((*head)->ctx.dxpl_id != H5P_DATASET_XFER_DEFAULT) {
/* Cache the value for later, marking it to set in DXPL when context popped */
(*head)->ctx.actual_selection_io_mode = actual_selection_io_mode;
(*head)->ctx.actual_selection_io_mode_set = true;
}

FUNC_LEAVE_NOAPI_VOID
} /* end H5CX_set_actual_selection_io_mode() */

/*-------------------------------------------------------------------------
* Function: H5CX_get_ohdr_flags
Expand Down Expand Up @@ -3529,7 +3614,17 @@ H5CX__pop_common(bool update_dxpl_props)

/* Check for cached DXPL properties to return to application */
if (update_dxpl_props) {
/* actual_selection_io_mode is a special case - we always want to set it in the property list even if
* it was never set by the library, in that case it indicates no I/O was performed and we don't want
* to leave the (possibly incorrect) old value in the property list, so set from the default property
* list */
if ((*head)->ctx.dxpl_id != H5P_DATASET_XFER_DEFAULT && !(*head)->ctx.actual_selection_io_mode_set) {
(*head)->ctx.actual_selection_io_mode = H5CX_def_dxpl_cache.actual_selection_io_mode;
(*head)->ctx.actual_selection_io_mode_set = true;
}

H5CX_SET_PROP(H5D_XFER_NO_SELECTION_IO_CAUSE_NAME, no_selection_io_cause)
H5CX_SET_PROP(H5D_XFER_ACTUAL_SELECTION_IO_MODE_NAME, actual_selection_io_mode)
#ifdef H5_HAVE_PARALLEL
H5CX_SET_PROP(H5D_MPIO_ACTUAL_CHUNK_OPT_MODE_NAME, mpio_actual_chunk_opt)
H5CX_SET_PROP(H5D_MPIO_ACTUAL_IO_MODE_NAME, mpio_actual_io_mode)
Expand Down
2 changes: 2 additions & 0 deletions src/H5CXprivate.h
Original file line number Diff line number Diff line change
Expand Up @@ -116,6 +116,7 @@ H5_DLL herr_t H5CX_get_vlen_alloc_info(H5T_vlen_alloc_info_t *vl_alloc_info);
H5_DLL herr_t H5CX_get_dt_conv_cb(H5T_conv_cb_t *cb_struct);
H5_DLL herr_t H5CX_get_selection_io_mode(H5D_selection_io_mode_t *selection_io_mode);
H5_DLL herr_t H5CX_get_no_selection_io_cause(uint32_t *no_selection_io_cause);
H5_DLL herr_t H5CX_get_actual_selection_io_mode(uint32_t *actual_selection_io_mode);
H5_DLL herr_t H5CX_get_modify_write_buf(bool *modify_write_buf);

/* "Getter" routines for LCPL properties cached in API context */
Expand Down Expand Up @@ -162,6 +163,7 @@ H5_DLL herr_t H5CX_init(void);
/* "Setter" routines for cached DXPL properties that must be returned to application */

H5_DLL void H5CX_set_no_selection_io_cause(uint32_t no_selection_io_cause);
H5_DLL void H5CX_set_actual_selection_io_mode(uint32_t actual_selection_io_mode);

#ifdef H5_HAVE_PARALLEL
H5_DLL void H5CX_set_mpio_actual_chunk_opt(H5D_mpio_actual_chunk_opt_mode_t chunk_opt);
Expand Down
19 changes: 10 additions & 9 deletions src/H5Dprivate.h
Original file line number Diff line number Diff line change
Expand Up @@ -78,15 +78,16 @@
#define H5D_MPIO_LOCAL_NO_COLLECTIVE_CAUSE_NAME \
"local_no_collective_cause" /* cause of broken collective I/O in each process */
#define H5D_MPIO_GLOBAL_NO_COLLECTIVE_CAUSE_NAME \
"global_no_collective_cause" /* cause of broken collective I/O in all processes */
#define H5D_XFER_EDC_NAME "err_detect" /* EDC */
#define H5D_XFER_FILTER_CB_NAME "filter_cb" /* Filter callback function */
#define H5D_XFER_CONV_CB_NAME "type_conv_cb" /* Type conversion callback function */
#define H5D_XFER_XFORM_NAME "data_transform" /* Data transform */
#define H5D_XFER_DSET_IO_SEL_NAME "dset_io_selection" /* Dataset I/O selection */
#define H5D_XFER_SELECTION_IO_MODE_NAME "selection_io_mode" /* Selection I/O mode */
#define H5D_XFER_NO_SELECTION_IO_CAUSE_NAME "no_selection_io_cause" /* Cause for no selection I/O */
#define H5D_XFER_MODIFY_WRITE_BUF_NAME "modify_write_buf" /* Modify write buffers */
"global_no_collective_cause" /* cause of broken collective I/O in all processes */
#define H5D_XFER_EDC_NAME "err_detect" /* EDC */
#define H5D_XFER_FILTER_CB_NAME "filter_cb" /* Filter callback function */
#define H5D_XFER_CONV_CB_NAME "type_conv_cb" /* Type conversion callback function */
#define H5D_XFER_XFORM_NAME "data_transform" /* Data transform */
#define H5D_XFER_DSET_IO_SEL_NAME "dset_io_selection" /* Dataset I/O selection */
#define H5D_XFER_SELECTION_IO_MODE_NAME "selection_io_mode" /* Selection I/O mode */
#define H5D_XFER_NO_SELECTION_IO_CAUSE_NAME "no_selection_io_cause" /* Cause for no selection I/O */
#define H5D_XFER_ACTUAL_SELECTION_IO_MODE_NAME "actual_selection_io_mode" /* Actual selection I/O mode */
#define H5D_XFER_MODIFY_WRITE_BUF_NAME "modify_write_buf" /* Modify write buffers */
#ifdef H5_HAVE_INSTRUMENTED_LIBRARY
/* Collective chunk instrumentation properties */
#define H5D_XFER_COLL_CHUNK_LINK_HARD_NAME "coll_chunk_link_hard"
Expand Down
Loading