Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

enable compact storage as an additional storage option for netCDF-4 vars #1568

Closed
edwardhartnett opened this issue Dec 4, 2019 · 0 comments · Fixed by #1570
Closed

enable compact storage as an additional storage option for netCDF-4 vars #1568

edwardhartnett opened this issue Dec 4, 2019 · 0 comments · Fixed by #1570

Comments

@edwardhartnett
Copy link
Contributor

edwardhartnett commented Dec 4, 2019

HDF5 supports a "compact" storage option. It helps a lot with parallel I/O performance, but may only be used for small variables (< 64 MB).

See #1554 for prototype implementation by @gsjaardema, who suggested this feature.

This can be accomplished without API changes. Currently we have two constants for nc_def_var_chunking():

/** In HDF5 files you can set storage for each variable to be either
 * contiguous or chunked, with nc_def_var_chunking().  This define is
 * used there. */
/**@{*/
#define NC_CHUNKED    0
#define NC_CONTIGUOUS 1
/**@}*/

By adding NC_COMPACT as a third constant, we can enable compact storage for users.

This feature is backward compatible to HDF5 1.8.x.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant