diff --git a/MANIFEST b/MANIFEST index 28b6826a815..f6094e9de5c 100644 --- a/MANIFEST +++ b/MANIFEST @@ -197,6 +197,7 @@ ./doxygen/aliases ./doxygen/Doxyfile.in ./doxygen/dox/api-compat-macros.dox +./doxygen/dox/H5AC_cache_config_t.dox ./doxygen/dox/H5Acreate.dox ./doxygen/dox/H5Aiterate.dox ./doxygen/dox/H5Fget_info.dox diff --git a/doxygen/Doxyfile.in b/doxygen/Doxyfile.in index 711fa7a2196..24fc4ba12e0 100644 --- a/doxygen/Doxyfile.in +++ b/doxygen/Doxyfile.in @@ -854,6 +854,7 @@ INPUT_ENCODING = UTF-8 FILE_PATTERNS = H5*public.h \ H5*module.h \ + H5FDmpio.h \ H5VLconnector.h \ H5VLconnector_passthru.h \ H5VLnative.h \ diff --git a/doxygen/dox/H5AC_cache_config_t.dox b/doxygen/dox/H5AC_cache_config_t.dox new file mode 100644 index 00000000000..9b9862b6c1e --- /dev/null +++ b/doxygen/dox/H5AC_cache_config_t.dox @@ -0,0 +1,415 @@ +/** + * \page H5AC-cache-config-t Metadata Cache Configuration + * \tableofcontents + * + * \section gcf General configuration fields + * + * \par version + * Integer field containing the version number of this version + * of the H5AC_cache_config_t structure. Any instance of + * H5AC_cache_config_t passed to the cache must have a known + * version number, or an error will be flagged. + * + * \par rpt_fcn_enabled + * \parblock + * Boolean field used to enable and disable the default + * reporting function. This function is invoked every time the + * automatic cache resize code is run, and reports on its activities. + * + * This is a debugging function, and should normally be turned off. + * \endparblock + * + * \par open_trace_file + * \parblock + * Boolean field indicating whether the trace_file_name + * field should be used to open a trace file for the cache. + * + * \Emph{*** DEPRECATED ***} Use \Code{H5Fstart/stop} logging functions instead + * + * The trace file is a debuging feature that allow the capture of + * top level metadata cache requests for purposes of debugging and/or + * optimization. This field should normally be set to \c FALSE, as + * trace file collection imposes considerable overhead. + * + * This field should only be set to \c TRUE when the trace_file_name + * contains the full path of the desired trace file, and either + * there is no open trace file on the cache, or the \c close_trace_file + * field is also \c TRUE. + * \endparblock + * + * \par close_trace_file + * \parblock + * Boolean field indicating whether the current trace + * file (if any) should be closed. + * + * \Emph{*** DEPRECATED ***} Use \Code{H5Fstart/stop} logging functions instead + * + * See the above comments on the open_trace_file field. This field + * should be set to \c FALSE unless there is an open trace file on the + * cache that you wish to close. + * \endparblock + * + * \par trace_file_name + * \parblock + * Full path of the trace file to be opened if the + * open_trace_file field is \c TRUE. + * + * \Emph{*** DEPRECATED ***} Use \Code{H5Fstart/stop} logging functions instead + * + * In the parallel case, an ascii representation of the mpi rank of + * the process will be appended to the file name to yield a unique + * trace file name for each process. + * + * The length of the path must not exceed #H5AC__MAX_TRACE_FILE_NAME_LEN + * characters. + * \endparblock + * + * \par evictions_enabled + * \parblock + * Boolean field used to either report the current + * evictions enabled status of the cache, or to set the cache's + * evictions enabled status. + * + * In general, the metadata cache should always be allowed to + * evict entries. However, in some cases it is advantageous to + * disable evictions briefly, and thereby postpone metadata + * writes. However, this must be done with care, as the cache + * can grow quickly. If you do this, re-enable evictions as + * soon as possible and monitor cache size. + * + * At present, evictions can only be disabled if automatic + * cache resizing is also disabled (that is, \Code{(incr_mode == + * H5C_incr__off ) && ( decr_mode == H5C_decr__off )}). There + * is no logical reason why this should be so, but it simplifies + * implementation and testing, and I can't think of any reason + * why it would be desireable. If you can think of one, I'll + * revisit the issue. (JM) + * \endparblock + * + * \par set_initial_size + * Boolean flag indicating whether the size of the + * initial size of the cache is to be set to the value given in + * the initial_size field. If set_initial_size is \c FALSE, the + * initial_size field is ignored. + * + * \par initial_size + * If enabled, this field contain the size the cache is + * to be set to upon receipt of this structure. Needless to say, + * initial_size must lie in the closed interval \Code{[min_size, max_size]}. + * + * \par min_clean_fraction + * \c double in the range 0 to 1 indicating the fraction + * of the cache that is to be kept clean. This field is only used + * in parallel mode. Typical values are 0.1 to 0.5. + * + * \par max_size + * Maximum size to which the cache can be adjusted. The + * supplied value must fall in the closed interval + * \Code{[MIN_MAX_CACHE_SIZE, MAX_MAX_CACHE_SIZE]}. Also, \c max_size must + * be greater than or equal to \c min_size. + * + * \par min_size + * Minimum size to which the cache can be adjusted. The + * supplied value must fall in the closed interval + * \Code{[H5C__MIN_MAX_CACHE_SIZE, H5C__MAX_MAX_CACHE_SIZE]}. Also, \c min_size + * must be less than or equal to \c max_size. + * + * \par epoch_length + * \parblock + * Number of accesses on the cache over which to collect + * hit rate stats before running the automatic cache resize code, + * if it is enabled. + * + * At the end of an epoch, we discard prior hit rate data and start + * collecting afresh. The epoch_length must lie in the closed + * interval \Code{[H5C__MIN_AR_EPOCH_LENGTH, H5C__MAX_AR_EPOCH_LENGTH]}. + * \endparblock + * + * + * \section csicf Cache size increase control fields + * + * \par incr_mode + * Instance of the \c H5C_cache_incr_mode enumerated type whose + * value indicates how we determine whether the cache size should be + * increased. At present there are two possible values: + * \li \c H5C_incr__off: Don't attempt to increase the size of the cache + * automatically.\n + * When this increment mode is selected, the remaining fields + * in the cache size increase section ar ignored. + * \li \c H5C_incr__threshold: Attempt to increase the size of the cache + * whenever the average hit rate over the last epoch drops + * below the value supplied in the \c lower_hr_threshold + * field.\n + * Note that this attempt will fail if the cache is already + * at its maximum size, or if the cache is not already using + * all available space. + * + * Note that you must set \c decr_mode to \c H5C_incr__off if you + * disable metadata cache entry evictions. + * + * \par lower_hr_threshold + * \parblock + * Lower hit rate threshold. If the increment mode + * (\c incr_mode) is \c H5C_incr__threshold and the hit rate drops below the + * value supplied in this field in an epoch, increment the cache size by + * \c size_increment. Note that cache size may not be incremented above + * \c max_size, and that the increment may be further restricted by the + * \c max_increment field if it is enabled. + * + * When enabled, this field must contain a value in the range [0.0, 1.0]. + * Depending on the \c incr_mode selected, it may also have to be less than + * \c upper_hr_threshold. + * \endparblock + * + * \par increment + * \parblock + * Double containing the multiplier used to derive the new + * cache size from the old if a cache size increment is triggered. + * The increment must be greater than 1.0, and should not exceed 2.0. + * + * The new cache size is obtained my multiplying the current max cache + * size by the increment, and then clamping to \c max_size and to stay + * within the \c max_increment as necessary. + * \endparblock + * + * \par apply_max_increment + * Boolean flag indicating whether the \c max_increment + * field should be used to limit the maximum cache size increment. + * + * \par max_increment + * If enabled by the \c apply_max_increment field described + * above, this field contains the maximum number of bytes by which the + * cache size can be increased in a single re-size. + * + * \par flash_incr_mode + * \parblock + * Instance of the \c H5C_cache_flash_incr_mode enumerated + * type whose value indicates whether and by which algorithm we should + * make flash increases in the size of the cache to accommodate insertion + * of large entries and large increases in the size of a single entry. + * + * The addition of the flash increment mode was occasioned by performance + * problems that appear when a local heap is increased to a size in excess + * of the current cache size. While the existing re-size code dealt with + * this eventually, performance was very bad for the remainder of the + * epoch. + * + * At present, there are two possible values for the \c flash_incr_mode: + * + * \li \c H5C_flash_incr__off: Don't perform flash increases in the size of the cache. + * + * \li \c H5C_flash_incr__add_space: Let \c x be either the size of a newly + * newly inserted entry, or the number of bytes by which the + * size of an existing entry has been increased.\n + * If \Code{x > flash_threshold * current max cache size}, + * increase the current maximum cache size by \Code{x * flash_multiple} + * less any free space in the cache, and star a new epoch. For + * now at least, pay no attention to the maximum increment. + * + * In both of the above cases, the flash increment pays no attention to + * the maximum increment (at least in this first incarnation), but DOES + * stay within max_size. + * + * With a little thought, it should be obvious that the above flash + * cache size increase algorithm is not sufficient for all circumstances + * -- for example, suppose the user round robins through + * \Code{(1/flash_threshold) +1} groups, adding one data set to each on each + * pass. Then all will increase in size at about the same time, requiring + * the max cache size to at least double to maintain acceptable + * performance, however the above flash increment algorithm will not be + * triggered. + * + * Hopefully, the add space algorithms detailed above will be sufficient + * for the performance problems encountered to date. However, we should + * expect to revisit the issue. + * \endparblock + * + * \par flash_multiple + * Double containing the multiple described above in the + * \c H5C_flash_incr__add_space section of the discussion of the + * \c flash_incr_mode section. This field is ignored unless \c flash_incr_mode + * is \c H5C_flash_incr__add_space. + * + * \par flash_threshold + * Double containing the factor by which current max cache + * size is multiplied to obtain the size threshold for the add_space flash + * increment algorithm. The field is ignored unless \c flash_incr_mode is + * \c H5C_flash_incr__add_space. + * + * + * \section csdcf Cache size decrease control fields + * + * \par decr_mode + * \parblock + * Instance of the \c H5C_cache_decr_mode enumerated type whose + * value indicates how we determine whether the cache size should be + * decreased. At present there are four possibilities. + * + * \li \c H5C_decr__off: Don't attempt to decrease the size of the cache + * automatically.\n + * When this increment mode is selected, the remaining fields + * in the cache size decrease section are ignored. + * \li \c H5C_decr__threshold: Attempt to decrease the size of the cache + * whenever the average hit rate over the last epoch rises + * above the value supplied in the \c upper_hr_threshold + * field. + * \li \c H5C_decr__age_out: At the end of each epoch, search the cache for + * entries that have not been accessed for at least the number + * of epochs specified in the epochs_before_eviction field, and + * evict these entries. Conceptually, the maximum cache size + * is then decreased to match the new actual cache size. However, + * this reduction may be modified by the \c min_size, the + * \c max_decrement, and/or the \c empty_reserve. + * \li \c H5C_decr__age_out_with_threshold: Same as age_out, but we only + * attempt to reduce the cache size when the hit rate observed + * over the last epoch exceeds the value provided in the + * \c upper_hr_threshold field. + * + * Note that you must set \c decr_mode to \c H5C_decr__off if you + * disable metadata cache entry evictions. + * \endparblock + * + * \par upper_hr_threshold + * \parblock + * Upper hit rate threshold. The use of this field + * varies according to the current \c decr_mode : + * + * \c H5C_decr__off or \c H5C_decr__age_out: The value of this field is + * ignored. + * + * \li \c H5C_decr__threshold: If the hit rate exceeds this threshold in any + * epoch, attempt to decrement the cache size by size_decrement.\n + * Note that cache size may not be decremented below \c min_size.\n + * Note also that if the \c upper_threshold is 1.0, the cache size\n + * will never be reduced. + * + * \li \c H5C_decr__age_out_with_threshold: If the hit rate exceeds this + * threshold in any epoch, attempt to reduce the cache size + * by evicting entries that have not been accessed for more + * than the specified number of epochs. + * \endparblock + * + * \par decrement + * \parblock + * This field is only used when the decr_mode is + * \c H5C_decr__threshold. + * + * The field is a double containing the multiplier used to derive the + * new cache size from the old if a cache size decrement is triggered. + * The decrement must be in the range 0.0 (in which case the cache will + * try to contract to its minimum size) to 1.0 (in which case the + * cache will never shrink). + * \endparblock + * + * \par apply_max_decrement + * Boolean flag used to determine whether decrements + * in cache size are to be limited by the \c max_decrement field. + * + * \par max_decrement + * Maximum number of bytes by which the cache size can be + * decreased in a single re-size. Note that decrements may also be + * restricted by the \c min_size of the cache, and (in age out modes) by + * the \c empty_reserve field. + * + * \par epochs_before_eviction + * \parblock + * Integer field used in \c H5C_decr__age_out and + * \c H5C_decr__age_out_with_threshold decrement modes. + * + * This field contains the number of epochs an entry must remain + * unaccessed before it is evicted in an attempt to reduce the + * cache size. If applicable, this field must lie in the range + * \Code{[1, H5C__MAX_EPOCH_MARKERS]}. + * \endparblock + * + * \par apply_empty_reserve + * Boolean field controlling whether the empty_reserve + * field is to be used in computing the new cache size when the + * decr_mode is H5C_decr__age_out or H5C_decr__age_out_with_threshold. + * + * \par empty_reserve + * \parblock + * To avoid a constant racheting down of cache size by small + * amounts in the \c H5C_decr__age_out and \c H5C_decr__age_out_with_threshold + * modes, this field allows one to require that any cache size + * reductions leave the specified fraction of unused space in the cache. + * + * The value of this field must be in the range [0.0, 1.0]. I would + * expect typical values to be in the range of 0.01 to 0.1. + * \endparblock + * + * + * \section pcf Parallel Configuration Fields + * + * In PHDF5, all operations that modify metadata must be executed collectively. + * + * We used to think that this was enough to ensure consistency across the + * metadata caches, but since we allow processes to read metadata individually, + * the order of dirty entries in the LRU list can vary across processes, + * which can result in inconsistencies between the caches. + * + * PHDF5 uses several strategies to prevent such inconsistencies in metadata, + * all of which use the fact that the same stream of dirty metadata is seen + * by all processes for purposes of synchronization. This is done by + * having each process count the number of bytes of dirty metadata generated, + * and then running a "sync point" whenever this count exceeds a user + * specified threshold (see \c dirty_bytes_threshold below). + * + * The current metadata write strategy is indicated by the + * \c metadata_write_strategy field. The possible values of this field, along + * with the associated metadata write strategies are discussed below. + * + * \par dirty_bytes_threshold + * \parblock + * Threshold of dirty byte creation used to + * synchronize updates between caches. (See above for outline and + * motivation.) + * + * This value MUST be consistent across all processes accessing the + * file. This field is ignored unless HDF5 has been compiled for + * parallel. + * \endparblock + * + * \par metadata_write_strategy + * Integer field containing a code indicating the + * desired metadata write strategy. The valid values of this field + * are enumerated and discussed below: + * + * \li #H5AC_METADATA_WRITE_STRATEGY__PROCESS_0_ONLY\n + * When metadata_write_strategy is set to this value, only process + * zero is allowed to write dirty metadata to disk. All other + * processes must retain dirty metadata until they are informed at + * a sync point that the dirty metadata in question has been written + * to disk.\n + * When the sync point is reached (or when there is a user generated + * flush), process zero flushes sufficient entries to bring it into + * complience with its min clean size (or flushes all dirty entries in + * the case of a user generated flush), broad casts the list of + * entries just cleaned to all the other processes, and then exits + * the sync point.\n + * Upon receipt of the broadcast, the other processes mark the indicated + * entries as clean, and leave the sync point as well. + * + * \li #H5AC_METADATA_WRITE_STRATEGY__DISTRIBUTED\n + * In the distributed metadata write strategy, process zero still makes + * the decisions as to what entries should be flushed, but the actual + * flushes are distributed across the processes in the computation to + * the extent possible.\n + * In this strategy, when a sync point is triggered (either by dirty + * metadata creation or manual flush), all processes enter a barrier.\n + * On the other side of the barrier, process 0 constructs an ordered + * list of the entries to be flushed, and then broadcasts this list + * to the caches in all the processes.\n + * All processes then scan the list of entries to be flushed, flushing + * some, and marking the rest as clean. The algorithm for this purpose + * ensures that each entry in the list is flushed exactly once, and + * all are marked clean in each cache.\n + * Note that in the case of a flush of the cache, no message passing + * is necessary, as all processes have the same list of dirty entries, + * and all of these entries must be flushed. Thus in this case it is + * sufficient for each process to sort its list of dirty entries after + * leaving the initial barrier, and use this list as if it had been + * received from process zero.\n + * To avoid possible messages from the past/future, all caches must + * wait until all caches are done before leaving the sync point. + */ \ No newline at end of file diff --git a/doxygen/dox/mainpage.dox b/doxygen/dox/mainpage.dox index 83fc323de0c..eda967b421b 100644 --- a/doxygen/dox/mainpage.dox +++ b/doxygen/dox/mainpage.dox @@ -1,36 +1,44 @@ -/*! \mainpage API Documentation for HDF5 Version 1.13 (Draft) +/*! \mainpage HDF5 C-API Reference + * + * The HDF5 C-API provides applications with fine-grained control over all + * aspects HDF5 functionality. This functionality is grouped into the following + * \Emph{modules}: + * \li \ref H5A "Attributes" — Management of HDF5 attributes (\ref H5A) + * \li \ref H5D "Datasets" — Management of HDF5 datasets (\ref H5D) + * \li \ref H5S "Dataspaces" — Management of HDF5 dataspaces which describe the shape of datasets and attributes (\ref H5S) + * \li \ref H5T "Datatypes" — Management of datatypes which describe elements of datasets and attributes (\ref H5T) + * \li \ref H5E "Error Handling" — Functions for handling errors that occur within HDF5 (\ref H5E) + * \li \ref H5F "Files" — Management of HDF5 files (\ref H5F) + * \li \ref H5Z "Filters" — Configuration of filters that process data during I/O operation (\ref H5Z) + * \li \ref H5G "Groups" — Management of groups in HDF5 files (\ref H5G) + * \li \ref H5I "Identifiers" — Management of object identifiers and object names (\ref H5I) + * \li \ref H5 "Library" — General purpose library functions (\ref H5) + * \li \ref H5L "Links" — Management of links in HDF5 groups (\ref H5L) + * \li \ref H5O "Objects" — Management of objects in HDF5 files (\ref H5O) + * \li \ref H5PL "Plugins" — Programmatic control over dynamically loaded plugins (\ref H5PL) + * \li \ref H5P "Property Lists" — Management of property lists to control HDF5 library behavior (\ref H5P) + * \li \ref H5R "References" — Management of references to specific objects and data regions in an HDF5 file (\ref H5R) + * \li \ref H5VL "Virtual Object Layer" — Management of the Virtual Object Layer (\ref H5VL) + * + * Here are a few simple rules to follow: + * + * \li \Bold{Handle discipline:} If you acquire a handle (by creation or coopy), \Emph{you own it!} (..., i.e., you have to close it.) + * \li \Bold{Dynamic memory allocation:} ... + * \li \Bold{Use of locations:} Identifier + name combo + * + * \attention \Bold{C++ Developers using HDF5 C-API functions beware:}\n + * If a C routine that takes a function pointer as an argument is called from + * within C++ code, the C routine should be returned from normally. + * Examples of this kind of routine include callbacks such as H5Pset_elink_cb() + * and H5Pset_type_conv_cb() and functions such as H5Tconvert() and H5Ewalk2().\n + * Exiting the routine in its normal fashion allows the HDF5 C library to clean + * up its work properly. In other words, if the C++ application jumps out of + * the routine back to the C++ \c catch statement, the library is not given the + * opportunity to close any temporary data structures that were set up when the + * routine was called. The C++ application should save some state as the + * routine is started so that any problem that occurs might be diagnosed. * * \todo Fix the search form for server deployments. * \todo Make it mobile-friendly * - * \section intro_sec Introduction - * - * \todo Write an introduction. - * - * \section quick_links Quick Links - * - *
int version |
+ * IN: Integer field indicating the the version of the H5AC_cache_config_t in use. This field should + * be set to #H5AC__CURR_CACHE_CONFIG_VERSION (defined in H5ACpublic.h). | + *
hbool_t rpt_fcn_enabled |
+ * OUT: Boolean flag indicating whether the adaptive cache resize report function is enabled. This
+ * field should almost always be set to disabled ( The + * report function is not supported code, and can be expected to change between versions of the + * library. Use it at your own risk. |
+ *
hbool_t open_trace_file |
+ * OUT: Boolean field indicating whether the trace_file_name field should be used to open
+ * a trace file for the cache. This field will always be set to 0 in this context. |
+ *
hbool_t close_trace_file |
+ * OUT: Boolean field indicating whether the current trace file (if any) should be closed. This field
+ * will always be set to 0 in this context. |
char*trace_file_name |
+ * OUT: Full path name of the trace file to be opened if the open_trace_file field is set
+ * to 1 . This field will always be set to the empty string in this context. |
hbool_t evictions_enabled |
+ * OUT: Boolean flag indicating whether metadata cache entry evictions are + * enabled. | + *
hbool_t set_initial_size |
+ * OUT: Boolean flag indicating whether the cache should be created with a user specified initial
+ * maximum size. If the configuration is loaded from the cache, this flag will always be set
+ * to |
+ *
size_t initial_size |
+ * OUT: Initial maximum size of the cache in bytes, if applicable. If the configuration is loaded + * from the cache, this field will contain the cache maximum size as of the time of the + * call. |
+ *
double min_clean_fraction |
+ * OUT: Float value specifying the minimum fraction of the cache that must be kept either clean or + * empty when possible. | + *
size_t max_size |
+ * OUT: Upper bound (in bytes) on the range of values that the adaptive cache resize code can select + * as the maximum cache size. | + *
size_t min_size |
+ * OUT: Lower bound (in bytes) on the range of values that the adaptive cache resize code can select + * as the maximum cache size. | + *
long int epoch_length |
+ * OUT: Number of cache accesses between runs of the adaptive cache resize + * code. | + *
enum H5C_cache_incr_mode incr_mode |
+ * OUT: Enumerated value indicating the operational mode of the automatic cache size increase code. At
+ * present, only the following values are legal: \c H5C_incr__off: Automatic cache size increase is + * disabled. \c H5C_incr__threshold: Automatic cache size increase is enabled using the hit rate + * threshold algorithm. |
+ *
double lower_hr_threshold |
+ * OUT: Hit rate threshold used in the hit rate threshold cache size increase algorithm. | + *
double increment |
+ * OUT: The factor by which the current maximum cache size is multiplied to obtain an initial new + * maximum cache size if a size increase is triggered in the hit rate threshold cache size increase + * algorithm. | + *
hbool_t apply_max_increment |
+ * OUT: Boolean flag indicating whether an upper limit will be applied to the size of cache size + * increases. | + *
size_t max_increment |
+ * OUT: The maximum number of bytes by which the maximum cache size can be increased in a single step + * -- if applicable. | + *
enum H5C_cache_flash_incr_mode flash_incr_mode |
+ * OUT: Enumerated value indicating the operational mode of the flash cache size increase code. At
+ * present, only the following values are legal: \c H5C_flash_incr__off: Flash cache size increase is + * disabled. \c H5C_flash_incr__add_space: Flash cache size increase is enabled using the add space + * algorithm. |
+ *
double flash_threshold |
+ * OUT: The factor by which the current maximum cache size is multiplied to obtain the minimum size + * entry / entry size increase which may trigger a flash cache size + * increase. | + *
double flash_multiple |
+ * OUT: The factor by which the size of the triggering entry / entry size increase is multiplied to
+ * obtain the initial cache size increment. This increment may be reduced to reflect existing free space
+ * in the cache and the max_size field above. |
+ *
Decrement configuration + * section: | + *|
enum H5C_cache_decr_mode decr_mode |
+ * OUT: Enumerated value indicating the operational mode of the automatic cache size decrease code. At
+ * present, the following values are legal: H5C_decr__off: Automatic cache size decrease is disabled, + * and the remaining decrement fields are ignored. H5C_decr__threshold: Automatic cache size + * decrease is enabled using the hit rate threshold algorithm. H5C_decr__age_out: Automatic cache + * size decrease is enabled using the ageout algorithm. H5C_decr__age_out_with_threshold: + * Automatic cache size decrease is enabled using the ageout with hit rate threshold + * algorithm |
+ *
double upper_hr_threshold |
+ * OUT: Upper hit rate threshold. This value is only used if the decr_mode is either + * H5C_decr__threshold or H5C_decr__age_out_with_threshold. | + *
double decrement |
+ * OUT: Factor by which the current max cache size is multiplied to obtain an initial value for the + * new cache size when cache size reduction is triggered in the hit rate threshold cache size reduction + * algorithm. | + *
hbool_t apply_max_decrement |
+ * OUT: Boolean flag indicating whether an upper limit should be applied to the size of cache size + * decreases. | + *
size_t max_decrement |
+ * OUT: The maximum number of bytes by which cache size can be decreased if any single step, if + * applicable. | + *
int epochs_before_eviction |
+ * OUT: The minimum number of epochs that an entry must reside unaccessed in cache before being + * evicted under either of the ageout cache size reduction algorithms. | + *
hbool_t apply_empty_reserve |
+ * OUT: Boolean flag indicating whether an empty reserve should be maintained under either of the + * ageout cache size reduction algorithms. | + *
double empty_reserve |
+ * OUT: Empty reserve for use with the ageout cache size reduction algorithms, if applicable. | + *
int dirty_bytes_threshold |
+ * OUT: Threshold number of bytes of dirty metadata generation for triggering synchronizations of the
+ * metadata caches serving the target file in the parallel case. Synchronization occurs whenever the + * number of bytes of dirty metadata created since the last synchronization exceeds this limit. |
+ *
int version |
+ * IN: Integer field indicating the the version of the H5AC_cache_config_t in use. This + * field should be set to #H5AC__CURR_CACHE_CONFIG_VERSION (defined + * in H5ACpublic.h). | + *
hbool_t rpt_fcn_enabled |
+ * IN: Boolean flag indicating whether the adaptive cache resize report function is enabled. This
+ * field should almost always be set to disabled (0 ). Since resize algorithm activity is reported
+ * via stdout, it MUST be set to disabled (0 ) on Windows machines.The report function is not + * supported code, and can be expected to change between versions of the library. Use it at your own + * risk. |
+ *
hbool_t open_trace_File |
+ * IN: Boolean field indicating whether the trace_file_name field should be used to open
+ * a trace file for the cache.The trace file is a debuging feature that allows the capture of top level
+ * metadata cache requests for purposes of debugging and/or optimization. This field should normally be set
+ * to This field should only be
+ * set to The trace file feature is unsupported unless used at the direction of The HDF + * Group. It is intended to allow The HDF Group to collect a trace of cache activity in cases of occult + * failures and/or poor performance seen in the field, so as to aid in reproduction in the lab. If you use it + * absent the direction of The HDF Group, you are on your + * own. |
+ *
hbool_t close_trace_file |
+ * IN: Boolean field indicating whether the current trace file (if any) should be closed. See the
+ * above comments on the The trace file feature is + * unsupported unless used at the direction of The HDF Group. It is intended to allow The HDF Group to collect + * a trace of cache activity in cases of occult failures and/or poor performance seen in the field, so as to + * aid in reproduction in the lab. If you use it absent the direction of The HDF Group, you are on your + * own. |
+ *
char trace_file_name[] |
+ * IN: Full path of the trace file to be opened if the open_trace_file field is set
+ * to 1 .In the parallel case, an ascii representation of the mpi rank of the process will be + * appended to the file name to yield a unique trace file name for each process. The length of the path + * must not exceed #H5AC__MAX_TRACE_FILE_NAME_LEN characters. The trace file feature is + * unsupported unless used at the direction of The HDF Group. It is intended to allow The HDF Group to collect + * a trace of cache activity in cases of occult failures and/or poor performance seen in the field, so as to + * aid in reproduction in the lab. If you use it absent the direction of The HDF Group, you are on your + * own. |
+ *
hbool_t evictions_enabled |
+ * IN: A boolean flag indicating whether evictions from the metadata cache are enabled. This flag is
+ * initially set to enabled (1 ).In rare circumstances, the raw data throughput requirements
+ * may be so high that the user wishes to postpone metadata writes so as to reserve I/O throughput for raw
+ * data. The The When
+ * this flag is set to disabled ( Evictions will be re-enabled when this field
+ * is set back to |
+ *
hbool_t set_initial_size |
+ * IN: Boolean flag indicating whether the cache should be forced to the user specified initial + * size. | + *
size_t initial_size |
+ * IN: If set_initial_size is set to 1 , then initial_size must
+ * contain the desired initial size in bytes. This value must lie in the closed interval [min_size,
+ * max_size] . (see below) |
+ *
double min_clean_fraction |
+ * IN: This field specifies the minimum fraction of the cache that must be kept either clean or
+ * empty. The value must lie in the interval [0.0, 1.0]. 0.01 is a good place to start in the serial case. + * In the parallel case, a larger value is needed -- + * see Metadata Caching in HDF5 in the collection + * "Advanced Topics in HDF5." |
+ *
size_t max_size |
+ * IN: Upper bound (in bytes) on the range of values that the adaptive cache resize code can select as + * the maximum cache size. | + *
size_t min_size |
+ * IN: Lower bound (in bytes) on the range of values that the adaptive cache resize code can select as + * the maximum cache size. | + *
long int epoch_length |
+ * IN: Number of cache accesses between runs of the adaptive cache resize code. 50,000 is a good + * starting number. | + *
enum H5C_cache_incr_mode incr_mode |
+ * IN: Enumerated value indicating the operational mode of the automatic cache size increase code. At
+ * present, only two values are legal: \c H5C_incr__off: Automatic cache size increase is disabled, and the + * remaining increment fields are ignored. \c H5C_incr__threshold: Automatic cache size increase is enabled + * using the hit rate threshold + * algorithm. |
+ *
double lower_hr_threshold |
+ * IN: Hit rate threshold used by the hit rate threshold cache size increment algorithm. When the + * hit rate over an epoch is below this threshold and the cache is full, the maximum size of the cache is + * multiplied by increment (below), and then clipped as necessary to stay within max_size, and possibly + * max_increment. This field must lie in the interval [0.0, 1.0]. 0.8 or 0.9 is a good starting + * point. |
+ *
double increment |
+ * IN: Factor by which the hit rate threshold cache size increment algorithm multiplies the current
+ * maximum cache size to obtain a tentative new cache size. The actual cache size increase will be clipped + * to satisfy the max_size specified in the general configuration, and possibly max_increment below. The + * parameter must be greater than or equal to 1.0 -- 2.0 is a reasonable value. If you set it to 1.0, + * you will effectively disable cache size + * increases. |
+ *
hbool_t apply_max_increment |
+ * IN: Boolean flag indicating whether an upper limit should be applied to the size of cache size + * increases. | + *
size_t max_increment |
+ * IN: Maximum number of bytes by which cache size can be increased in a single step -- if + * applicable. | + *
enum H5C_cache_flash_incr_mode flash_incr_mode |
+ * IN: Enumerated value indicating the operational mode of the flash cache size increase code. At
+ * present, only the following values are legal: \c H5C_flash_incr__off: Flash cache size increase is + * disabled. \c H5C_flash_incr__add_space: Flash cache size increase is enabled using the add space + * algorithm. |
+ *
double flash_threshold |
+ * IN: The factor by which the current maximum cache size is multiplied to obtain the minimum size
+ * entry / entry size increase which may trigger a flash cache size increase. At present, this value must + * lie in the range [0.1, 1.0]. |
+ *
double flash_multiple |
+ * IN: The factor by which the size of the triggering entry / entry size increase is multiplied to
+ * obtain the initial cache size increment. This increment may be reduced to reflect existing free space in
+ * the cache and the max_size field above.At present, this field must lie in the range [0.1, + * 10.0]. |
+ *
enum H5C_cache_decr_mode decr_mode |
+ * IN: Enumerated value indicating the operational mode of the automatic cache size decrease code. At
+ * present, the following values are legal: \c H5C_decr__off: Automatic cache size decrease is + * disabled. \c H5C_decr__threshold: Automatic cache size decrease is enabled using the hit rate threshold + * algorithm. \c H5C_decr__age_out: Automatic cache size decrease is enabled using the ageout + * algorithm. \c H5C_decr__age_out_with_threshold: Automatic cache size decrease is enabled using the + * ageout with hit rate threshold + * algorithm |
+ *
double upper_hr_threshold |
+ * IN: Hit rate threshold for the hit rate threshold and ageout with hit rate threshold cache size
+ * decrement algorithms. When \c decr_mode is \c H5C_decr__threshold, and the hit rate over a given epoch exceeds + * the supplied threshold, the current maximum cache size is multiplied by decrement to obtain a tentative new + * (and smaller) maximum cache size. When \c decr_mode is \c H5C_decr__age_out_with_threshold, there is no + * attempt to find and evict aged out entries unless the hit rate in the previous epoch exceeded the supplied + * threshold. This field must lie in the interval [0.0, 1.0]. For \c H5C_incr__threshold, .9995 or + * .99995 is a good place to start. For \c H5C_decr__age_out_with_threshold, .999 might be more + * useful. |
+ *
double decrement |
+ * IN: In the hit rate threshold cache size decrease algorithm, this parameter contains the factor by
+ * which the current max cache size is multiplied to produce a tentative new cache size. The actual cache + * size decrease will be clipped to satisfy the min_size specified in the general configuration, and possibly + * max_decrement below. The parameter must be be in the interval [0.0, 1.0]. If you set it to 1.0, + * you will effectively disable cache size decreases. 0.9 is a reasonable starting + * point. |
+ *
hbool_t apply_max_decrement |
+ * IN: Boolean flag indicating whether an upper limit should be applied to the size of cache size + * decreases. | + *
size_t max_decrement |
+ * IN: Maximum number of bytes by which the maximum cache size can be decreased in any single step -- + * if applicable. | + *
int epochs_before_eviction |
+ * IN: In the ageout based cache size reduction algorithms, this field contains the minimum number of + * epochs an entry must remain unaccessed in cache before the cache size reduction algorithm tries to evict + * it. 3 is a reasonable value. | + *
hbool_t apply_empty_reserve |
+ * IN: Boolean flag indicating whether the ageout based decrement algorithms will maintain a empty + * reserve when decreasing cache size. | + *
double empty_reserve |
+ * IN: Empty reserve as a fraction of maximum cache size if applicable. When so directed, the ageout + * based algorithms will not decrease the maximum cache size unless the empty reserve can be met. The + * parameter must lie in the interval [0.0, 1.0]. 0.1 or 0.05 is a good place to + * start. |
+ *
int dirty_bytes_threshold |
+ * IN: Threshold number of bytes of dirty metadata generation for triggering synchronizations of the
+ * metadata caches serving the target file in the parallel case. Synchronization occurs whenever the number + * of bytes of dirty metadata created since the last synchronization exceeds this limit. This field only + * applies to the parallel case. While it is ignored elsewhere, it can still draw a value out of bounds + * error. It must be consistant across all caches on any given file. By default, this field is set + * to 256 KB. It shouldn't be more than half the current maximum cache size times the minimum clean + * fraction. |
+ *
Index for \Code{retries[]} | + *Metadata entries* | + *
---|---|
0 | Object header (version 2) |
1 | Object header chunk (version 2) |
2 | B-tree header (version 2) |
3 | B-tree internal node (version 2) |
4 | B-tree leaf node (version 2) |
5 | Fractal heap header |
6 | Fractal heap direct block (optional checksum) |
7 | Fractal heap indirect block |
8 | Free-space header |
9 | Free-space sections |
10 | Shared object header message table |
11 | Shared message record list |
12 | Extensive array header |
13 | Extensive array index block |
14 | Extensive array super block |
15 | Extensive array data block |
16 | Extensive array data block page |
17 | Fixed array super block |
18 | Fixed array data block |
19 | Fixed array data block page |
20 | File's superblock (version 2) |
* All entries are of version 0 (zero) unless indicated otherwise. |