Resizable Datasets

[franzpoeschel]

Based on topic-available-chunks since that one will be merged soon anyway.
Based on #927 since this requires ADIOS 2.7.0, see here for the diff.

Motivation: For some simulations, it is impossible to know at the beginning of writing an iteration's data e.g. the number of particles to expect until the end of the iteration. Hence, an iteration's datasets need to be resized while writing the iteration.

Hence, allow calling RecordComponent::resetDataset multiple times with multiple dimensions.

• This does not (yet) allow turning (1) empty (2) constant (3) array-form datasets into one another. This would be much trickier to implement and is not really necessary for the use cases above.
  EDIT: empty and constant record components can now be turned into one another and extending the extents of any of those two is fully supported.
• →Throw exceptions if empty/constant datasets are turned into an array-formed one. Other directions are already tested.
• The frontend does not compare the new extent to the old extent in any form whatsoever. Supporting (1) shrinking or (2) changing the dimensionality is the decision of the backend. Growing should generally be supported.

TODO:

• Documentation
• Implementation in the frontend.
• ADIOS2: Apparently, the ADIOS2 semantics for resizing datasets are targeted for a different use case: Variable<T>::SetShape does nothing to a variable within the same ADIOS step. So, the ADIOS semantics for resizing datasets are for a use case that we have been supporting for a long time anyway (by creating new datasets for each iteration): Having different-sized datasets across steps/iterations. Unless I'm missing something, implementing this in ADIOS2 is currently not possible. Within the same step, ADIOS2 will currently fully ignore the resizing and any write operation to any of the extended places. Enable changing Variable shape (global dimensions) of global arrays w… ornladios/ADIOS2#2545 fixes this. This means, we'll need to wait for ADIOS 2.7.0 to merge this PR. This is also the reason for the failing tests.
• JSON
• HDF5. There is already an old implementation of resizing datasets in there, it's just never been used and I'll need to check whether it works. HDF5 only allows this when writing with chunked storage HDF5: Chunking #406. So I suppose that this one is postponed until we implement writing chunked storage in HDF5. HDF5IOHandlerImpl::extendDataset will now throw an exception that gives notice of this.

[franzpoeschel]

Note to self: Rebase this back onto #927 once that one is rebased on the current dev.

[ax3l]

Oh no, this does not execute. The backend name is MPI_ADIOS2.
Correct syntax: if( ext == "bp" && write.backend() == "MPI_ADIOS1" )

#1126