Operators do not work with HDF5

[rupertnash]

Describe the bug
Plugin operators do not run when the engine is set to HDF5.

To Reproduce

1. Compile ADIOS 2 (I'm using release 2.9.2) with Sodium
2. Modify the Sodium plugin example XML to use HDF5 output
3. Run the write sodium plugin example
4. Observe that HDF5 output is not encrypted

rwn-e609@ln01:~/adios2-build/debug-sodium> cp ~/ADIOS2/examples/plugins/operator/example_operator.xml operator_hdf5.xml
rwn-e609@ln01:~/adios2-build/debug-sodium> emacs operator_hdf5.xml 
rwn-e609@ln01:~/adios2-build/debug-sodium> cat operator_hdf5.xml
<?xml version="1.0"?>
<!-- Config XML file for the
        hello_skeleton*
     executables in build/bin -->

<adios-config>

    <!--====================================
           Configuration for the Writer
        ====================================-->

    <io name="writer">
        <variable name="data">
            <operation type="plugin">
                <parameter key="PluginName" value="OperatorPlugin"/>
                <parameter key="PluginLibrary" value="EncryptionOperator" />
                <parameter key="SecretKeyFile" value="test-key" />
            </operation>
        </variable>
        <engine type="HDF5">
        </engine>
    </io>


    <!--====================================
           Configuration for the Reader
        ====================================-->

    <io name="reader">
        <variable name="data">
            <operation type="plugin">
                <parameter key="PluginName" value="OperatorPlugin"/>
                <parameter key="PluginLibrary" value="EncryptionOperator" />
                <parameter key="SecretKeyFile" value="test-key" />
            </operation>
        </variable>
        <engine type="HDF5">
        </engine>
    </io>
</adios-config>
rwn-e609@ln01:~/adios2-build/debug-sodium> bin/exampleOperatorPlugin_write operator_hdf5.xml
rwn-e609@ln01:~/adios2-build/debug-sodium> file testOperator.bp 
testOperator.bp: Hierarchical Data Format (version 5) data
rwn-e609@ln01:~/adios2-build/debug-sodium> h5dump testOperator.bp 
HDF5 "testOperator.bp" {
GROUP "/" {
   ATTRIBUTE "NumSteps" {
      DATATYPE  H5T_STD_U32LE
      DATASPACE  SCALAR
      DATA {
      (0): 1
      }
   }
   GROUP "Step0" {
      DATASET "data" {
         DATATYPE  H5T_IEEE_F64LE
         DATASPACE  SIMPLE { ( 100 ) / ( 100 ) }
         DATA {
         (0): 0.0001, 1.0001, 2.0001, 3.0001, 4.0001, 5.0001, 6.0001, 7.0001,
         (8): 8.0001, 9.0001, 1.0001, 2.0001, 3.0001, 4.0001, 5.0001, 6.0001,
         (16): 7.0001, 8.0001, 9.0001, 8.0001, 2.0001, 3.0001, 4.0001,
         (23): 5.0001, 6.0001, 7.0001, 8.0001, 9.0001, 8.0001, 7.0001,
         (30): 3.0001, 4.0001, 5.0001, 6.0001, 7.0001, 8.0001, 9.0001,
         (37): 8.0001, 7.0001, 6.0001, 4.0001, 5.0001, 6.0001, 7.0001,
         (44): 8.0001, 9.0001, 8.0001, 7.0001, 6.0001, 5.0001, 5.0001,
         (51): 6.0001, 7.0001, 8.0001, 9.0001, 8.0001, 7.0001, 6.0001,
         (58): 5.0001, 4.0001, 6.0001, 7.0001, 8.0001, 9.0001, 8.0001,
         (65): 7.0001, 6.0001, 5.0001, 4.0001, 3.0001, 7.0001, 8.0001,
         (72): 9.0001, 8.0001, 7.0001, 6.0001, 5.0001, 4.0001, 3.0001,
         (79): 2.0001, 8.0001, 9.0001, 8.0001, 7.0001, 6.0001, 5.0001,
         (86): 4.0001, 3.0001, 2.0001, 1.0001, 9.0001, 8.0001, 7.0001,
         (93): 6.0001, 5.0001, 4.0001, 3.0001, 2.0001, 1.0001, 0.0001
         }
      }
   }
}
}

Expected behavior
HDF5 file contains encrypted data

Desktop (please complete the following information):

• OS/Platform: ARCHER 2 HPE Cray Linux (based on SLES 15.4)
• Build: Cray clang version 15.0.0, CMAKE_BUILD_TYPE=Debug, Cray PrgEnv 8.3.3

Additional context
A brief search of the ADIOS source suggests that operators are only called by the BP, BP5, and DataMan engines. This is perhaps by design, in which case this is a documentation bug / paper erratum (no mention of this limitation in the Software X paper)

[pnorbert]

The HDF5 engine does not support any operators. The HDF5 engine is using the parallel HDF5 library functions to write/read data as is and produce a proper HDF5 file readable by HDF5 tools. We have no intention to somehow match an operator's code to existing HDF5 functions.

We could use operators to transform the data before writing and inverse transform the data at reading (by calling the operator code from the HDF5 Engine of ADIOS), and use HDF5 file format of storing the transformed data. However, the result file would not be readable by HDF5 tools, only by ADIOS. Hence, we have no intention to pursue this either.

[rupertnash]

Thanks Norbert - I appreciate that it's not possible to match the operator's code to HDF5, since operators are arbitrary C++ functions, but why not just store the transformed bytes? I take your point about transformed data being more difficult to use but that is a problem that users have to ask for. Silently ignoring requested operations is probably worse.

There is no mention I could see in the docs about HDF5 not supporting this. The SoftwareX paper says an Operator "Applies Variable data operations inside an Engine", so you can see why I am surprised that this is not the case...

I would suggest that an exception should be thrown if the HDF5 engine encounters a variable with associated operators.

[rupertnash]

I briefly attempted a solution using HDF5 filters but this was not trivial. Particular problems were:

• the way that ADIOS2 operators expect to have parameters like the block count/start
• differing views on memory allocation (HFD5 expects filters to do it if needed, ADIOS expects operators to just write where they're told)

Instead I have made PR #3951 to error if the user does something unsupported.

Closing that PR and updating docs will resolve this issue. Thanks

[guj]

PR #3951 is merged. Issue closed.