Add tutorials

docs/user_guide/source/advanced/ecp_hardware.rst

@@ -33,27 +33,29 @@ your tests.
 Examples of launching ADIOS2 SST unit tests using MPI DP:
 
 .. code-block:: bash
- # We omit some of the srun (SLURM) arguments which are specific of the project
- # you are working on. Note that you could avoid calling srun directly by
- # setting the CMAKE variable `MPIEXEC_EXECUTABLE`.
 
- # Launch simple writer test instance
- srun {PROJFLAGS }-N 1 /gpfs/alpine/proj-shared/csc331/vbolea/ADIOS2-build/bin/TestCommonWrite SST mpi_dp_test CPCommPattern=Min,MarshalMethod=BP5'
+  # We omit some of the srun (SLURM) arguments which are specific of the project
+  # you are working on. Note that you could avoid calling srun directly by
+  # setting the CMAKE variable `MPIEXEC_EXECUTABLE`.
 
- # On another terminal launch multiple instances of the Reader test
- srun {PROJFLAGS} -N 2 /gpfs/alpine/proj-shared/csc331/vbolea/ADIOS2-build/bin/TestCommonRead SST mpi_dp_test
+  # Launch simple writer test instance
+  srun {PROJFLAGS } -N 1 /gpfs/alpine/proj-shared/csc331/vbolea/ADIOS2-build/bin/TestCommonWrite SST mpi_dp_test CPCommPattern=Min,MarshalMethod=BP5
+
+  # On another terminal launch multiple instances of the Reader test
+  srun {PROJFLAGS} -N 2 /gpfs/alpine/proj-shared/csc331/vbolea/ADIOS2-build/bin/TestCommonRead SST mpi_dp_test
 
 Alternatively, you can configure your CMake build to use srun directly:
 
 .. code-block:: bash
- cmake . -DMPIEXEC_EXECUTABLE:FILEPATH="/usr/bin/srun" \
-         -DMPIEXEC_EXTRA_FLAGS:STRING="-A{YourProject} -pbatch -t10" \
-         -DMPIEXEC_NUMPROC_FLAG:STRING="-N" \
-         -DMPIEXEC_MAX_NUMPROCS:STRING="-8" \
-         -DADIOS2_RUN_MPI_MPMD_TESTS=OFF
 
- cmake --build .
- ctest
+  cmake . -DMPIEXEC_EXECUTABLE:FILEPATH="/usr/bin/srun" \
+       -DMPIEXEC_EXTRA_FLAGS:STRING="-A{YourProject} -pbatch -t10" \
+       -DMPIEXEC_NUMPROC_FLAG:STRING="-N" \
+       -DMPIEXEC_MAX_NUMPROCS:STRING="-8" \
+       -DADIOS2_RUN_MPI_MPMD_TESTS=OFF
+
+  cmake --build .
+  ctest
 
- # monitor your jobs
- watch -n1 squeue -l -u $USER
+  # monitor your jobs
+  watch -n1 squeue -l -u $USER

docs/user_guide/source/advanced/gpu_aware.rst

@@ -35,17 +35,17 @@ If there is no CUDA toolkit installed, cmake will turn CUDA off automatically. A
 When building ADIOS2 with CUDA enabled, the user is responsible with setting the correct ``CMAKE_CUDA_ARCHITECTURES`` (e.g. for Summit the ``CMAKE_CUDA_ARCHITECTURES`` needs to be set to 70 to match the NVIDIA Volta V100).
 
 Building with Kokkos enabled
---------------------------
+----------------------------
 
 The Kokkos library can be used to enable GPU within ADIOS2. Based on how Kokkos is build, either the CUDA, HIP or SYCL backend will be enabled. Building with Kokkos requires ``-DADIOS2_USE_Kokkos=ON``. The ``CMAKE_CUDA_ARCHITECTURES`` is set automanically to point to the same architecture used when configuring the Kokkos library.
 
 .. note::
     Kokkos version >= 3.7 is required to enable the GPU backend in ADIOS2
 
 
-*************
+****************
 Writing GPU code
-*************
+****************
 
 The following is a simple example of writing data to storage directly from a GPU buffer allocated with CUDA relying on the automatic detection of device pointers in ADIOS2. The ADIOS2 API is identical to codes using Host buffers for both the read and write logic.
 

docs/user_guide/source/advanced/query.rst

@@ -3,7 +3,7 @@ ADIOS2 query API
 #################
 
 The query API in ADIOS2 allows a client to pass a query in XML or json format,
-and get back a list of blocks or subblocks that contains hits. 
+and get back a list of blocks or sub-blocks that contains hits.
 Both BP4 and BP5 engines are supported.  
 
 
@@ -21,14 +21,14 @@ to construct a query and evaluate using the engine.
         // configFile has query, can be either xml or json
         QueryWorker(const std::string &configFile, adios2::Engine &engine);
 
-	// touched_blocks is a list of regions specified by (start, count),
-	// that contains data that satisfies the query file
+	     // touched_blocks is a list of regions specified by (start, count),
+	     // that contains data that satisfies the query file
         void GetResultCoverage(std::vector<adios2::Box<adios2::Dims>> &touched_blocks);
     ... 
     }
 
 A Sample Compound Query  
-----------------------
+-----------------------
 
 This query targets a 1D variable "doubleV", data of interest is (x  > 6.6) or (x < -0.17) or (2.8 < x < 2.9) 
 In addition, this query also specied an output region [start=5,count=80]. 

docs/user_guide/source/components/engine.rst

@@ -2,6 +2,8 @@
 Engine
 ******
 
+.. _sec:basics_interface_components_engine:
+
 The Engine abstraction component serves as the base interface to the actual IO systems executing the heavy-load tasks performed when producing and consuming data.
 
 Engine functionality works around two concepts:
@@ -309,8 +311,8 @@ The ``data`` fed to the ``Put`` function is assumed to be allocated on the Host
    Only the BP4 and BP5 engines are capable of receiving device allocated buffers.
 
 
-PerformsPuts
-------------
+PerformPuts
+-----------
 
    Executes all pending ``Put`` calls in deferred mode and collects
    span data.  Specifically this call copies Put(Deferred) data into
@@ -322,8 +324,8 @@ PerformsPuts
    impact performance on some engines.
 
 
-PerformsDataWrite
-------------
+PerformDataWrite
+----------------
 
    If supported by the engine, moves data from prior ``Put`` calls to disk
 
@@ -361,13 +363,13 @@ The following table summarizes the memory contracts required by ADIOS2 engines b
 +----------+-------------+-----------------------------------------------+
 | Get      | Data Memory | Contract                                      |
 +----------+-------------+-----------------------------------------------+
-|          | Pointer     | do not modify until PerformPuts/EndStep/Close |
+|          | Pointer     | do not modify until PerformGets/EndStep/Close |
 | Deferred |             |                                               |
-|          | Contents    | populated at Put or PerformPuts/EndStep/Close |
+|          | Contents    | populated at Get or PerformGets/EndStep/Close |
 +----------+-------------+-----------------------------------------------+
-|          | Pointer     | modify after Put                              |
+|          | Pointer     | modify after Get                              |
 | Sync     |             |                                               |
-|          | Contents    | populated at Put                              |
+|          | Contents    | populated at Get                              |
 +----------+-------------+-----------------------------------------------+
 
 
@@ -452,8 +454,8 @@ Only use it if absolutely necessary (*e.g.* memory bound application or out of s
    ``Get`` doesn't support returning spans.
 
 
-PerformsGets
-------------
+PerformGets
+-----------
 
    Executes all pending ``Get`` calls in deferred mode.
 

docs/user_guide/source/components/operator.rst

@@ -2,6 +2,8 @@
 Operator
 ********
 
+.. _sec:basics_interface_components_operator:
+
 The Operator abstraction allows ADIOS2 to act upon the user application data, either from a ``adios2::Variable`` or a set of Variables in an ``adios2::IO`` object.
 Current supported operations are:
 

docs/user_guide/source/components/variable.rst

@@ -142,18 +142,17 @@ Dims value.
 For example, the definition below defines a 2-D Joined array where the
 first dimension is the one along which blocks will be joined and the
 2nd dimension is 5.  Here this rank is contributing two rows to this array.
-   .. code-block:: c++
-    auto var = outIO.DefineVariable<double>("table", {adios2::JoinedDim, 5},
-                                            {}, {2, 5});
+
+.. code-block:: c++
+
+  auto var = outIO.DefineVariable<double>("table", {adios2::JoinedDim, 5}, {}, {2, 5});
 
 If each of N writer ranks were to declare a variable like this and do
 a single Put() in a timestep, the reader-side GlobalArray would have
 shape {2*N, 5} and all normal reader-side GlobalArray operations would
 be applicable to it.  
 
 
-
-
 .. note::
 
    JoinedArrays are currently only supported by BP4 and BP5 engines,

docs/user_guide/source/ecosystem/h5vol/vol.rst

@@ -1,11 +1,12 @@
-***********
+**********
 Disclaimer
-***********
+**********
+
 .. note::
 
-The Virtual Object Layer (VOL) is a feature introduced in recent release of  HDF5 1.12 (https://hdf5.wiki/index.php/New_Features_in_HDF5_Release_1.12). 
+   The Virtual Object Layer (VOL) is a feature introduced in recent release of  HDF5 1.12 (https://hdf5.wiki/index.php/New_Features_in_HDF5_Release_1.12).
 
-So please do make sure your HDF5 version supports the latest VOL. 
+   So please do make sure your HDF5 version supports the latest VOL.
 
 Once the ADIOS VOL is compiled, There are two ways to apply it: 
 
@@ -20,8 +21,8 @@ External
 
 .. code-block:: c++
 
-  HDF5_VOL_CONNECTOR=ADIOS2_VOL
-  HDF5_PLUGIN_PATH=/replace/with/your/adios2_vol/lib/path/
+   HDF5_VOL_CONNECTOR=ADIOS2_VOL
+   HDF5_PLUGIN_PATH=/replace/with/your/adios2_vol/lib/path/
 
 
 Without code change, ADIOS2 VOL will be loaded at runtime by HDF5, to access ADIOS files without changing user code.
@@ -36,26 +37,22 @@ Internal
 
 .. code-block:: c++
 
-     // other includes 
-     #include <adios2/h5vol/H5Vol.h> // ADD THIS LINE TO INCLUDE ADIOS VOL
-
-     hid_t  pid = H5Pcreate(H5P_FILE_ACCESS);
-     // other declarations
-     hid_t fid = H5Fopen(filename, mode, pid);
-
-     H5VL_ADIOS2_set(pid); // ADD THIS LINE TO USE ADIOS VOL
+   // other includes
+   #include <adios2/h5vol/H5Vol.h> // ADD THIS LINE TO INCLUDE ADIOS VOL
 
-     H5Fclose(fid);
+   hid_t  pid = H5Pcreate(H5P_FILE_ACCESS);
+   // other declarations
+   hid_t fid = H5Fopen(filename, mode, pid);
 
-     H5VL_ADIOS2_unset();  // ADD THIS LINE TO EXIT ADIOS VOL
+   H5VL_ADIOS2_set(pid); // ADD THIS LINE TO USE ADIOS VOL
 
+   H5Fclose(fid);
 
+   H5VL_ADIOS2_unset();  // ADD THIS LINE TO EXIT ADIOS VOL
 
 ..  To choose what ADIOS2 Engine to use, set env variable: ADIOS2_ENGINE (default is BP5)
 
 
-
-
 **Note:** The following features are not supported in this VOL:
 
   * hyperslab support

docs/user_guide/source/engines/bp3.rst

@@ -1,6 +1,6 @@
-****
+***
 BP3 
-****
+***
 
 The BP3 Engine writes and reads files in ADIOS2 native binary-pack (bp) format. BP files are backwards compatible with ADIOS1.x and have the following structure given a "name" string passed as the first argument of ``IO::Open``:
 

docs/user_guide/source/engines/dataspaces.rst

@@ -1,6 +1,6 @@
-*********************************
+**********
 DataSpaces
-*********************************
+**********
 
 The DataSpaces engine for ADIOS2 is experimental. DataSpaces is an asynchronous I/O transfer method within ADIOS that enables 
 low-overhead, high-throughput data extraction from a running simulation. 

docs/user_guide/source/engines/engines.rst

@@ -23,6 +23,7 @@ Parameters are passed at:
 .. include:: sst.rst
 .. include:: ssc.rst
 .. include:: dataman.rst
+.. include:: dataspaces.rst
 .. include:: inline.rst
 .. include:: null.rst
 .. include:: plugin.rst

docs/user_guide/source/engines/hdf5.rst

@@ -9,9 +9,9 @@ or, set it in client code. For example, here is how to create a hdf5 reader:
 
 .. code-block:: c++
 
- adios2::IO h5IO = adios.DeclareIO("SomeName");
- h5IO.SetEngine("HDF5");
- adios2::Engine h5Reader = h5IO.Open(filename, adios2::Mode::Read);
+  adios2::IO h5IO = adios.DeclareIO("SomeName");
+  h5IO.SetEngine("HDF5");
+  adios2::Engine h5Reader = h5IO.Open(filename, adios2::Mode::Read);
 
 To read back the h5 files generated with VDS to ADIOS2, one can use the HDF5 engine. Please make sure you are using the HDF5 library that has version greater than or equal to 1.11 in ADIOS2.
 
@@ -33,10 +33,11 @@ After the subfile feature is introduced  in HDF5 version 1.14, the ADIOS2 HDF5 e
 
 To use the subfile feature, client needs to support MPI_Init_thread with MPI_THREAD_MULTIPLE. 
 
-Useful parameters from the  HDF lirbary to tune subfiles are:
+Useful parameters from the HDF library to tune subfiles are:
+
 .. code-block:: xml
 		
-H5FD_SUBFILING_IOC_PER_NODE (num of subfiles per node)
-    set H5FD_SUBFILING_IOC_PER_NODE to 0 if the regular h5 file is prefered, before using ADIOS2 HDF5 engine. 
-H5FD_SUBFILING_STRIPE_SIZE
-H5FD_IOC_THREAD_POOL_SIZE
+  H5FD_SUBFILING_IOC_PER_NODE (num of subfiles per node)
+    set H5FD_SUBFILING_IOC_PER_NODE to 0 if the regular h5 file is preferred, before using ADIOS2 HDF5 engine.
+  H5FD_SUBFILING_STRIPE_SIZE
+  H5FD_IOC_THREAD_POOL_SIZE

docs/user_guide/source/engines/sst.rst

@@ -280,24 +280,24 @@ single reader, but only upon request (with a request being initiated
 by the reader doing BeginStep()).  Normal reader-side rules (like
 BeginStep timeouts) and writer-side rules (like queue limit behavior) apply.
 
-============================= ===================== ====================================================
- **Key**                        **Value Format**      **Default** and Examples
-============================= ===================== ====================================================
- RendezvousReaderCount           integer             **1**
- RegistrationMethod              string              **File**, Screen
- QueueLimit                      integer             **0** (no queue limits)
- QueueFullPolicy                 string              **Block**, Discard
- ReserveQueueLimit               integer             **0** (no queue limits)
- DataTransport                   string              **default varies by platform**, UCX, MPI, RDMA, WAN
- WANDataTransport                string              **sockets**, enet, ib
- ControlTransport                string              **TCP**, Scalable
- MarshalMethod                   string              **BP5**, BP, FFS
- NetworkInterface                string              **NULL**
- ControlInterface                string              **NULL**
- DataInterface                   string              **NULL**
- FirstTimestepPrecious           boolean             **FALSE**, true, no, yes
- AlwaysProvideLatestTimestep     boolean             **FALSE**, true, no, yes
- OpenTimeoutSecs                 integer             **60**
- SpeculativePreloadMode          string              **AUTO**, ON, OFF
- SpecAutoNodeThreshold           integer             **1**
-============================= ===================== =====================================================
++-----------------------------+---------------------+----------------------------------------------------+
+| **Key**                     | **Value Format**    | **Default** and Examples                           |
++-----------------------------+---------------------+----------------------------------------------------+
+| RendezvousReaderCount       | integer             | **1**                                              |
+| RegistrationMethod          | string              | **File**, Screen                                   |
+| QueueLimit                  | integer             | **0** (no queue limits)                            |
+| QueueFullPolicy             | string              | **Block**, Discard                                 |
+| ReserveQueueLimit           | integer             | **0** (no queue limits)                            |
+| DataTransport               | string              | **default varies by platform**, UCX, MPI, RDMA, WAN|
+| WANDataTransport            | string              | **sockets**, enet, ib                              |
+| ControlTransport            | string              | **TCP**, Scalable                                  |
+| MarshalMethod               | string              | **BP5**, BP, FFS                                   |
+| NetworkInterface            | string              | **NULL**                                           |
+| ControlInterface            | string              | **NULL**                                           |
+| DataInterface               | string              | **NULL**                                           |
+| FirstTimestepPrecious       | boolean             | **FALSE**, true, no, yes                           |
+| AlwaysProvideLatestTimestep | boolean             | **FALSE**, true, no, yes                           |
+| OpenTimeoutSecs             | integer             | **60**                                             |
+| SpeculativePreloadMode      | string              | **AUTO**, ON, OFF                                  |
+| SpecAutoNodeThreshold       | integer             | **1**                                              |
++-----------------------------+---------------------+----------------------------------------------------+

docs/user_guide/source/index.rst

@@ -37,9 +37,17 @@ Funded by the `Exascale Computing Project (ECP) <https://www.exascaleproject.org
    advanced/aggregation
    advanced/memory_management
    advanced/gpu_aware
+   advanced/query
    advanced/plugins
    advanced/ecp_hardware
-
+
+.. toctree::
+   :caption: Tutorials
+
+   tutorials/overview
+   tutorials/downloadAndBuild
+   tutorials/basicTutorials
+
 .. toctree::
    :caption: Ecosystem Tools
 

docs/user_guide/source/operators/operators.rst

@@ -8,7 +8,7 @@ object.  Current supported operations are:
 
 1. Data compression/decompression, lossy and lossless.
 
-This section provides a description of the :ref:`Available Operators` in ADIOS2
+This section provides a description of the supported operators in ADIOS2
 and their specific parameters to allow extra-control from the user. Parameters
 are passed in key-value pairs for:
 

docs/user_guide/source/setting_up/source/cmake.rst

@@ -76,6 +76,8 @@ And finally, use the standard invocation to install:
 CMake Options
 *************
 
+.. _sec:source_cmake_options:
+
 The following options can be specified with CMake's ``-DVAR=VALUE`` syntax. The default option is highlighted.
 
 ============================= ================ ==========================================================================================================================================================================================================================