Skip to content

Commit

Permalink
Additional documentation improvements.
Browse files Browse the repository at this point in the history
  • Loading branch information
NAThompson committed Jun 28, 2019
1 parent 2401101 commit d953778
Show file tree
Hide file tree
Showing 9 changed files with 116 additions and 134 deletions.
30 changes: 14 additions & 16 deletions docs/user_guide/source/installation/cpp_c.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2,26 +2,24 @@
Installing the ADIOS2 library and the C++ and C bindings
********************************************************

ADIOS2 by default will build and install the C++11 ``libadios2`` library and the C++11, C bindings.
By default, ADIOS2 will build the C++11 ``libadios2`` library and the C and C++ bindings.

1. **Minimum requirements:**
1. **Minimum requirements:**

* C++ compiler supporting C++11
* If MPI is enabled, a MPI C (not MPI C++ bindings) implementation must be installed.
* A C++11 compliant compiler
* An MPI C implementation on the syspath, or in a location identifiable by CMake.

2. **Linking** ``make install`` will generate the required header and libraries to link applications with ADIOS2 under the installation directory (from ``CMAKE_INSTALL_PREFIX``):
2. **Linking** ``make install`` will copy the required headers and libraries into the directory specified by ``CMAKE_INSTALL_PREFIX``:

* Libraries:

* Libraries:

- ``lib/libadios2.*`` C++11 and C bindings
* Headers:

* Headers:

- ``include/adios2.h`` C++11 ``namespace adios2``
- ``include/adios2_c.h`` C prefix ``adios2_``

* Config file: run this command to get installation info

- ``bin/adios2-config``



* Config file: run this command to get installation info

- ``bin/adios2-config``
12 changes: 6 additions & 6 deletions docs/user_guide/source/installation/examples.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,14 @@
Running Examples
****************

The best way to have a grasp for ADIOS2 usage is to follow the examples under ADIOS2/examples. They are self-describing so the user can modify these for their own applications.
ADIOS2 is best learned by `examples <https://github.com/ornladios/ADIOS2/tree/master/examples>`_.

An incremental way to look at examples:
A few very basic examples are described below:

================================ ==========================================================================================================================
Directory Description
Directory Description
================================ ==========================================================================================================================
``ADIOS2/examples/hello`` very basic "hello world" style examples for Writing and Reading.
``ADIOS2/examples/heatTransfer`` Poisson 2D solver for transient temperature in Fourier's heat transfer. Outputs ``bp.dir`` using ADIOS2, or HDF5.
``ADIOS2/examples/basics`` covers different Variable usage cases classified by the nature of their dimensions.
``ADIOS2/examples/hello`` very basic "hello world"-style examples for reading and writing `.bp` files.
``ADIOS2/examples/heatTransfer`` 2D Poisson solver for transients in Fourier's modeo of heat transfer. Outputs ``bp.dir`` or HDF5.
``ADIOS2/examples/basics`` covers different ``Variable`` use cases classified by the dimension.
================================ ==========================================================================================================================
14 changes: 6 additions & 8 deletions docs/user_guide/source/installation/fortran.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2,20 +2,18 @@
Enabling the Fortran bindings
*****************************

To enable the Fortran bindings in ADIOS2 follow these guidelines:
1. **Minimum requirements:**

1. **Minimum requirements:**
* A Fortran 90 compliant compiler
* A Fortran MPI implementation

* Compiler supporting Fortran 90 or more recent standards
* If MPI is enabled a MPI Fortran implementation must be installed.

2. **Linking the Fortran bindings:** ``make install`` will generate under the installation directory (from ``CMAKE_INSTALL_PREFIX``) the required library and modules to link ADIOS2 with Fortran applications:
2. **Linking the Fortran bindings:** ``make install`` will copy the required library and modules into the directory specified by ``CMAKE_INSTALL_PREFIX``

* Library (note that ``libadios2`` must also be linked)
- ``lib/libadios2_f.*``
- ``lib/libadios2.*``
* Modules

* Modules
- ``include/adios2/fortran/*.mod``

3. **Module adios2:** only module required to be used in an application ``use adios``
30 changes: 17 additions & 13 deletions docs/user_guide/source/installation/hpc_systems.rst
Original file line number Diff line number Diff line change
@@ -1,25 +1,29 @@
****************************************************
Building on High Performance Computing, HPC, Systems
****************************************************
.. _HPCBuild:

Please read this section before building ADIOS2 with CMake on a Supercomputer:
***********************
Building on HPC Systems
***********************

#. **Modules with minimum requirements:** Make sure the corresponding "module" dependencies are loaded and that minimum requirements are satisfied. Load the latest cmake module as many systems have older cmake versions as default. The same with your build environment, make sure compilers support C++11. This includes gcc >= 4.8.1, Intel >= 15, PGI >= 15, etc.
#. **Modules:** Make sure all "module" dependencies are loaded and that minimum requirements are satisfied.
Load the latest CMake module as many HPC systems default to an outdated version.
Build with a C++11-compliant compiler, such as gcc >= 4.8.1, Intel >= 15, and PGI >= 15.

#. **Optimization flags:** For actual performance tests and operations, it is recommended to set ``-DCMAKE_BUILD_TYPE=Release`` explicitly so CMake can turn on appropriate optimization flags.

#. **Static/Dynamic build:** On most Cray systems, like `Titan <https://www.olcf.ornl.gov/kb_articles/compiling-and-node-types/>`_, the default library link behavior is "static", thus CMake builds ADIOS2 statically (``libadios2.a``) by default. Read the system's documentation to enable dynamic compilation, usually by setting an environment variable such as ``CRAYPE_LINK_TYPE=dynamic``. Click `here <https://github.com/ornladios/ADIOS2/tree/master/scripts/runconf/runconf_olcf.sh>`_ for a full configurable script example on OLCF systems.
#. **Static/Dynamic build:** On Cray systems such as `Titan <https://www.olcf.ornl.gov/kb_articles/compiling-and-node-types/>`_,
the default behavior is static linkage, thus CMake builds ADIOS2 creates the static library ``libadios2.a`` by default.
Read the system documentation to enable dynamic compilation, usually by setting an environment variable such as ``CRAYPE_LINK_TYPE=dynamic``.
Click `here <https://github.com/ornladios/ADIOS2/tree/master/scripts/runconf/runconf_olcf.sh>`_ for a fully configurable script example on OLCF systems.

#. **Big Endian and 32-bit systems:** ADIOS2 hasn't been tested on big endian or 32-bit systems. Please be aware before attempting to run.

#. **System guidelines and policies:** Follow your system guidelines and policies for launching jobs, executing MPI communication, memory allocation, and I/O storage before attempting any type of unconventional usage of ADIOS2.

#. **CMake minimum version:** The ADIOS2 build system requires a minimum CMake version of 3.6.0, however, if using IBM XL, Cray, or PGI compilers, 3.9.0 or newer is required. ADIOS2 is using CMake's language level abstraction to instruct the compiler to build with C++11 support. This feature was available in CMake all the way back to 3.3, but with the feature only supporting GCC, Clang, and MSVC compilers. Langiuage level abstraction support was added for Intel compilers in CMake 3.6.0 and for all other compilers supproted by CMake (including XL, Cray, and PGI) in 3.9.0.
#. **CMake minimum version:** The ADIOS2 build system requires a minimum CMake version of 3.6.0. However, IBM XL, Cray, and PGI compilers require version 3.9.0 or newer.

#. **PGI compilers and C++11 support:** Recent versions of the PGI compiler >= v15 support the C++11 language standard, however they rely on the set of C++ standard library headers supplied by GCC. On most systems, the default configuration at the time compiler is instaled is to use the headers supplied by the system gcc in /usr. On many systems (Titan at OLCF, for example) even though the PGI compiler supports C++11 the configured GCC and it's headers do not (4.3.x on Cray Linux Environment v5 systems like Titan). To configure the PGI compiler to use a newer GCC, you will need to create a configuration file in your home directory that overrides the PGI compiler's default configuration. On Titan, the following steps will re-configure the PGI compiler to use GCC 6.3.0 provided by a module:
#. **PGI compilers and C++11 support:** Version 15 of the PGI compiler is C++11 compliant.
However it relies on the C++ standard library headers supplied by the system version of GCC, which may or may support all the C++11 features used in ADIOS2.
On many systems (Titan at OLCF, for example) even though the PGI compiler supports C++11, the configured GCC and its headers do not (4.3.x on Cray Linux Environment, and v5 systems like Titan).
To configure the PGI compiler to use a newer GCC, you must create a configuration file in your home directory that overrides the PGI compiler's default configuration.
On Titan, the following steps will re-configure the PGI compiler to use GCC 6.3.0 provided by a module:

.. code-block:: bash
$ module load gcc/6.3.0
$ makelocalrc $(dirname $(which pgc++)) -gcc $(which gcc) -gpp $(which g++) -g77 $(which gfortran) -o -net 1>${HOME}/.mypgirc 2>/dev/null
42 changes: 21 additions & 21 deletions docs/user_guide/source/installation/python.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,32 +4,32 @@ Enabling the Python bindings

To enable the Python bindings in ADIOS2, based on `PyBind11 <http://pybind11.readthedocs.io/en/stable/>`_, make sure to follow these guidelines:

1. **Dynamic library support:** The Python bindings in ADIOS2 require that your platform supports shared libraries, which for Cray systems (and some BlueGene systems) can require additional steps. See the previous section on :ref:`Building on High Performance Computing, HPC, Systems`. Note that you can still build the Python bindings with a static libadios2.a library, as long as your platform supports it.

2. **Minimum requirements:**
- **Minimum requirements:**

* Python 2.7 and above.
* Corresponding NumPy for the Python version used for the build
* If MPI is enabled, a compatible version of mpi4py needs to be installed.
* If Python versions 2.7 and 3 are installed in the system, CMake will autodetect Python 3 requirements as the default option. A Python executable can be passed explicitly to CMake builds using ``-DPYTHON_EXECUTABLE=...``
* `numpy`
* `mpi4py`

- **Running:** If CMake enables Python compilation, an ``adios2.so`` library containing the Python module is generated in the build directory under ``lib/pythonX.X/site-packages/``

* make sure your ``PYTHONPATH`` environment variable contains the path to ``adios2.so``.

* make sure the Python interpreter is compatible with the version used for compilation via ``python --version``.

* Run the Python tests with ``ctest -R Python``

3. **Running:** if CMake enables Python compilation, an ``adios2.so`` library containing the Python modules is generated in the build directory under ``lib/pythonX.X/site-packages/``
* Run `helloBPWriter.py <https://github.com/ornladios/ADIOS2/blob/master/examples/hello/bpWriter/helloBPWriter.py>`_ and `helloBPTimeWriter.py <https://github.com/ornladios/ADIOS2/blob/master/examples/hello/bpTimeWriter/helloBPTimeWriter.py>`_

* make sure your ``PYTHONPATH`` environment variable contains the path to the ``adios2.so`` module.

* make sure the Python interpreter is compatible with the version used for compilation.

* try running Python tests with: ``ctest -R Python``

* try running the `helloBPWriter.py <https://github.com/ornladios/ADIOS2/blob/master/examples/hello/bpWriter/helloBPWriter.py>`_ example under ADIOS2/examples/hello/bpWriter, and `helloBPTimeWriter.py <https://github.com/ornladios/ADIOS2/blob/master/examples/hello/bpTimeWriter/helloBPTimeWriter.py>`_ under ADIOS2/examples/hello/bpTimeWriter
* In Python 2.7:

.. code-block:: bash
Python 2.7:
$ mpirun -n 4 python helloBPWriter.py
$ mpirun -n 4 python helloBPWriter.py
$ python helloBPWriter.py
Python 3:
$ mpirun -n 4 python3 helloBPWriter.py
* In Python 3:

.. code-block:: bash
$ mpirun -n 4 python3 helloBPWriter.py
$ python3 helloBPWriter.py
25 changes: 13 additions & 12 deletions docs/user_guide/source/introduction/adaptable_io.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2,22 +2,23 @@
Adaptable IO beyond files in Scientific Data Lifecycles
*******************************************************

Exascale computing, Big Data, Internet of Things (IoT), Burst Buffers, High Bandwidth Memory (HBM), Remote Direct Memory Access (RDMA)...all these novel terms and technologies have one thing in common: Data Management at large scales has become more relevant when making informed decisions. The goal of ADIOS2 is to provide an **adaptable**, **scalable**, and **unified** framework to aid scientific applications in their data transfer needs going beyond file input/output (IO) storage. As shown in the figure, there are many data management challenges in the scientific data life cycle beyond file I/O that ADIOS2 aspires to address
Performant and usable tools for data management at scale are essential in an era where scientific breakthroughs are collaborative, multidisciplinary, and computational.
ADIOS2 is an *adaptable*, *scalable*, and *unified* framework to aid scientific applications when data transfer volumes exceed the capabilities of traditional file I/O.

.. image:: http://i65.tinypic.com/2h5k38w.png : alt: my-picture1
.. image:: http://i65.tinypic.com/2h5k38w.png : alt: my-picture1


In that regard, we are attempting to create a framework to provide:
ADIOS2 provides


* Custom application management of massive data sets from generation, analysis, movement, to short-term and long-term storage.

* Self-describing data in binary-packed (BP) format for quick information extraction
* Custom application management of massive data sets, starting from generation, analysis, and movement, as well as short-term and long-term storage.

* Ability to separate and extract relevant information from large data sets for better decision making

* Ability to make near real-time decisions based on in-transit or in-situ analytics
* Self-describing data in binary-packed (`.bp`) format for rapid metadata extraction

* Expand to other transport mechanisms with minimal overhead to the user: Wide-Area-Network (WAN), Remote Direct Memory Access (ibverbs, NVLink, etc.), Shared-Memory

* Exploit new memory hierarchy and scalability in novel hardware architectures: HBM, Burst Buffers, many-cores, etc.
* An ability to separate and extract relevant information from large data sets

* The capability to make real-time decisions based on in-transit or in-situ analytics

* The ability to expand to other transport mechanisms such wide area networks, remote direct memory access, and shared memory, with minimal overhead

* The ability to utilize the full capabilities of emergent hardware technologies, such as high-bandwidth memory and burst buffers
2 changes: 0 additions & 2 deletions docs/user_guide/source/introduction/introduction.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,6 @@
Introduction
############

You can find ADIOS2 source code on `github`_

.. _`github`: https://github.com/ornladios/ADIOS2

.. include:: nutshell.rst
Expand Down
Loading

0 comments on commit d953778

Please sign in to comment.