INSTALL

Installing
**********

PyHarm (the Python wrapper) can either be installed via "pip" or it
can be build from source.  Users having no or little experience with
compiling from the command line are advised to use "pip".  Advanced
users are recommended to build from source.  This gives you the
freedom to tailor PyHarm to your CPU which is likely to improve the
performance compared to the generic binaries installed by "pip".

CHarm (the C library) has to be build from source.

* Installation of PyHarm with pip

* Building from source



Installation of PyHarm with pip
*******************************

On Linux (x86_64), macOS (x86_64, ARM64) and Windows (x86_64), install
PyHarm using the command:

   pip install pyharm

This will install PyHarm and all the dependencies, most notably the C
library CHarm, which is internally called by PyHarm.

The CHarm binaries were compiled using the following settings.

* "gcc" compiler on Linux and macOS and "MSVC" on Windows.

* "--enable-double-precision" to compile in double precision.

* "--enable-openmp" to enable parallelization among CPU cores.

* On x86_64 CPUs, "--enable-avx" to enable SIMD parallelism using the
  AVX instruction sets of x86_64 CPUs.  AVX has been with us for more
  than 10 years now, so the binaries should run on any reasonably old
  CPU.  Should you get an error like "Illegal instruction", this means
  your CPU does not support AVX. In that case, you have to build
  PyHarm from source and avoid using any set of AVX instructions (see
  Building from source).

* On 64-bit ARM CPUs, "--enable-neon" to enable SIMD parallelization
  using the NEON instruction set.  NEON instructions should be
  supported by all ARM64-based MacBooks.

* "-O3 -ffast-math -Wall -Wpedantic" compiler flags on Linux and macOS
  and "/O2 /fp:fast /FS /GL" on Windows to optimize the library for
  best performance.  Importantly, you should be aware that "-ffast-
  math" and "/fp:fast" relax some rules on floating point arithmetics
  in favour of gaining some additional computing speed.  As a result,
  the accuracy may be slightly *worse* than when not using these
  flags.  Normally, however, one does not need to bother by this.
  Should you need compliance with the IEEE specifications, compile the
  library on your own without these flags.

* "--enable-python" to build the PyHarm wrapper.

Tip:

  All this information is also available directly from PyHarm by
  calling:

        >>> import pyharm as ph
        >>> ph.misc.print_info()

  This will print some useful compilation details about your
  particular PyHarm build.



Building from source
********************

This chapter will guide you through building from source on Unix-based
operating systems.  On other operating systems, the library should
compile, too, but a detailed recipe is missing for the time being.
Nonetheless, provided are some hints for Building from source on
Windows 10 and later.

Start by downloading the Source code from GitHub in any way you like
and then enter the main directory of CHarm.  For instance,

   git clone https://github.com/blazej-bucha/charm
   cd charm

The installation steps that follow depend on your operating system.

* Unix-based operating systems

* Windows 10 and later



Building from source on Unix-based operating systems
****************************************************

This section provides a guide through the build of CHarm and/or PyHarm
from source on Unix-based operating systems, particularly Linux and
macOS.  First discussed are the steps to compile CHarm (the C library)
and then follow instructions on how to (optionally) build PyHarm (the
Python wrapper).


CHarm (the C library)
=====================


Requirements
------------

* C compiler supporting C99.  GCC is recommended (available by default
  on most Linux distributions).  Other compilers that are known to
  successfully compile CHarm are listed in the Tested platforms
  chapter.

  *To compile in quadruple precision, GCC is mandatory (v4.6 or
  later)* and no other compiler is allowed.

* Make to build the library (available by default on most Linux
  distributions).

* pkg-config for managing library compile and link flags (available by
  default on most Linux distributions).

* FFTW for discrete fast Fourier transforms.


Installation on Linux
---------------------

At first, we will install FFTW and then we will proceed with the
installation of CHarm.


FFTW installation
~~~~~~~~~~~~~~~~~

FFTW can either be installed via you package manager or built from the
source.

* *Using your package manager*

  * Debian-based distributions:

       sudo apt install libfftw3-dev

  * Arch Linux: You may want to install, for instance, fftw-mpi using
    your package manager.

  After the installation, make sure the following libraries were
  successfully installed, depending on whether you want to use CHarm
  in single/double/quadruple precision and with OpenMP
  enabled/disabled:

  * "libfftw3f.so" for single precision version of CHarm with OpenMP
    disabled,

  * "libfftw3f_omp.so" for single precision version of CHarm with
    OpenMP enabled,

  * "libfftw3.so" for double precision version of CHarm with OpenMP
    disabled,

  * "libfftw3_omp.so" for double precision version of CHarm with
    OpenMP enabled,

  * "libfftw3q.so" for quadruple precision version of CHarm with
    OpenMP disabled,

  * "libfftw3q_omp.so" for quadruple precision version of CHarm with
    OpenMP enabled.

* *Compilation from source*

  Please read the manual on the homepage of FFTW on how to install the
  library. Example FFTW installation could be:

        ./configure --enable-openmp --enable-shared
        make
        make check
        sudo make install

  Do not forget to add the "--enable-shared" flag to compile FFTW also
  as a shared library (mandatory to install CHarm).  To install CHarm
  in single or quadruple precision, add also the "--enable-single" or
  "--enable-quad-precision" flag, respectively, when calling the
  "configure" script.  If you do not want to parallelize FFTW, you may
  omit the "--enable-openmp" flag.


Default CHarm installation
~~~~~~~~~~~~~~~~~~~~~~~~~~

If you

* want to install CHarm to "/usr/local",

* have installed FFTW (version "3.X.X") to the default path available
  to the compiler,

* want a double precision version of CHarm,

* do not want OpenMP parallelization,

* do not want to enable SIMD CPU instructions, and

* have root privileges,

you may simply execute the following commands:

   ./configure
   make
   make check
   sudo make install

Briefly, "./configure" checks the availability of all components
necessary to build CHarm and prepares makefiles and a few other files.
"make" compiles the library.  "make check" compiles and executes a
test program.  "make install" installs the library.


Customized CHarm installation
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The installation process can be tailored by appending one or more of
the following flags to the "./configure" call.

* "--enable-single-precision" or "--enable-double-precision" or "--
  enable-quad-precision" to compile CHarm in single, double or
  quadruple precision, respectively ("float", "double" and
  "__float128" data types for floating point numbers, respectively).
  If not specified, double precision is used as default.

* "--enable-openmp" to enable OpenMP parallelization (no
  parallelization by default).

  The number of threads can be set either in your code by
  "omp_set_num_threads(N)" or by using the "OMP_NUM_THREADS"
  environment variable.

* "--enable-avx" or "--enable-avx2" or "--enable-avx-512" or "--
  enable-neon" to enable AVX, AVX2, AVX-512 or NEON CPU instructions,
  respectively (all disabled by default).

  AVX, AVX2 and AVX-512 are SIMD instructions introduced by Intel in
  2011, 2013 and 2017, respectively, that are most likely available on
  any recent x86_64 CPUs.  The most critical number crunching parts of
  CHarm are hand-written to take advantage of these instructions in
  order to significantly improve the performance.  As a general rule,
  it is strongly recommended to enable the latest set of AVX
  instructions that are supported by your processor.  On many Linux
  distributions, you can find all the supported CPU instructions by
  executing "lscpu".

  NEON are SIMD instructions available on ARM CPUs.  64-bit ARMv8 CPUs
  or newer are required.  To enable NEON, you may need to manually
  specify proper compiler flags in addition to using the "--enable-
  neon" option.  The reason is that there is a large number of
  associated flags, so the "configure" script does not try to guess
  the proper one(s).  To specify the compiler flags, use the "CFLAGS"
  environment variable (see below; with the AVX family of CPUs, no
  additional compiler flags are needed).

  On the hardware level, SIMD instructions are not supported in
  quadruple precision, therefore they can be enabled only when
  compiling in single or double precision.

* "--prefix=/your/custom/path" to specify a custom installation path
  for CHarm (default is "--prefix=/usr/local").

* "LDFLAGS=-L/your/path/to/FFTW/lib" to specify a custom path to your
  FFTW libs (empty by default, that is, default is to assume that FFTW
  is accessible to the compiler).

  You only need to specify the path to the FFTW library; the lib files
  themselves are linked automatically.

* "CPPFLAGS=-I/your/path/to/FFTW/include" to specify a custom path to
  your FFTW header file (empty by default, that is, default is to
  assume that FFTW is accessible to the compiler).

* "--disable-shared" to not compile CHarm as a shared library.

* Other useful variables:

  * "CC" selects other than your system’s default C compiler, e.g.
    "CC=clang" for Clang, and

  * "CFLAGS" defines user-defined compiler flags, e.g.,  "CFLAGS="-O3
    -ffast-math"" (GCC).

* To get a summary of all the supported flags, execute "./configure
  --help".

An example installation

* with a custom CHarm installation directory,

* with a custom FFTW installation directory,

* in quadruple precision,

* with OpenMP parallelization enabled, and

* with SIMD instructions disabled

looks like:

   ./configure --prefix=/opt/charm --enable-openmp --enable-quad-precision \
        LDFLAGS=-L/opt/fftwq-3.3.9/lib \
        CPPFLAGS=-I/opt/fftwq-3.3.9/include
   make
   make check
   sudo make install


Installation on macOS
---------------------

At first, we will install FFTW and then we will proceed with the
installation of CHarm.


FFTW installation
~~~~~~~~~~~~~~~~~

FFTW can either be installed via you package manager or built from the
source, preferably with GCC.  The latter is strongly recommended on
macOS.

* *Using your package manager*

  You can use one of the following commands, depending on the package
  manager you use:

     sudo port install fftw-3
     brew install fftw

  This, however, most likely does not install FFTW in quadruple
  precision and/or with OpenMP support.  You may therefore be able to
  compile CHarm only in single or double precision with OpenMP
  disabled.

* *Compilation from source*

  It is recommended to compile FFTW using GCC.  If you do not have GCC
  installed yet, you may execute one of the following commands:

     sudo port install gcc10
     brew install gcc@10

  Now, you should be ready to build FFTW by following the instructions
  in the FFTW installation chapter (Linux, compilation from source).
  There is, however, one **important** additional remark.  When
  calling the FFTW’s "./configure" script, specify also your GCC
  compiler, including its version number, e.g.:

     ./configure --enable-openmp CC=gcc-10

  Without the "CC" flag, Clang will most likely be used which may
  cause an installation failure when using the "--enable-openmp"
  and/or "--enable-quad-precision" flag(s).  It may **not** be
  sufficient to add "CC=gcc" (GCC version number omitted), as this
  will still likely call Clang.


CHarm installation
~~~~~~~~~~~~~~~~~~

Having installed FFTW, you may proceed with the same instructions as
given in the Default CHarm installation and Customized CHarm
installation chapters for Linux.  Similarly as when installing FFTW,
it is recommended to use the GCC compiler via the "CC" variable when
calling the "./configure" script from the CHarm installation.


A few installation notes
------------------------

* The output lib names depend on the precision used to compile CHarm:

     * "libcharmf" – single precision,

     * "libcharm" – double precision,

     * "libcharmq" – quadruple precision.

* You may install CHarm in single, double and quadruple precision to
  the same installation path.  You don’t have to worry about
  overwriting the header and lib files.


Uninstallation
--------------

Execute "sudo make uninstall".


PyHarm (the Python wrapper)
===========================

Before reading this chapter, make sure you know how to compile CHarm.
Otherwise, you won’t be able to build PyHarm.


Requirements
------------

*Additional* prerequisites when compared with requirements:

* Python interpreter 3.6 or newer,

* Python module pip,

* Python module numpy (reasonably old version),

* Python module ctypes (reasonably old version).


Building PyHarm
---------------

Installation of PyHarm is disabled by default.  To enable it, you have
to add the "--enable-python" flag to the "configure" call *in
addition* to the flags discussed in the CHarm (the C library) chapter.

The following flags may be used in addition to "--enable-python".

* The "PYTHON" variable specifies the Python interpreter you want to
  use. For instance, "PYTHON=python3.9" will ensure that the build is
  done with/for Python version 3.9.  Use the appropriate version
  (depends on your machine).

* By default, PyHarm is built to the "${prefix}/lib" directory.  The
  path in "${prefix}" is taken from the "--prefix" flag (see CHarm
  (the C library)).  The default installation path can be replaced by
  a custom one using the "--with-python_prefix" flag, for instance, "
  --with-python_prefix=/home/isaac/pyharm".

  Using the correct path in "--with-python_prefix" is crucial for
  Python to find PyHarm.  Otherwise, when calling

     >>> import pyharm

  from within the Python shell or a Python script,
  "ModuleNotFoundError" will be thrown.

  There are several strategies to choose the installation path.

  * If you are not really confident with all this, create and activate
    a Python virtual environment:

       python3 -m venv /path/to/your/virtual/environment/
       source /path/to/your/virtual/environment/bin/activate

    Then use "--with-python_prefix=/path/to/your/virtual/environment"
    when calling the "configure" script.  After executing "make" and
    "make install", you are ready to import PyHarm in a Python shell
    or a Python script:

       >>> import pyharm

  * If you want to install PyHarm as a user, find the lib path of your
    Python user packages, for instance,

       python3 -m site --user-site

    The output might like "/home/isaac/.local/lib/python3.9/site-
    packages", depending on the version of your Python and on your OS.
    Based on this path, you can specify your installation path; in
    this case it is "--with-python_prefix=/home/isaac/.local".  Note
    that the "lib/python3.9/site-packages" directories have to be
    omitted, as they are added to the installation path automatically.

  * If you want to install PyHarm next to your system Python packages,
    you must specify neither "--prefix" nor "--with-python_prefix".

  Note that if you use a custom path in "--prefix" but do not specify
  "--with-python_prefix", you will most likely not be able to (easily)
  import PyHarm.


Example installation
~~~~~~~~~~~~~~~~~~~~

A typical installation of PyHarm to a Python virtual environment looks
like this:

   python3 -m venv /tmp/python-venv
   source /tmp/python-venv/bin/activate
   ./configure --prefix=/tmp/charm --enable-openmp \
      LDFLAGS=-L/opt/fftw-3.3.9/lib CPPFLAGS=-I/opt/fftw-3.3.9/include \
      --enable-python PYTHON=python3.9 --with-python_prefix=/tmp/python-venv
   make
   make check
   make install

Then, open Python:

   python3

From within Python, you can now work with PyHarm:

   >>> import pyharm as ph
   >>> ph.misc.print_info()
   >>> quit()

Deactivate the virtual environment from the shell:

   deactivate


Uninstallation
--------------

See Uninstallation.



Building from source on Windows 10 and later
********************************************

Perhaps the easiest way is to install the Windows Subsystem for Linux
(WSL).  WSL offers a fully functional Linux distribution that you can
run from within Windows 10 (no dual boot).  WSL is officially
supported by Microsoft and its installation is very easy.  Having
installed WSL, you may proceed with Building from source on Unix-based
operating systems.

Other solution is, for instance, Cygwin.  However, to properly compile
CHarm under Cygwin with SIMD instructions enabled ("--enable-avx", "--
enable-avx2" or "--enable-avx-512"), do *not* use GCC.  Under Cygwin,
GCC doesn’t seem to properly align the memory which is required to
execute SIMD CPU instructions.  You won’t get any compilation error,
but when running your program or the check program ("make check"), you
will most likely get a segfault error.  Clang, for instance, does its
job properly, even under Cygwin, so is recommended in this case.

The last path you can (but do not want to) take is to build the
library using the Microsoft’s MSVC compiler.  To make it work, install
some layer that will add the standard installation utilities (e.g.
make) that are not found on Windows normally.  Next, you may want to
use the CCCL wrapper for MSVC.  Finally, follow the instructions found
in Building from source on Unix-based operating systems.  In fact,
this is how we build the Python wheels for Windows that are available
from PyPI.