Linux (GCC Clang ICC)

This page contains instructions for compiling and using SG⁺⁺ with GCC, Clang or ICC under Linux. For brevity, we assume you want to compile with GCC; the compiler can be changed easily, however.

Dependencies

Required

The following software is required in order to build SG⁺⁺:

GCC (≥ 4.8) or Clang (≥ 3.8)
SCons (≥ 2.3) is required for building (build system). SCons is written in Python, which is therefore needed by SG⁺⁺ as well.
For Ubuntu ≥ 20.04 the python-is-python3 package is required for SCons to work.

Boost.Test for compiling and running the unit tests. You can also skip the unit tests, but this is not recommended.
SWIG (≥ 3.0.4), if you want to use SG⁺⁺ within Python, Java, or MATLAB. SWIG creates bindings from C/C++ functions in order to make them available as a Python module or prepare them to be called from Java code.
Python development headers are needed, if you want to compile the Python bindings.
NumPy for Python is needed, if you want to run the Python tests/examples.
Java Development Kit (JDK), if you want to compile the Java bindings.
Doxygen is required to create and compile the documentation, which you are reading right now. It is also required to automatically annotate the Python bindings generated by SWIG with docstrings.

Optional

The following software can be installed for full functionality:

dot in the Graphviz package is optional and generates inheritance diagrams in the Doxygen documentation.
MATLAB, if you want to use SG⁺⁺ within MATLAB.
GNU Scientific Library (GSL), is required for some functionality of the SG_DATADRIVEN module.
Eigen is required for some functionality of the SG_COMBIGRID module and recommended for the SG_OPTIMIZATION module.
UMFPACK (in SuiteSparse) is recommended for the SG_OPTIMIZATION module.
Armadillo is recommended for the SG_OPTIMIZATION module.
Gmm++ can be used in conjunction with the SG_OPTIMIZATION module.
DAKOTA is required for the Polynomial Chaos Expansion functionalities of the SG_COMBIGRID module. Installation instructions
CGAL is required for a new Sparse Grid Density Estimation of the SG_DATADRIVEN module. It is required to solve a quadratic optimization problem to guarantee positive function values at the grid points.
zlib is recommended for the SG_DATADRIVEN module.
Python development headers is required if you want to use the combigrid technique together with the data mining pipline (you need to compile the library with USE_PYTHON_EMBEDDING=1 in order to activate this feature)

Installation

On a recent Ubuntu system (≥ 14.04 LTS), you can install most dependencies by installing the following packages:

GCC: g++
Clang: clang on Ubuntu ≥ 16.04. On the LLVM download page, there are binaries for many other systems, including older Ubuntu versions.
SCons: scons
Boost.Test: libboost-test-dev
SWIG: swig on Ubuntu ≥ 14.10. For Ubuntu 14.04 LTS, download the SWIG 3.0 .deb file here.
Python development headers: python3-dev
NumPy: python3-numpy
Java: openjdk-7-jdk or openjdk-8-jdk on Ubuntu ≥ 15.10
Doxygen: doxygen
Dot: graphviz
GSL: libgsl-dev
Eigen: libeigen3-dev
UMFPACK: libsuitesparse-dev
Armadillo: libarmadillo-dev
Gmm++: libgmm++-dev
zlib: zlib1g-dev and zlib1g

Compilation with SCons

Compilation of the C++ libraries is done with SCons. Execute

scons -j <number of cores>

in the main folder to compile SG⁺⁺ with GCC. For configuration (including other compilers) and optimization, see below. If SCons does not seem to find external dependencies even if they are installed, you might want to clear the SCons cache before trying again:

rm -r .sconf_temp .sconsign.dblite

To obtain help on parameters for compilation, type

scons --help

After compilation, all unit-tests (located in the tests-folder of each module) are executed, if Boost.Test is installed. There are also some tests written in Python, but the majority is written with Boost.Test.

When the build is finished, the shared libraries are installed in lib. If you use it, add this directory to your LD_LIBRARY_PATH. Instructions are also displayed at the end of the build.

Configuration

SCons uses the file SConstruct. This file contains all information for compiling SGpp. If you just execute scons, the default compilation with GCC for SSE3 instruction set is selected. The compiler can be changed using COMPILER=clang or COMPILER=intel, e.g. To use other instruction sets, use ARCH=avx, etc. You are able to compile different SG⁺⁺ modules independently. However, you should take into account the dependencies between the modules to avoid "undefined symbol" errors: When using them, depending on the dependencies, other modules might have to be included, too. The currently available modules are (see themodules page):

SG_BASE: basic functionality
SG_DATADRIVEN: operations on data
SG_SOLVER: classes for solving the systems of equations
SG_PDE: partial differential equations
SG_FINANCE: financial module
SG_PARALLEL: classes for parallel computing
SG_COMBIGRID: combigrid classes
SG_OPTIMIZATION: optimization of objective functions

Also, there are two switches for supported high-level back-ends:

SG_PYTHON: Python bindings
SG_JAVA: Java bindings

For example, the command

scons SG_OPTIMIZATION=0

will compile all modules except optimization.

Additionally, you can pass some specific flags to the compiler using the CPPFLAGS environment variable:

scons CPPFLAGS='-g,-O0'

Python Bindings

The Python bindings are important, because some unit tests are written in Python. By default, the Python bindings are built, too. If not, then some prerequisites are missing (see Dependencies).

By default, the Python bindings will be annotated with Python docstrings, if Doxygen is installed. Disabling this feature, which is recommended if you have to recompile the whole codebase frequently, is done by setting PYDOC=0 in the SCons command line.

When the build is finished, the Python bindings are installed in lib/pysgpp. If you use them, add the lib directory to your PYTHONPATH. Alternatively, you can install the bindings into your local site-packages directory:

python setup.py install --user

Instructions are also displayed at the end of the build. In Python, you can import the library and print its contents via

import pysgpp
dir(pysgpp)

Java Bindings

By default, the Java bindings are built, too. If not, then the JDK is missing (see Dependencies).

Eclipse and SCons

Create a Makefile project and change the project properties as follows:

Properties → C/C++ Build → Builder Settings: Disable Use default build command and set Build command to scons.
Properties → C/C++ Build → Behaviour: Set Build (Incremental build) to, e.g., -j 2 and Clean to -c.

Using SG⁺⁺

In this section, we show how SG⁺⁺ can be used as a library in other programs. For C++, this includes compilation, linking, and execution of the program using SG⁺⁺. We also show how to use SG⁺⁺ from the other supported languages (Python, Java, and MATLAB). As an example application, we consider the base quick start tutorial tutorial.cpp; however, the instructions can be analogously applied to other programs.

Note that all examples, are automatically built after each SCons run. Therefore, the following steps are not necessary to compile the examples; rather, the intent is to show the steps to build an application using SG⁺⁺.

In the following, the current directory is always base/examples and /PATH_TO_SGPP refers to the absolute path of the SG⁺⁺ directory. We assume that SG⁺⁺ or its bindings have been successfully built before.

C++

First, compile the program while supplying the include paths of the relevant modules:

g++ tutorial.cpp \
-c -std=c++11 -fopenmp \
-I/PATH_TO_SGPP/base/src \
-o tutorial.o

Then, link the program by indicating the SG⁺⁺ library path and the modules you want to link against:

g++ tutorial.o \
-fopenmp \
-L/PATH_TO_SGPP/lib \
-lsgppbase \
-o tutorial

To run the program, note that you have to set the LD_LIBRARY_PATH environment variable to include the SG⁺⁺ library path:

export LD_LIBRARY_PATH="/PATH_TO_SGPP/lib:$LD_LIBRARY_PATH"
./tutorial

Python

The Python bindings pysgpp can be used either by setting the PYTHONPATH environment variable to include the lib directory, i.e.

export PYTHONPATH="/PATH_TO_SGPP/lib:$PYTHONPATH"

or by installing pysgpp in the local site-packages folder:

python setup.py install --user

To run your Python program, don't forget to update the LD_LIBRARY_PATH environment variable in any case:

export LD_LIBRARY_PATH="/PATH_TO_SGPP/lib:$LD_LIBRARY_PATH"
python tutorial.py

Java

Java programs using the Java bindings jsgpp have to be compiled in this way:

javac -cp .:/PATH_TO_SGPP/lib/jsgpp/jsgpp.jar tutorial.java

When running Java programs, you have to augment LD_LIBRARY_PATH not only by the SG⁺⁺ library path, but also by a path specific for jsgpp:

export LD_LIBRARY_PATH="/PATH_TO_SGPP/lib:/PATH_TO_SGPP/lib/jsgpp:$LD_LIBRARY_PATH"
java -cp .:/PATH_TO_SGPP/lib/jsgpp/jsgpp.jar tutorial

MATLAB

MATLAB can use SG⁺⁺ in three ways.

Via Binaries

The recommended way is to download the binaries for use with MATLAB that are part of the SGpp releases For instructions, please see the matlab page.

Via MEX Interface

If the binaries don't work for you, then it is possible to write and compile a MEX interface yourself (similar to the interface that the binaries would provide). This means that you have to write a C++ program interacting with SG⁺⁺ directly in C++ and converting input and output arguments from and to MATLAB's data structures. The parameters to be passed to MATLAB's 'mex' function which compiles the program are largely the same as for plain C++. Alternatively, you can link and compile by yourself:

g++ your_mex_program.cpp \
-c -std=c++11 -fopenmp -fPIC \
-Wall -Wextra \
-I/PATH_TO_MATLAB/extern/include \
-I/PATH_TO_SGPP/base/src \
-o your_mex_program.o
g++ your_mex_program.o \
-shared -fopenmp \
-L/PATH_TO_MATLAB/bin/glnxa64 \
-L/PATH_TO_SGPP/lib \
-lmex \
-lsgppbase \
-o your_mex_program.mexa64

Of course, you have to add include paths and library switches for each module that 'your_mex_program' uses.

Via Mex Interface created by SWIG

This is a short explanation of how to use a fork of SWIG to directly generate a MATLAB interface (via MEX files) for SG++ without Java.

The creator of the SWIG fork has put some info to https://github.com/RobotLocomotion/drake/issues/1267. He forked SWIG to provide bindings to his own library (see https://github.com/casadi/casadi/wiki/matlab). There is an outdated version of the fork in the SWIG repository (https://github.com/swig/swig/tree/matlab). However, it doesn't seem to have been merged yet. Therefore, you have to compile the fork yourself. See http://www.swig.org/svn.html for generic instructions and http://www.swig.org/Doc2.0/Windows.html#Windows_swig_exe for compiling on Windows.

Instructions

Install MATLAB.
Clone the SWIG fork from https://github.com/jaeandersson/swig and change to the extracted directory.
Apply the patch below to the source of the SWIG fork. The first hunk fixes a problem that occurs when calling an overridden method that has one or more arguments. The check is not correctly done in this case. The second hunk fixes the problem that inheritance is not done correctly. Otherwise, the MATLAB correspondents of subclasses don't inherit from their superclasses.
If you don't have PCRE installed, you have to download the PCRE source tarball, place it in the root folder of the SWIG fork, and run Tools/pcre-build.sh.
Run ./autogen.sh.
Run ./configure.
Run make.
The executable of the SWIG fork is now located at ./swig.
Clone SG++ and change to the directory of the clone.
Compile SG++ with PATH=/DIRECTORY_OF_SWIG_FORK:$PATH SWIG_LIB=/DIRECTORY_OF_SWIG_FORK/Lib scons -j 4 SG_ALL=0 SG_BASE=1 SG_MATLAB=1 MATLAB_INCLUDE_PATH=/PATH_TO_MATLAB/extern/include MATLAB_LIBRARY_PATH=/PATH_TO_MATLAB/bin/glnxa64 (for Linux, adapt the last path for other platforms).
The compiled MATLAB interface can be found at lib/matsgpp.
export LD_LIBRARY_PATH="/PATH_TO_SGPP/lib/matsgpp/:$LD_LIBRARY_PATH"
export LD_PRELOAD="/usr/lib/x86_64-linux-gnu/libstdc++.so.6:$LD_PRELOAD"
Open MATLAB
Use pathtool to add /PATH_TO_SGPP/matsgpp/matlab and /PATH_TO_SGPP/matsgpp/matlab
Test with "sgpp.createLinearGrid(2)"

Additional Instructions for Windows

On Windows, you have to install MSYS first:

Install MinGW Installation Manager (mingw-get)
Install msys-base and mingw-developer-tools (inside mingw-get)
Follow the instructions at http://swig.org/Doc2.0/Windows.html#Windows_swig_exe
1. Download PCRE source tarball from their website http://www.pcre.org/
2. Extract SWIG-matlab ZIP (from step 1) to msys/1.0 directory
3. Start MSYS shell, change to extracted directory
4. Run Tools/pcre-build.sh
5. Run ./autogen.sh, ./configure, make
6. Move extracted directory (now contains swig.exe) to desired location

Patch

diff --git a/Source/Modules/matlab.cxx b/Source/Modules/matlab.cxx
index 11c6fb9..d2f8418 100644
--- a/Source/Modules/matlab.cxx
+++ b/Source/Modules/matlab.cxx
@@ -168,7 +168,7 @@ MATLAB::MATLAB():
   /* Add code to manage protected constructors and directors */
   director_prot_ctor_code = NewString("");
   Printv(director_prot_ctor_code,
-	 "if ( argc==1 ) { /* subclassed */\n",
+	 "if ( $comparison ) { /* subclassed */\n",
 	 "  $director_new \n",
 	 "} else {\n", "   mexErrMsgIdAndTxt(\"SWIG:RuntimeError\",\"accessing abstract class or protected constructor\"); \n", "  SWIG_fail;\n", "}\n", NIL);
 
@@ -2111,6 +2111,8 @@ int MATLAB::classHandler(Node *n) {
       if (bmodoptions) {
           bpkg = Getattr(bmodoptions, "package");
       }
+      if (!bpkg)
+          bpkg = pkg_name;
       if (!bname || !bpkg || GetFlag(b.item, "feature:ignore"))
 	continue;
       base_count++;

Via jsgpp

The third way of using SG⁺⁺ from within MATLAB is jsgpp, i.e., using the Java library of SG⁺⁺ and import it to MATLAB. Before we can use these methods in MATLAB, we have to add /PATH_TO_SGPP/lib/jsgpp to the librarypath.txt file of MATLAB. (Hint: Typing 'matlabroot' in MATLAB returns the path of your MATLAB installation.) Open the file /PATH_TO_MATLAB/toolbox/local/librarypath.txt in a text editor and add the line

/PATH_TO_SGPP/lib/jsgpp

at the end of the file. Now we can start MATLAB. However, we have to set the environment variables LD_LIBRARY_PATH and LD_PRELOAD before:

export LD_LIBRARY_PATH="/PATH_TO_SGPP/lib:/PATH_TO_SGPP/lib/jsgpp:$LD_LIBRARY_PATH"
export LD_PRELOAD="/usr/lib/x86_64-linux-gnu/libstdc++.so.6:$LD_PRELOAD"
matlab

The variable LD_LIBRARY_PATH has to be set to allow MATLAB to find the shared libraries (see above). However, this does not suffice: MATLAB ships its own version of libstdc++ (usually in /PATH_TO_MATLAB/sys/os/glnxa64) and prepends its location to MATLAB's internal LD_LIBRARY_PATH. Since this version of libstdc++ is most likely incompatible with the version used to compile SG⁺⁺, you will get errors like this when executing SG⁺⁺ programs in MATLAB:

Error using tutorial (line 5)
Java exception occurred:
java.lang.UnsatisfiedLinkError: /PATH_TO_SGPP/lib/jsgpp/libjsgpp.so:
PATH_TO_MATLAB/bin/glnxa64/libstdc++.so.6: version
`GLIBCXX_3.4.20' not found (required by /PATH_TO_SGPP/lib/jsgpp/libjsgpp.so)
at java.lang.ClassLoader$NativeLibrary.load(Native Method)
at java.lang.ClassLoader.loadLibrary1(Unknown Source)
at java.lang.ClassLoader.loadLibrary0(Unknown Source)
at java.lang.ClassLoader.loadLibrary(Unknown Source)
at java.lang.Runtime.loadLibrary0(Unknown Source)
at java.lang.System.loadLibrary(Unknown Source)
at sgpp.LoadJSGPPLib.loadJSGPPLib(LoadJSGPPLib.java:10)

To work around this issue, set the LD_PRELOAD variable to load the correct version libstdc++ before calling MATLAB. You can find the correct version by examining the output of

ldd /PATH_TO_SGPP/lib/jsgpp/libjsgpp.so

and searching for a line containing libstdc++. On 64-bit Ubuntu systems, it can be located in /usr/lib/x86_64-linux-gnu/libstdc++.so.6, but it can differ on other systems. When starting MATLAB, make sure that the Java version of MATLAB's internal Java Runtime Environment (check with version -java in MATLAB) is newer than that of the Java you built jsgpp with (check with java -version in a terminal). Sometimes, you have to force MATLAB to use your installed JDK by using the MATLAB_JAVA environment variable:

export MATLAB_JAVA="/PATH_TO_JDK/jre"

(e.g., MATLAB_JAVA="/usr/lib/jvm/java-7-openjdk-amd64/jre"). You can get errors like this otherwise:

Undefined variable "sgpp" or class "sgpp.LoadJSGPPLib.loadJSGPPLib".

Keep in mind that, however, MATLAB seems not to be compatible with Java 8 yet. After starting MATLAB, we have to add the jsgpp.jar file to MATLAB's class path with the command

javaaddpath('/PATH_TO_SGPP/lib/jsgpp/jsgpp.jar');

The final step consists in loading the jsgpp library via

sgpp.LoadJSGPPLib.loadJSGPPLib();

You should now be able to use SG⁺⁺ in MATLAB. For an example, look at base/examples/tutorial.m. However, note that the example was written for the MATLAB SG⁺⁺ binaries explained above. Therefore, you likely have to change some calls especially to static methods like sgpp.createOperationEval, which are now located at sgpp.jsgpp.*.

Hints

Use

javaclasspath();

to see the loaded jar files (upper part static, lower part dynamic; our jsgpp.jar should be in the latter part).

Write

import sgpp.*

in MATLAB to not have to write sgpp. in front of every method.

Call

methods('sgpp.Classname')

to see all methods of the class "Classname" (e.g. Grid).

The methods itself can also be called like Java methods in the MATLAB command window, e.g.:

dataVector = sgpp.DataVector(10)
dataVector.setAll(0)
dataVector

Provide feedback

Saved searches

Use saved searches to filter your results more quickly