Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding documentation on how to build and run on top of EESSI without EB #175

Merged
merged 21 commits into from
Dec 12, 2024
Merged
Changes from 20 commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
c0a8f40
adding documentation on how to build and run on top of EESSI without EB
Apr 30, 2024
0b5b8d0
Update docs/using_eessi/building_on_eessi.md
hvelab May 27, 2024
b4819b8
Update docs/using_eessi/building_on_eessi.md
hvelab May 27, 2024
c99b797
Update docs/using_eessi/building_on_eessi.md
hvelab May 27, 2024
820c2d4
Update docs/using_eessi/building_on_eessi.md
hvelab May 27, 2024
1a66170
Update docs/using_eessi/building_on_eessi.md
hvelab May 27, 2024
d7edeaa
Update docs/using_eessi/building_on_eessi.md
hvelab May 27, 2024
99ba875
Update docs/using_eessi/building_on_eessi.md
hvelab May 27, 2024
dcb8508
Update docs/using_eessi/building_on_eessi.md
hvelab May 27, 2024
b06e3d5
Update docs/using_eessi/building_on_eessi.md
hvelab May 27, 2024
bd83794
Update docs/using_eessi/building_on_eessi.md
hvelab May 27, 2024
ed0c5dd
updating on LD_RUN_PATH
Jun 17, 2024
f1fd341
tab indent fix
hvelab Sep 30, 2024
f70fff4
Update docs/using_eessi/building_on_eessi.md
hvelab Oct 10, 2024
1b7f1eb
Update docs/using_eessi/building_on_eessi.md
hvelab Oct 10, 2024
6523acc
Update docs/using_eessi/building_on_eessi.md
hvelab Oct 10, 2024
6a98870
Update docs/using_eessi/building_on_eessi.md
hvelab Oct 10, 2024
08d9aa5
Update docs/using_eessi/building_on_eessi.md
hvelab Oct 10, 2024
3db1ac6
Update docs/using_eessi/building_on_eessi.md
hvelab Oct 10, 2024
d862469
Update docs/using_eessi/building_on_eessi.md
ocaisa Dec 12, 2024
fb8ef09
Merge branch 'main' into issue_48_build_without_eb
ocaisa Dec 12, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 34 additions & 3 deletions docs/using_eessi/building_on_eessi.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,38 @@ Finally, you should be able to load our newly build module:
module load netCDF/4.9.0-gompi-2022a
```

## Manually building software op top of EESSI
Building software on top of EESSI would require your linker to use the same system-dependencies as the software in EESSI does. In other words: it requires you to link against libraries from the compatibility layer, instead of from your host OS.
## Manually building software op top of EESSI (without EasyBuild)

hvelab marked this conversation as resolved.
Show resolved Hide resolved
!!! warning

We are working on a module file that should make building on top of EESSI (without using EasyBuild)
more straightforward, particularly when using `Autotools` or `CMake`. Right now, it is a little convoluted
and requires you to have a decent grasp of
* What a runtime dynamic linker (`ld-linux*.so`) is and does
* How to influence the behaviour of the runtime linker with `LD_LIBRARY_PATH`
* The difference between `LIBRARY_PATH` and `LD_LIBRARY_PATH`

As such, this documentation is intended for "experts" in the runtime linker and it's behaviour,
and most cases are untested. Any feedback on this topic is highly appreciated.

Building and running software on top of EESSI without EasyBuild is not straightforward and requires some considerations to take care of.

It is expected that you will have loaded all of your required dependencies as modules from the EESSI environment. Since EESSI sets `LIBRARY_PATH` for all of the modules and the `GCC` compiler is configured to use the compat layer, there should be no additional configuration required to execute a standard build process. On the other hand, EESSI does not set `LD_LIBRARY_PATH` so, _at runtime_, the executable will need help finding the libraries that it needs to actually execute. The easiest way to circumvent this requirement is by setting the environment variable `LD_RUN_PATH` during compile time as well. With `LD_RUN_PATH` set, the program will be able to tell the dynamic linker to search in those paths when the program is being executed.

EESSI uses a [compatibility layer](../compatibility_layer.md) to ensure that it takes as few libraries from the host as possible. The safest way to make sure all libraries will point to the required locations in the compatibility layer (and do not leak in from the host operating system) is starting an EESSI prefix shell before building. To do this:

* First of all, load the environment by starting an EESSI shell as described [here](https://www.eessi.io/docs/using_eessi/setting_up_environment).
ocaisa marked this conversation as resolved.
Show resolved Hide resolved
* Load all dependencies you need to build your software. You must use at least a toolchain from EESSI to compile it (`foss` is a good option as it will also include MPI with OpenMPI and math libraries via FlexiBLAS/FFTW).
* Set manually `LD_RUN_PATH` to resolve libraries at runtime. `LIBRARY_PATH` should contain all the paths we need, and we also need to include the path to `libstdc++` from our GCC installation to avoid picking up the one from the host:
```sh
export LD_RUN_PATH=$LIBRARY_PATH:$EBROOTGCCcore/lib64
```
* Compile and make sure the library resolution points to the EESSI stack. For this, `ldd` from compatibility layer and **not** `/usr/bin/ldd` should be used when checking the binary.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably best to provide a running example here so people have a clearer idea of what you are talking about

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about an MPI "Hello world" example?


* Run!
hvelab marked this conversation as resolved.
Show resolved Hide resolved


!!! Note RPATH should never point to a compatibility layer directory, only to software layer ones, as all resolving is done via the runtime linker (`ld-linux*.so`) that is shipped with EESSI, which automatically searches these locations.

The biggest downside of this approach is that your executable becomes bound to the architecture you linked your libraries for, i.e., if you add to your executable RPATH a `libhdf5.so`compiled for `intel_avx512`, you will not be able to run that binary on a machine with a different architecture. If this is an issue for you, you should look into how EESSI itself organises the location of binaries and perhaps leverage the relevant environment variables (e.g., `EESSI_SOFTWARE_SUBDIR`).

While we plan to support this in the future, manually building on top of EESSI is currently not supported yet in a trivial way.