Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Syncing code-base and CVMFS repository with EESSI (January 2024) #223

Open
69 of 74 tasks
TopRichard opened this issue Jan 17, 2024 · 2 comments
Open
69 of 74 tasks

Syncing code-base and CVMFS repository with EESSI (January 2024) #223

TopRichard opened this issue Jan 17, 2024 · 2 comments

Comments

@TopRichard
Copy link

TopRichard commented Jan 17, 2024

NESSI and EESSI have diverted, so in the attempt of syncing them back as close as possible, the following PRs have been merged:

NOTE: CUDA/NVIDIA MODIFICATIONS ARE NOT TESTED!!!

Test PRs

For now (EESSI Merged PR#454) there exist differences within 74 files as listed below, must carry out resolving those differences carefully:

  • .github/workflows/test-software.eessi.io.yml
  • .github/workflows/test_eessi_container_script.yml
  • .github/workflows/test_licenses.yml
  • .github/workflows/tests_archdetect.yml
  • .github/workflows/tests_scripts.yml
  • EESSI-install-software.sh
  • README.md --> differences are because of NESSI vs EESSI
  • bot/build.sh
  • bot/check-result.sh
  • bot/check-test.sh
  • bot/test.sh
  • bot/inspect.sh
  • build_container.sh
  • check_missing_installations.sh --> differences are because of NESSI vs EESSI
  • configure_easybuild --> addressed in PR#225, requires rebuild of gnuplot as a test(after the removal of Lua)which we already don't have
  • create_directory_tarballs.sh --> addressed in PR#225
  • create_lmodrc.py --> addressed in PR#232 requires some naming changes but will carry that on while testing CUDA modifications
  • create_tarball.sh --> addressed in PR#225
  • easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.7.2-2021a.yml --> moved to easystacks/pilot.nessi.no/2023.06
  • easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.7.2-2021b.yml --> moved to easystacks/pilot.nessi.no/2023.06
  • easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.7.2-2022a.yml --> moved to easystacks/pilot.nessi.no/2023.06
  • easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.7.2-2022b.yml --> moved to easystacks/pilot.nessi.no/2023.06
  • easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.7.2-system.yml --> moved to easystacks/pilot.nessi.no/2023.06
  • easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.8.0-system.yml --> moved to easystacks/pilot.nessi.no/2023.06
  • easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.8.1-2022a.yml --> moved to easystacks/pilot.nessi.no/2023.06
  • easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.8.2-2022a.yml --> moved to easystacks/pilot.nessi.no/2023.06
  • easystacks/software.eessi.io/2023.06/README.md
  • easystacks/software.eessi.io/2023.06/eessi-2023.06-eb-4.8.2-001-system.yml --> moved here
  • (LATER) easystacks/software.eessi.io/2023.06/eessi-2023.06-eb-4.8.2-010-CUDA.yml --> Will be handled later
  • easystacks/software.eessi.io/2023.06/eessi-2023.06-eb-4.8.2-2022b.yml --> moved here
  • easystacks/software.eessi.io/2023.06/eessi-2023.06-eb-4.8.2-2023a.yml --> moved here
  • (LATER) easystacks/software.eessi.io/2023.06/eessi-2023.06-eb-4.9.0-001-system.yml --> Will be handled later
  • (LATER) easystacks/software.eessi.io/2023.06/eessi-2023.06-eb-4.9.0-2022b.yml --> Will be handled later
  • (LATER) easystacks/software.eessi.io/2023.06/eessi-2023.06-eb-4.9.0-2023a.yml --> Will be handled later
  • (LATER) easystacks/software.eessi.io/2023.06/eessi-2023.06-eb-4.9.0-2023b.yml --> Will be handled later
  • eb_hooks.py --> addressed in PR#232
  • eessi-2021.06.yml
  • eessi-2021.12.yml
  • eessi_container.sh --> addressed in PR#225, also addresses
    some GPU modifications, check the note in the PR
  • init/Magic_Castle/bash --> addressed in PR#222
  • init/Magic_Castle/eessi_python3 --> addressed in PR#222
  • init/README.md --> addressed in PR#222
  • init/arch_specs/eessi_arch_arm.spec --> addressed in PR#224
  • init/bash --> addressed in PR#224
  • init/eessi_archdetect.sh --> addressed in PR#224
  • init/eessi_defaults --> addressed in PR#218
    difference in EESSI_CVMFS_REPO_OVERRIDE and using archspec instead of archdetect
  • init/eessi_environment_variables --> addressed in PR#224
  • init/eessi_software_subdir_for_host.py --> addressed in PR#222
  • init/minimal_eessi_env --> addressed in PR#222
  • install_scripts.sh --> addressed in PR#232
  • licenses/README.md --> addressed in PR#231
  • licenses/licenses.json --> addressed in PR#231
  • licenses/spdx.py --> addressed in PR#231
  • load_easybuild_module.sh --> addressed in PR#232
  • scripts/gpu_support/nvidia/install_cuda_host_injections.sh --> addressed in PR#232
  • scripts/gpu_support/nvidia/link_nvidia_host_libraries.sh --> addressed in PR#232
  • scripts/utils.sh --> addressed in PR#232
  • tests/archdetect/aarch64/arm/neoverse-n1/AWS-awslinux-graviton2.output --> addressed in PR#231
  • tests/archdetect/aarch64/arm/neoverse-n1/Azure-Ubuntu20-Altra.output --> addressed in PR#231
  • tests/archdetect/aarch64/arm/neoverse-v1/AWS-awslinux-graviton3.output --> addressed in PR#231
  • tests/archdetect/aarch64/neoverse_n1/AWS-awslinux-graviton2.all.output --> addressed in PR#231
  • tests/archdetect/aarch64/neoverse_n1/AWS-awslinux-graviton2.cpuinfo --> addressed in PR#231
  • tests/archdetect/aarch64/neoverse_n1/AWS-awslinux-graviton2.output --> addressed in PR#231
  • tests/archdetect/aarch64/neoverse_n1/Azure-Ubuntu20-Altra.all.output --> addressed in PR#231
  • tests/archdetect/aarch64/neoverse_n1/Azure-Ubuntu20-Altra.cpuinfo --> addressed in PR#231
  • tests/archdetect/aarch64/neoverse_n1/Azure-Ubuntu20-Altra.output --> addressed in PR#231
  • tests/archdetect/aarch64/neoverse_v1/AWS-awslinux-graviton3.all.output --> addressed in PR#231
  • tests/archdetect/aarch64/neoverse_v1/AWS-awslinux-graviton3.cpuinfo --> addressed in PR#231
  • tests/archdetect/aarch64/neoverse_v1/AWS-awslinux-graviton3.output --> addressed in PR#231
  • tests/archdetect/ppc64le/power9le/unknown-power9le.all.output --> addressed in PR#231
  • tests/archdetect/x86_64/amd/zen2/Azure-CentOS7-7V12.all.output --> addressed in PR#231
  • tests/archdetect/x86_64/amd/zen3/Azure-CentOS7-7V73X.all.output --> addressed in PR#231
  • tests/archdetect/x86_64/intel/haswell/archspec-linux-E5-2680-v3.all.output --> addressed in PR#231
  • tests/archdetect/x86_64/intel/skylake_avx512/archspec-linux-6132.all.output --> addressed in PR#231
@trz42
Copy link
Collaborator

trz42 commented Jan 21, 2024

First complete pass (except for some easystack files) is done. Remaining todos:

  • do second pass on current differences

    git diff nessi_2023.06 2023.06-software.eessi.io
    • .github/workflows/test-pilot.nessi.no.yml
      only differences due to repo naming, repo config, supported CPU architectures --> ✅

      diff
      diff --git a/.github/workflows/test-pilot.nessi.no.yml b/.github/workflows/test-software.eessi.io.yml
      similarity index 76%
      rename from .github/workflows/test-pilot.nessi.no.yml
      rename to .github/workflows/test-software.eessi.io.yml
      index 6342d5d..3ac341a 100644
      --- a/.github/workflows/test-pilot.nessi.no.yml
      +++ b/.github/workflows/test-software.eessi.io.yml
      @@ -1,5 +1,5 @@
       # documentation: https://help.github.com/en/articles/workflow-syntax-for-github-actions
      -name: Check for missing software installations in pilot.nessi.no
      +name: Check for missing software installations in software.eessi.io
       on: [push, pull_request, workflow_dispatch]
       permissions:
         contents: read # to fetch code (actions/checkout)
      @@ -12,37 +12,40 @@ jobs:
               EESSI_VERSION:
               - 2023.06
               EESSI_SOFTWARE_SUBDIR_OVERRIDE:
      -#        - aarch64/generic
      +        - aarch64/generic
      +        - aarch64/neoverse_n1
      +        - aarch64/neoverse_v1
               - x86_64/amd/zen2
      -        - x86_64/intel/broadwell
      +        - x86_64/amd/zen3
      +        - x86_64/intel/haswell
               - x86_64/intel/skylake_avx512
               - x86_64/generic
           steps:
               - name: Check out software-layer repository
                 uses: actions/checkout@93ea575cb5d8a053eaa0ac8fa3b40d7e05a33cc8 # v3.1.0
      
      -        - name: Mount NESSI CernVM-FS repository
      +        - name: Mount EESSI CernVM-FS pilot repository
                 uses: cvmfs-contrib/github-action-cvmfs@d4641d0d591c9a5c3be23835ced2fb648b44c04b # v3.1
                 with:
      -              cvmfs_config_package: https://github.com/NorESSI/filesystem-layer/releases/download/latest/cvmfs-config-nessi_latest_all.deb
      +              cvmfs_config_package: https://github.com/EESSI/filesystem-layer/releases/download/latest/cvmfs-config-eessi_latest_all.deb
                     cvmfs_http_proxy: DIRECT
      -              cvmfs_repositories: pilot.nessi.no
      +              cvmfs_repositories: software.eessi.io
      
               - name: Test check_missing_installations.sh script
                 run: |
                     export EESSI_SOFTWARE_SUBDIR_OVERRIDE=${{matrix.EESSI_SOFTWARE_SUBDIR_OVERRIDE}}
      -              source /cvmfs/pilot.nessi.no/versions/${{matrix.EESSI_VERSION}}/init/bash
      +              source /cvmfs/software.eessi.io/versions/${{matrix.EESSI_VERSION}}/init/bash
                     # set $EESSI_CPU_FAMILY to the CPU architecture that corresponds to $EESSI_SOFTWARE_SUBDIR_OVERRIDE (part before the first slash),
                     # to prevent issues with checks in the Easybuild configuration that use this variable
                     export EESSI_CPU_FAMILY=${EESSI_SOFTWARE_SUBDIR_OVERRIDE%%/*}
                     module load EasyBuild
                     which eb
                     eb --version
      -              export EESSI_PREFIX=/cvmfs/pilot.nessi.no/versions/${{matrix.EESSI_VERSION}}
      +              export EESSI_PREFIX=/cvmfs/software.eessi.io/versions/${{matrix.EESSI_VERSION}}
                     export EESSI_OS_TYPE=linux
                     env | grep ^EESSI | sort
      -              echo "just run check_missing_installations.sh (should use easystacks/pilot.nessi.no/${{matrix.EESSI_VERSION}}/eessi-${{matrix.EESSI_VERSION}}-*.yml)"
      -              for easystack_file in $(ls easystacks/pilot.nessi.no/${{matrix.EESSI_VERSION}}/eessi-${{matrix.EESSI_VERSION}}-eb-*.yml); do
      +              echo "just run check_missing_installations.sh (should use easystacks/software.eessi.io/${{matrix.EESSI_VERSION}}/eessi-${{matrix.EESSI_VERSION}}-*.yml)"
      +              for easystack_file in $(ls easystacks/software.eessi.io/${{matrix.EESSI_VERSION}}/eessi-${{matrix.EESSI_VERSION}}-eb-*.yml); do
                         echo "check missing installations for ${easystack_file}..."
                         ./check_missing_installations.sh ${easystack_file}
                         ec=$?
      @@ -52,14 +55,14 @@ jobs:
               - name: Test check_missing_installations.sh with missing package (GCC/8.3.0)
                 run: |
                     export EESSI_SOFTWARE_SUBDIR_OVERRIDE=${{matrix.EESSI_SOFTWARE_SUBDIR_OVERRIDE}}
      -              source /cvmfs/pilot.nessi.no/versions/${{matrix.EESSI_VERSION}}/init/bash
      +              source /cvmfs/software.eessi.io/versions/${{matrix.EESSI_VERSION}}/init/bash
                     # set $EESSI_CPU_FAMILY to the CPU architecture that corresponds to $EESSI_SOFTWARE_SUBDIR_OVERRIDE (part before the first slash),
                     # to prevent issues with checks in the Easybuild configuration that use this variable
                     export EESSI_CPU_FAMILY=${EESSI_SOFTWARE_SUBDIR_OVERRIDE%%/*}
                     module load EasyBuild
                     which eb
                     eb --version
      -              export EESSI_PREFIX=/cvmfs/pilot.nessi.no/versions/${{matrix.EESSI_VERSION}}
      +              export EESSI_PREFIX=/cvmfs/software.eessi.io/versions/${{matrix.EESSI_VERSION}}
                     export EESSI_OS_TYPE=linux
                     env | grep ^EESSI | sort
                     # create dummy easystack file with a single entry (something that is not installed in EESSI)
    • .github/workflows/test_eessi_container_script.yml

      diff
      diff --git a/.github/workflows/test_eessi_container_script.yml b/.github/workflows/test_eessi_container_script.yml
      index 3bb67b4..33122e6 100644
      --- a/.github/workflows/test_eessi_container_script.yml
      +++ b/.github/workflows/test_eessi_container_script.yml
      @@ -90,15 +90,15 @@ jobs:
                     elif [[ ${{matrix.SCRIPT_TEST}} == 'readwrite' ]]; then
                       outfile=out_readwrite.txt
                       fn="test_${RANDOM}.txt"
      -                echo "touch /cvmfs/pilot.nessi.no/${fn}" > test_script.sh
      +                echo "touch /cvmfs/software.eessi.io/${fn}" > test_script.sh
                       chmod u+x test_script.sh
                       export SINGULARITY_BIND="$PWD:/test"
                       ./eessi_container.sh --verbose --access rw --mode run /test/test_script.sh > ${outfile}
      
                       tmpdir=$(grep "\-\-resume" ${outfile} | sed "s/.*--resume \([^']*\).*/\1/g")
                       # note: must use '--access rw' again here, since touched file is in overlay upper dir
      -                ./eessi_container.sh --verbose --resume ${tmpdir} --access rw --mode shell <<< "ls -l /cvmfs/pilot.nessi.no/${fn}" > ${outfile}
      -                grep "/cvmfs/pilot.nessi.no/${fn}$" $outfile
      +                ./eessi_container.sh --verbose --resume ${tmpdir} --access rw --mode shell <<< "ls -l /cvmfs/software.eessi.io/${fn}" > ${outfile}
      +                grep "/cvmfs/software.eessi.io/${fn}$" $outfile
      
                     # test use of --resume
                     elif [[ ${{matrix.SCRIPT_TEST}} == 'resume' ]]; then
      @@ -120,12 +120,12 @@ jobs:
                     elif [[ ${{matrix.SCRIPT_TEST}} == 'save' ]]; then
                       outfile=out_save.txt
                       fn="test_${RANDOM}.txt"
      -                test_cmd="touch /cvmfs/pilot.nessi.no/${fn}"
      +                test_cmd="touch /cvmfs/software.eessi.io/${fn}"
                       ./eessi_container.sh --verbose --mode shell --access rw --save test-save.tar <<< "${test_cmd}" 2>&1 | tee ${outfile}
                       rm -f ${outfile}
      
      -                ./eessi_container.sh --verbose --mode shell --access rw --resume test-save.tar <<< "ls -l /cvmfs/pilot.nessi.no/${fn}" > ${outfile}
      -                grep "/cvmfs/pilot.nessi.no/${fn}$" $outfile
      +                ./eessi_container.sh --verbose --mode shell --access rw --resume test-save.tar <<< "ls -l /cvmfs/software.eessi.io/${fn}" > ${outfile}
      +                grep "/cvmfs/software.eessi.io/${fn}$" $outfile
      
                       tar tfv test-save.tar | grep "overlay-upper/${fn}"
      
    • .github/workflows/tests_archdetect.yml

      diff
      diff --git a/.github/workflows/tests_archdetect.yml b/.github/workflows/tests_archdetect.yml
      index 74dbf03..1e8b830 100644
      --- a/.github/workflows/tests_archdetect.yml
      +++ b/.github/workflows/tests_archdetect.yml
      @@ -13,23 +13,23 @@ jobs:
                 - x86_64/intel/skylake_avx512/archspec-linux-6132
                 - x86_64/amd/zen2/Azure-CentOS7-7V12
                 - x86_64/amd/zen3/Azure-CentOS7-7V73X
      -          # commented out since these targets are currently not supported in pilot.nessi.no repo
      +          - aarch64/neoverse_n1/Azure-Ubuntu20-Altra
      +          - aarch64/neoverse_n1/AWS-awslinux-graviton2
      +          - aarch64/neoverse_v1/AWS-awslinux-graviton3
      +          # commented out since these targets are currently not supported in software.eessi.io repo
                 # (and some tests assume that the corresponding subdirectory in software layer is there)
                 # - ppc64le/power9le/unknown-power9le
      -          # - aarch64/neoverse-n1/Azure-Ubuntu20-Altra
      -          # - aarch64/neoverse-n1/AWS-awslinux-graviton2
      -          # - aarch64/neoverse-v1/AWS-awslinux-graviton3
             fail-fast: false
           steps:
           - name: checkout
             uses: actions/checkout@93ea575cb5d8a053eaa0ac8fa3b40d7e05a33cc8 # v3.1.0
      
      -    - name: Mount NESSI CernVM-FS repository
      +    - name: Mount EESSI CernVM-FS pilot repository
             uses: cvmfs-contrib/github-action-cvmfs@d4641d0d591c9a5c3be23835ced2fb648b44c04b # v3.1
             with:
      -          cvmfs_config_package: https://github.com/NorESSI/filesystem-layer/releases/download/latest/cvmfs-config-nessi_latest_all.deb
      +          cvmfs_config_package: https://github.com/EESSI/filesystem-layer/releases/download/latest/cvmfs-config-eessi_latest_all.deb
                 cvmfs_http_proxy: DIRECT
      -          cvmfs_repositories: pilot.nessi.no
      +          cvmfs_repositories: software.eessi.io
      
           - name: test eessi_archdetect.sh
             run: |
    • EESSI-install-software.sh

      diff
      diff --git a/EESSI-install-software.sh b/EESSI-install-software.sh
      index eb616b1..69de9d1 100755
      --- a/EESSI-install-software.sh
      +++ b/EESSI-install-software.sh
      @@ -1,6 +1,6 @@
       #!/bin/bash
       #
      -# Script to install NESSI software stack (version set through init/eessi_defaults)
      +# Script to install EESSI software stack (version set through init/eessi_defaults)
      
       # see example parsing of command line arguments at
       #   https://wiki.bash-hackers.org/scripting/posparams#using_a_while_loop
    • README.md

      diff
      diff --git a/README.md b/README.md
      index 4f06139..ab73ad7 100644
      --- a/README.md
      +++ b/README.md
      @@ -1,25 +1,25 @@
       # Software layer
      
      -The software layer of the NESSI project uses [EasyBuild](https://docs.easybuild.io), [Lmod](https://lmod.readthedocs.io) and [archspec](https://archspec.readthedocs.io).
      +The software layer of the EESSI project uses [EasyBuild](https://docs.easybuild.io), [Lmod](https://lmod.readthedocs.io) and [archspec](https://archspec.readthedocs.io).
      
       See also https://www.eessi.io/docs/software_layer .
      
      -## Accessing the software stack
      +## Pilot software stack
      
       You can set up your environment by sourcing the init script:
      

      -$ source /cvmfs/pilot.nessi.no/versions/2023.06/init/bash
      -Found NESSI repo @ /cvmfs/pilot.nessi.no/versions/2023.06!
      -Derived subdirectory for software layer: x86_64/intel/broadwell
      +$ source /cvmfs/software.eessi.io/versions/2023.06/init/bash
      +Found EESSI repo @ /cvmfs/software.eessi.io/versions/2023.06!
      +Derived subdirectory for software layer: x86_64/intel/haswell
      Using x86_64/intel/haswell subdirectory for software layer
      Initializing Lmod...
      -Prepending /cvmfs/pilot.nessi.no/versions/2023.06/software/x86_64/intel/broadwell/modules/all to $MODULEPATH...
      -Environment set up to use NESSI software stack, have fun!
      -{NESSI 2023.06} $
      +Prepending /cvmfs/software.eessi.io/versions/2023.06/software/x86_64/intel/haswell/modules/all to $MODULEPATH...
      +Environment set up to use EESSI (2023.06), have fun!
      +[EESSI 2023.06] $

      
      -### Accessing NESSI via a container
      +### Accessing EESSI via a container
      
      You need Singularity version 3.7 or newer. Then, simply run
      
      @@ -40,6 +40,6 @@ Note, not all features/arguments listed via `./eessi_container.sh --help` are im
      The software in this repository is distributed under the terms of the
      [GNU General Public License v2.0](https://opensource.org/licenses/GPL-2.0).
      
      -See [LICENSE](https://github.com/NorESSI/software-layer/blob/main/LICENSE) for more information.
      +See [LICENSE](https://github.com/EESSI/software-layer/blob/main/LICENSE) for more information.
      
      SPDX-License-Identifier: GPL-2.0-only
      
    • bot/build.sh

      diff
      diff --git a/bot/build.sh b/bot/build.sh
      index 7eb15f3..1622e75 100755
      --- a/bot/build.sh
      +++ b/bot/build.sh
      @@ -128,7 +128,7 @@ EESSI_REPOS_CFG_DIR_OVERRIDE=$(cfg_get_value "repository" "repos_cfg_dir")
       export EESSI_REPOS_CFG_DIR_OVERRIDE=${EESSI_REPOS_CFG_DIR_OVERRIDE:-${PWD}/cfg}
       echo "bot/build.sh: EESSI_REPOS_CFG_DIR_OVERRIDE='${EESSI_REPOS_CFG_DIR_OVERRIDE}'"
      
      -# determine NESSI version to be used from .repository.repo_version in ${JOB_CFG_FILE}
      +# determine EESSI version to be used from .repository.repo_version in ${JOB_CFG_FILE}
       # here, just set & export EESSI_VERSION_OVERRIDE
       # next script (eessi_container.sh) makes use of it via sourcing init scripts
       # (e.g., init/eessi_defaults or init/minimal_eessi_env)
    • build_container.sh

      diff
      diff --git a/build_container.sh b/build_container.sh
      new file mode 100755
      index 0000000..23a9e66
      --- /dev/null
      +++ b/build_container.sh
      @@ -0,0 +1,69 @@
      +#!/bin/bash
      +
      +base_dir=$(dirname $(realpath $0))
      +
      +BUILD_CONTAINER="docker://ghcr.io/eessi/build-node:debian11"
      +
      +if [ $# -lt 2 ]; then
      +    echo "Usage: $0 <shell|run> <path for temporary directories>" >&2
      +    exit 1
      +fi
      +SHELL_OR_RUN=$1
      +EESSI_TMPDIR=$2
      +shift 2
      +
      +if [ "$SHELL_OR_RUN" == "run" ] && [ $# -eq 0 ]; then
      +    echo "ERROR: No command specified to run?!" >&2
      +    exit 1
      +fi
      +
      +# make sure specified temporary directory exists
      +mkdir -p $EESSI_TMPDIR
      +
      +echo "Using $EESSI_TMPDIR as parent for temporary directories..."
      +
      +# create temporary directories
      +mkdir -p $EESSI_TMPDIR/{home,overlay-upper,overlay-work}
      +mkdir -p $EESSI_TMPDIR/{var-lib-cvmfs,var-run-cvmfs}
      +# configure Singularity
      +export SINGULARITY_CACHEDIR=$EESSI_TMPDIR/singularity_cache
      +
      +# take into account that $SINGULARITY_BIND may be defined already, to bind additional paths into the build container
      +BIND_PATHS="$EESSI_TMPDIR/var-run-cvmfs:/var/run/cvmfs,$EESSI_TMPDIR/var-lib-cvmfs:/var/lib/cvmfs,$EESSI_TMPDIR"
      +if [ -z $SINGULARITY_BIND ]; then
      +    export SINGULARITY_BIND="$BIND_PATHS"
      +else
      +    export SINGULARITY_BIND="$SINGULARITY_BIND,$BIND_PATHS"
      +fi
      +
      +# allow that SINGULARITY_HOME is defined before script is run
      +if [ -z $SINGULARITY_HOME ]; then
      +    export SINGULARITY_HOME="$EESSI_TMPDIR/home:/home/$USER"
      +fi
      +
      +source ${base_dir}/init/eessi_defaults
      +# strip "/cvmfs/" from default setting
      +repo_name=${EESSI_CVMFS_REPO/\/cvmfs\//}
      +
      +# set environment variables for fuse mounts in Singularity container
      +export EESSI_PILOT_READONLY="container:cvmfs2 ${repo_name} /cvmfs_ro/${repo_name}"
      +export EESSI_PILOT_WRITABLE_OVERLAY="container:fuse-overlayfs -o lowerdir=/cvmfs_ro/${repo_name} -o upperdir=$EESSI_TMPDIR/overlay-upper -o workdir=$EESSI_TMPDIR/overlay-work ${EESSI_CVMFS_REPO}"
      +
      +# pass $EESSI_SOFTWARE_SUBDIR_OVERRIDE into build container (if set)
      +if [ ! -z ${EESSI_SOFTWARE_SUBDIR_OVERRIDE} ]; then
      +    export SINGULARITYENV_EESSI_SOFTWARE_SUBDIR_OVERRIDE=${EESSI_SOFTWARE_SUBDIR_OVERRIDE}
      +    # also specify via $APPTAINERENV_* (future proof, cfr. https://apptainer.org/docs/user/latest/singularity_compatibility.html#singularity-environment-variable-compatibility)
      +    export APPTAINERENV_EESSI_SOFTWARE_SUBDIR_OVERRIDE=${EESSI_SOFTWARE_SUBDIR_OVERRIDE}
      +fi
      +
      +if [ "$SHELL_OR_RUN" == "shell" ]; then
      +    # start shell in Singularity container, with EESSI repository mounted with writable overlay
      +    echo "Starting Singularity build container..."
      +    singularity shell --fusemount "$EESSI_PILOT_READONLY" --fusemount "$EESSI_PILOT_WRITABLE_OVERLAY" $BUILD_CONTAINER
      +elif [ "$SHELL_OR_RUN" == "run" ]; then
      +    echo "Running '$@' in Singularity build container..."
      +    singularity exec --fusemount "$EESSI_PILOT_READONLY" --fusemount "$EESSI_PILOT_WRITABLE_OVERLAY" $BUILD_CONTAINER "$@"
      +else
      +    echo "ERROR: Unknown action specified: $SHELL_OR_RUN (should be either 'shell' or 'run')" >&2
      +    exit 1
      +fi
    • check_missing_installations.sh

      diff
      diff --git a/check_missing_installations.sh b/check_missing_installations.sh
      index dd292e4..5ea7c5a 100755
      --- a/check_missing_installations.sh
      +++ b/check_missing_installations.sh
      @@ -1,6 +1,6 @@
       #!/bin/bash
       #
      -# Script to check for missing installations in NESSI software stack (version 2023.06)
      +# Script to check for missing installations in EESSI software stack
       #
       # author: Kenneth Hoste (@boegel)
       # author: Thomas Roeblitz (@trz42)
    • configure_easybuild

      diff
      diff --git a/configure_easybuild b/configure_easybuild
      index c1bd1d3..c67b879 100644
      --- a/configure_easybuild
      +++ b/configure_easybuild
      @@ -25,7 +25,6 @@ fi
      
       # note: filtering Bison may break some installations, like Qt5 (see https://github.com/EESSI/software-layer/issues/49)
       # filtering pkg-config breaks R-bundle-Bioconductor installation (see also https://github.com/easybuilders/easybuild-easyconfigs/pull/11104)
      -# problems occur when filtering pkg-config with gnuplot too (picks up Lua 5.1 from $EPREFIX rather than from Lua 5.3 dependency)
       DEPS_TO_FILTER=Autoconf,Automake,Autotools,binutils,bzip2,DBus,flex,gettext,gperf,help2man,intltool,libreadline,libtool,M4,makeinfo,ncurses,util-linux,XZ,zlib
       # For aarch64 we need to also filter out Yasm.
       # See https://github.com/easybuilders/easybuild-easyconfigs/issues/11190
    • create_directory_tarballs.sh

      diff
      diff --git a/create_directory_tarballs.sh b/create_directory_tarballs.sh
      index a1f05b5..0270719 100755
      --- a/create_directory_tarballs.sh
      +++ b/create_directory_tarballs.sh
      @@ -9,7 +9,7 @@ fi
      
       version=$1
      
      -SOFTWARE_LAYER_TARBALL_URL=https://github.com/NorESSI/software-layer/tarball/nessi.no-2023.06
      +SOFTWARE_LAYER_TARBALL_URL="https://github.com/EESSI/software-layer/tarball/${version}-software.eessi.io"
      
       TOPDIR=$(dirname $(realpath $0))
      
    • create_tarball.sh

      diff
      diff --git a/create_tarball.sh b/create_tarball.sh
      index f6239cf..a619df9 100755
      --- a/create_tarball.sh
      +++ b/create_tarball.sh
      @@ -43,7 +43,7 @@ module_files_list=${tmpdir}/module_files.list.txt
       if [ -d ${eessi_version}/software/${os}/${cpu_arch_subdir}/.lmod ]; then
           # include Lmod cache and configuration file (lmodrc.lua),
           # skip whiteout files and backup copies of Lmod cache (spiderT.old.*)
      -    find ${eessi_version}/software/${os}/${cpu_arch_subdir}/.lmod -type f | egrep -v '/\.wh\.|spiderT.old' > ${files_list}
      +    find ${eessi_version}/software/${os}/${cpu_arch_subdir}/.lmod -type f | egrep -v '/\.wh\.|spiderT.old' >> ${files_list}
       fi
      
       # include scripts that were copied by install_scripts.sh, which we want to ship in EESSI repository
    • easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.7.2-2021a.yml

      diff
      diff --git a/easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.7.2-2021a.yml b/easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.7.2-2021a.yml
      deleted file mode 100644
      index 28612c5..0000000
      --- a/easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.7.2-2021a.yml
      +++ /dev/null
      @@ -1,27 +0,0 @@
      -easyconfigs:
      -  - OpenSSL-1.1.eb
      -  # - OpenSSL-1.1.eb:
      -  #     options:
      -  #       include-easyblocks-from-pr: 2922
      -  - git-2.32.0-GCCcore-10.3.0-nodocs.eb
      -  - GCC-10.3.0
      -  - CMake-3.20.1-GCCcore-10.3.0.eb
      -  # - CMake-3.20.1-GCCcore-10.3.0.eb:
      -  #     options:
      -  #       include-easyblocks-from-pr: 2248
      -  - Rust-1.52.1-GCCcore-10.3.0.eb
      -  - foss-2021a.eb
      -  - QuantumESPRESSO-6.7-foss-2021a.eb
      -  - GROMACS-2021.3-foss-2021a.eb:
      -       options:
      -         download-timeout: 1000
      -  - libGLU-9.0.1-GCCcore-10.3.0.eb
      -  # since the BAGEL ec file is new we have to keep the from-pr or eb fails
      -  - BAGEL-1.2.2-foss-2021a.eb:
      -        options:
      -          from-pr: 18446
      -  - WRF-4.3-foss-2021a-dmpar.eb
      -  - Yambo-5.1.1-foss-2021a.eb:
      -        options:
      -          from-pr: 18905
      -
    • easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.7.2-2021b.yml

      diff
      diff --git a/easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.7.2-2021b.yml b/easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.7.2-2021b.yml
      deleted file mode 100644
      index d838e15..0000000
      --- a/easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.7.2-2021b.yml
      +++ /dev/null
      @@ -1,18 +0,0 @@
      -easyconfigs:
      -  - GCC-11.2.0
      -  - git-2.33.1-GCCcore-11.2.0-nodocs.eb
      -  - CMake-3.21.1-GCCcore-11.2.0.eb
      -  # - CMake-3.21.1-GCCcore-11.2.0.eb:
      -  #     options:
      -  #       include-easyblocks-from-pr: 2248
      -  - OpenMPI-4.1.1-GCC-11.2.0.eb
      -  - FFTW-3.3.10-gompi-2021b.eb
      -  - BLIS-0.8.1-GCC-11.2.0.eb
      -  - foss-2021b.eb
      -  # note, the below CMake has never been installed, just put here to use the
      -  # same version as easy does
      -  # - CMake-3.22.1-GCCcore-11.2.0.eb:
      -  #     options:
      -  #       include-easyblocks-from-pr: 2248
      -  # - zlib-1.2.11-GCCcore-11.2.0
      -  - QuantumESPRESSO-6.8-foss-2021b.eb
    • easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.7.2-2022a.yml

      diff
      diff --git a/easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.7.2-2022a.yml b/easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.7.2-2022a.yml
      deleted file mode 100644
      index 9991de1..0000000
      --- a/easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.7.2-2022a.yml
      +++ /dev/null
      @@ -1,12 +0,0 @@
      -easyconfigs:
      -  - GCC-11.3.0
      -  - git-2.36.0-GCCcore-11.3.0-nodocs.eb
      -  - CMake-3.23.1-GCCcore-11.3.0.eb
      -  # - CMake-3.23.1-GCCcore-11.3.0.eb:
      -  #     options:
      -  #       include-easyblocks-from-pr: 2248
      -  - BLIS-0.9.0-GCC-11.3.0.eb
      -  - OpenMPI-4.1.4-GCC-11.3.0.eb
      -  - FFTW.MPI-3.3.10-gompi-2022a.eb
      -  - foss-2022a.eb
      -  - Python-3.10.4-GCCcore-11.3.0.eb
    • easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.7.2-2022b.yml

      diff
      diff --git a/easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.7.2-2022b.yml b/easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.7.2-2022b.yml
      deleted file mode 100644
      index 129b709..0000000
      --- a/easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.7.2-2022b.yml
      +++ /dev/null
      @@ -1,12 +0,0 @@
      -easyconfigs:
      -  - GCC-12.2.0
      -  - git-2.38.1-GCCcore-12.2.0-nodocs.eb
      -  - CMake-3.24.3-GCCcore-12.2.0.eb
      -  # - CMake-3.24.3-GCCcore-12.2.0.eb:
      -  #     options:
      -  #       include-easyblocks-from-pr: 2248
      -  - BLIS-0.9.0-GCC-12.2.0.eb
      -  - OpenMPI-4.1.4-GCC-12.2.0.eb
      -  - FFTW.MPI-3.3.10-gompi-2022b.eb
      -  - Python-3.10.8-GCCcore-12.2.0.eb
      -  - foss-2022b.eb
    • easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.7.2-system.yml

      diff
      diff --git a/easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.7.2-system.yml b/easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.7.2-system.yml
      deleted file mode 100644
      index a5f84c7..0000000
      --- a/easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.7.2-system.yml
      +++ /dev/null
      @@ -1,4 +0,0 @@
      -easyconfigs:
      -  - EasyBuild-4.8.0.eb:
      -      options:
      -        from-pr: 18282
    • easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.8.0-system.yml

      diff
      diff --git a/easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.8.0-system.yml b/easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.8.0-system.yml
      deleted file mode 100644
      index 96d79de..0000000
      --- a/easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.8.0-system.yml
      +++ /dev/null
      @@ -1,4 +0,0 @@
      -easyconfigs:
      -  - EasyBuild-4.8.1.eb:
      -      options:
      -        from-pr: 18761
    • easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.8.1-2022a.yml

      diff
      diff --git a/easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.8.1-2022a.yml b/easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.8.1-2022a.yml
      deleted file mode 100644
      index 14f936b..0000000
      --- a/easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.8.1-2022a.yml
      +++ /dev/null
      @@ -1,66 +0,0 @@
      -easyconfigs:
      -  - BAMM-2.5.0-foss-2022a.eb
      -  - BamTools-2.5.2-GCC-11.3.0.eb
      -  - BCFtools-1.15.1-GCC-11.3.0.eb
      -  - Bowtie2-2.4.5-GCC-11.3.0.eb
      -  - FastTree-2.1.11-GCCcore-11.3.0.eb
      -  - SAMtools-1.16.1-GCC-11.3.0.eb
      -  - gzip-1.12-GCCcore-11.3.0.eb
      -  - lz4-1.9.3-GCCcore-11.3.0.eb
      -  - zstd-1.5.2-GCCcore-11.3.0.eb
      -  - Pillow-9.1.1-GCCcore-11.3.0.eb:
      -      # avoid that hardcoded paths like /usr/include are used in build commands
      -        options:
      -          from-pr: 18881
      -  - Boost.Python-1.79.0-GCC-11.3.0.eb
      -  - netCDF-4.9.0-gompi-2022a.eb:
      -      # use updated CMakeMake easyblock to avoid that -DCMAKE_SKIP_RPATH=ON is used, which breaks the netCDF test step
      -      # see https://github.com/easybuilders/easybuild-easyblocks/pull/3012
      -      options:
      -        include-easyblocks-from-pr: 3012
      -  - NCO-5.1.0-foss-2022a.eb
      -  - AdapterRemoval-2.3.3-GCC-11.3.0.eb
      -  - BEDTools-2.30.0-GCC-11.3.0.eb
      -  - Pysam-0.20.0-GCC-11.3.0.eb
      -  - TransDecoder-5.5.0-GCC-11.3.0.eb
      -  - VCFtools-0.1.16-GCC-11.3.0.eb
      -  - XML-LibXML-2.0207-GCCcore-11.3.0.eb
      -  - CMake-3.24.3-GCCcore-11.3.0.eb
      -  - elfutils-0.187-GCCcore-11.3.0.eb
      -  - Ninja-1.10.2-GCCcore-11.3.0.eb
      -  - Z3-4.10.2-GCCcore-11.3.0.eb
      -  - SciPy-bundle-2022.05-foss-2022a.eb
      -  - Xvfb-21.1.3-GCCcore-11.3.0.eb:
      -      # enable exec permissions for xvfb-run after copying;
      -      # need to also enable user write permissions on xvfb-run to ensure that copying with preserved permissions works
      -      options:
      -        from-pr: 18834
      -  - tbb-2021.5.0-GCCcore-11.3.0.eb
      -  - CMSeq-1.0.4-foss-2022a.eb
      -  - Pango-1.50.7-GCCcore-11.3.0.eb
      -  - nodejs-16.15.1-GCCcore-11.3.0.eb:
      -      options:
      -        download-timeout: 1000
      -  - ecCodes-2.27.0-gompi-2022a.eb:
      -        options:
      -          download-timeout: 1000
      -  - GLPK-5.0-GCCcore-11.3.0.eb
      -  - libgit2-1.4.3-GCCcore-11.3.0.eb
      -  - libGLU-9.0.2-GCCcore-11.3.0.eb
      -  - libsndfile-1.1.0-GCCcore-11.3.0.eb
      -  - LibTIFF-4.3.0-GCCcore-11.3.0.eb
      -  - MPFR-4.1.0-GCCcore-11.3.0.eb
      -  - NLopt-2.7.1-GCCcore-11.3.0.eb
      -  - PCRE2-10.40-GCCcore-11.3.0.eb
      -  - Tk-8.6.12-GCCcore-11.3.0.eb
      -  - GROMACS-2023.1-foss-2022a.eb
      -  - MUMPS-5.5.1-foss-2022a-metis.eb
      -  - Pillow-9.1.1-GCCcore-11.3.0.eb:
      -      # Uses a custom hook since has zlib as dependency which has hard coded header and library path within Pillow code.
      -      options:
      -        from-pr: 18881
      -  - ParaView-5.10.1-foss-2022a-mpi.eb:
      -        options:
      -          download-timeout: 1000
      -  - ASE-3.22.1-foss-2022a.eb
      -  - HISAT2-2.2.1-gompi-2022a.eb
    • easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.8.2-2022a.yml

      diff
      diff --git a/easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.8.2-2022a.yml b/easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.8.2-2022a.yml
      deleted file mode 100644
      index f191e76..0000000
      --- a/easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.8.2-2022a.yml
      +++ /dev/null
      @@ -1,3 +0,0 @@
      -easyconfigs:
      -  - CDO-2.0.6-gompi-2022a.eb
      -  - OpenBabel-3.1.1-gompi-2022a.eb
    • easystacks/pilot.nessi.no/2023.06/README.md

      diff
      diff --git a/easystacks/pilot.nessi.no/2023.06/README.md b/easystacks/software.eessi.io/2023.06/README.md
      similarity index 100%
      rename from easystacks/pilot.nessi.no/2023.06/README.md
      rename to easystacks/software.eessi.io/2023.06/README.md
    • easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.8.1-system.yml

      diff
      diff --git a/easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.8.1-system.yml b/easystacks/software.eessi.io/2023.06/eessi-2023.06-eb-4.8.2-001-system.yml
      similarity index 54%
      rename from easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.8.1-system.yml
      rename to easystacks/software.eessi.io/2023.06/eessi-2023.06-eb-4.8.2-001-system.yml
      index 7418c0a..f02b9f2 100644
      --- a/easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.8.1-system.yml
      +++ b/easystacks/software.eessi.io/2023.06/eessi-2023.06-eb-4.8.2-001-system.yml
      @@ -1,6 +1,7 @@
       easyconfigs:
      -  # wraps around Java/11.0.20
      -  - Java-11.eb
         - EasyBuild-4.8.2.eb:
             options:
               from-pr: 19105
      +  - Nextflow-23.10.0.eb:
      +      options:
      +        from-pr: 19172
    • easystacks/software.eessi.io/2023.06/eessi-2023.06-eb-4.8.2-010-CUDA.yml

      diff
      diff --git a/easystacks/software.eessi.io/2023.06/eessi-2023.06-eb-4.8.2-010-CUDA.yml b/easystacks/software.eessi.io/2023.06/eessi-2023.06-eb-4.8.2-010-CUDA.yml
      new file mode 100644
      index 0000000..dda274b
      --- /dev/null
      +++ b/easystacks/software.eessi.io/2023.06/eessi-2023.06-eb-4.8.2-010-CUDA.yml
      @@ -0,0 +1,5 @@
      +easyconfigs:
      +  - CUDA-12.1.1.eb:
      +      options:
      +        include-easyblocks-from-pr: 3045
      +        accept-eula-for: CUDA
    • easystacks/software.eessi.io/2023.06/eessi-2023.06-eb-4.8.2-2022b.yml

      diff
      diff --git a/easystacks/software.eessi.io/2023.06/eessi-2023.06-eb-4.8.2-2022b.yml b/easystacks/software.eessi.io/2023.06/eessi-2023.06-eb-4.8.2-2022b.yml
      new file mode 100644
      index 0000000..69ffb75
      --- /dev/null
      +++ b/easystacks/software.eessi.io/2023.06/eessi-2023.06-eb-4.8.2-2022b.yml
      @@ -0,0 +1,6 @@
      +easyconfigs:
      +  - foss-2022b.eb
      +  - HarfBuzz-5.3.1-GCCcore-12.2.0.eb:
      +      options:
      +        from-pr: 19339
      +  - Qt5-5.15.7-GCCcore-12.2.0.eb
    • easystacks/software.eessi.io/2023.06/eessi-2023.06-eb-4.8.2-2023a.yml

      diff
      diff --git a/easystacks/software.eessi.io/2023.06/eessi-2023.06-eb-4.8.2-2023a.yml b/easystacks/software.eessi.io/2023.06/eessi-2023.06-eb-4.8.2-2023a.yml
      new file mode 100644
      index 0000000..aab910d
      --- /dev/null
      +++ b/easystacks/software.eessi.io/2023.06/eessi-2023.06-eb-4.8.2-2023a.yml
      @@ -0,0 +1,48 @@
      +easyconfigs:
      +  - GCC-12.3.0.eb
      +  - Rust-1.70.0-GCCcore-12.3.0.eb:
      +      # fix build of Rust 1.70.0 by disabling download of pre-built LLVM;
      +      # see https://github.com/easybuilders/easybuild-easyblocks/pull/3038
      +      options:
      +        include-easyblocks-from-pr: 3038
      +  - foss-2023a.eb
      +  - pybind11-2.11.1-GCCcore-12.3.0.eb:
      +      # avoid indirect dependency on old CMake version built with GCCcore/10.2.0 via Catch2 build dependency;
      +      # see https://github.com/easybuilders/easybuild-easyconfigs/pull/19270
      +      options:
      +        from-pr: 19270
      +  - SciPy-bundle-2023.07-gfbf-2023a.eb
      +  - TensorFlow-2.13.0-foss-2023a.eb:
      +      # patch setup.py for grpcio extension in TensorFlow 2.13.0 easyconfigs to take into account alternate sysroot;
      +      # see https://github.com/easybuilders/easybuild-easyconfigs/pull/19268
      +      options:
      +        from-pr: 19268
      +  - X11-20230603-GCCcore-12.3.0.eb
      +  - HarfBuzz-5.3.1-GCCcore-12.3.0.eb:
      +      options:
      +        from-pr: 19339
      +  - Qt5-5.15.10-GCCcore-12.3.0.eb
      +  - OSU-Micro-Benchmarks-7.1-1-gompi-2023a.eb
      +  - LHAPDF-6.5.4-GCC-12.3.0.eb:
      +      options:
      +        from-pr: 19363
      +  - LoopTools-2.15-GCC-12.3.0.eb:
      +      options:
      +        from-pr: 19397
      +  - R-4.3.2-gfbf-2023a.eb:
      +      options:
      +        from-pr: 19185
      +  - Boost-1.82.0-GCC-12.3.0.eb
      +  - netCDF-4.9.2-gompi-2023a.eb
      +  - FFmpeg-6.0-GCCcore-12.3.0.eb
      +  - CUDA-Samples-12.1-GCC-12.3.0-CUDA-12.1.1.eb:
      +      # use easyconfig that only install subset of CUDA samples,
      +      # to circumvent problem with nvcc linking to glibc of host OS,
      +      # see https://github.com/easybuilders/easybuild-easyconfigs/pull/19189;
      +      # and where additional samples are excluded because they fail to build on aarch64,
      +      # see https://github.com/easybuilders/easybuild-easyconfigs/pull/19451;
      +      options:
      +        from-pr: 19451
      +  - ALL-0.9.2-foss-2023a.eb:
      +      options:
      +        from-pr: 19455
    • easystacks/software.eessi.io/2023.06/eessi-2023.06-eb-4.9.0-001-system.yml

      diff
      diff --git a/easystacks/software.eessi.io/2023.06/eessi-2023.06-eb-4.9.0-001-system.yml b/easystacks/software.eessi.io/2023.06/eessi-2023.06-eb-4.9.0-001-system.yml
      new file mode 100644
      index 0000000..25c13e4
      --- /dev/null
      +++ b/easystacks/software.eessi.io/2023.06/eessi-2023.06-eb-4.9.0-001-system.yml
      @@ -0,0 +1,5 @@
      +easyconfigs:
      +  - EasyBuild-4.9.0.eb:
      +      options:
      +        from-pr: 19464
      +  - ReFrame-4.3.3.eb
    • easystacks/software.eessi.io/2023.06/eessi-2023.06-eb-4.9.0-2022b.yml

      diff
      diff --git a/easystacks/software.eessi.io/2023.06/eessi-2023.06-eb-4.9.0-2022b.yml b/easystacks/software.eessi.io/2023.06/eessi-2023.06-eb-4.9.0-2022b.yml
      new file mode 100644
      index 0000000..fdacd95
      --- /dev/null
      +++ b/easystacks/software.eessi.io/2023.06/eessi-2023.06-eb-4.9.0-2022b.yml
      @@ -0,0 +1,2 @@
      +easyconfigs:
      +  - SciPy-bundle-2023.02-gfbf-2022b.eb
    • easystacks/software.eessi.io/2023.06/eessi-2023.06-eb-4.9.0-2023a.yml

      diff
      diff --git a/easystacks/software.eessi.io/2023.06/eessi-2023.06-eb-4.9.0-2023a.yml b/easystacks/software.eessi.io/2023.06/eessi-2023.06-eb-4.9.0-2023a.yml
      new file mode 100644
      index 0000000..eba776a
      --- /dev/null
      +++ b/easystacks/software.eessi.io/2023.06/eessi-2023.06-eb-4.9.0-2023a.yml
      @@ -0,0 +1,10 @@
      +easyconfigs:
      +  - OpenFOAM-11-foss-2023a.eb:
      +      # see https://github.com/easybuilders/easybuild-easyconfigs/pull/19545
      +      options:
      +        from-pr: 19545
      +  - at-spi2-core-2.49.91-GCCcore-12.3.0.eb
      +  - ESPResSo-4.2.1-foss-2023a.eb:
      +      # see https://github.com/easybuilders/easybuild-easyconfigs/pull/19592
      +      options:
      +        from-pr: 19592
    • easystacks/software.eessi.io/2023.06/eessi-2023.06-eb-4.9.0-2023b.yml

      diff
      diff --git a/easystacks/software.eessi.io/2023.06/eessi-2023.06-eb-4.9.0-2023b.yml b/easystacks/software.eessi.io/2023.06/eessi-2023.06-eb-4.9.0-2023b.yml
      new file mode 100644
      index 0000000..9a3b8ef
      --- /dev/null
      +++ b/easystacks/software.eessi.io/2023.06/eessi-2023.06-eb-4.9.0-2023b.yml
      @@ -0,0 +1,7 @@
      +easyconfigs:
      +  - GCC-13.2.0.eb
      +  - foss-2023b.eb
      +  - SciPy-bundle-2023.11-gfbf-2023b.eb
      +  - netCDF-4.9.2-gompi-2023b.eb:
      +      options:
      +        from-pr: 19534
    • eb_hooks.py

      diff
      diff --git a/eb_hooks.py b/eb_hooks.py
      index b3cbc4b..2bc2fde 100644
      --- a/eb_hooks.py
      +++ b/eb_hooks.py
      @@ -185,20 +185,21 @@ def parse_hook_fontconfig_add_fonts(ec, eprefix):
      
      
       def parse_hook_openblas_relax_lapack_tests_num_errors(ec, eprefix):
      -    """Relax number of failing numerical LAPACK tests for aarch64/neoverse_v1 CPU target."""
      +    """Relax number of failing numerical LAPACK tests for aarch64/neoverse_v1 CPU target for OpenBLAS < 0.3.23"""
           cpu_target = get_eessi_envvar('EESSI_SOFTWARE_SUBDIR')
           if ec.name == 'OpenBLAS':
      -        # relax maximum number of failed numerical LAPACK tests for aarch64/neoverse_v1 CPU target
      -        # since the default setting of 150 that works well on other aarch64 targets and x86_64 is a bit too strict
      -        # See https://github.com/EESSI/software-layer/issues/314
      -        cfg_option = 'max_failing_lapack_tests_num_errors'
      -        if cpu_target == CPU_TARGET_NEOVERSE_V1:
      -            orig_value = ec[cfg_option]
      -            ec[cfg_option] = 400
      -            print_msg("Maximum number of failing LAPACK tests with numerical errors for %s relaxed to %s (was %s)",
      -                      ec.name, ec[cfg_option], orig_value)
      -        else:
      -            print_msg("Not changing option %s for %s on non-AARCH64", cfg_option, ec.name)
      +        if LooseVersion(ec.version) < LooseVersion('0.3.23'):
      +            # relax maximum number of failed numerical LAPACK tests for aarch64/neoverse_v1 CPU target
      +            # since the default setting of 150 that works well on other aarch64 targets and x86_64 is a bit too strict
      +            # See https://github.com/EESSI/software-layer/issues/314
      +            cfg_option = 'max_failing_lapack_tests_num_errors'
      +            if cpu_target == CPU_TARGET_NEOVERSE_V1:
      +                orig_value = ec[cfg_option]
      +                ec[cfg_option] = 400
      +                print_msg("Maximum number of failing LAPACK tests with numerical errors for %s relaxed to %s (was %s)",
      +                          ec.name, ec[cfg_option], orig_value)
      +            else:
      +                print_msg("Not changing option %s for %s on non-AARCH64", cfg_option, ec.name)
           else:
               raise EasyBuildError("OpenBLAS-specific hook triggered for non-OpenBLAS easyconfig?!")
      
      @@ -393,6 +394,18 @@ def pre_test_hook_ignore_failing_tests_SciPybundle(self, *args, **kwargs):
           if self.name == 'SciPy-bundle' and self.version in scipy_bundle_versions and cpu_target == CPU_TARGET_NEOVERSE_V1:
               self.cfg['testopts'] = "|| echo ignoring failing tests"
      
      +def pre_test_hook_ignore_failing_tests_netCDF(self, *args, **kwargs):
      +    """
      +    Pre-test hook for netCDF: skip failing tests for selected netCDF versions on neoverse_v1
      +    cfr. https://github.com/EESSI/software-layer/issues/425
      +    The following tests are problematic:
      +        163 - nc_test4_run_par_test (Timeout)
      +        190 - h5_test_run_par_tests (Timeout)
      +    A few other tests are skipped in the easyconfig and patches for similar issues, see above issue for details.
      +    """
      +    cpu_target = get_eessi_envvar('EESSI_SOFTWARE_SUBDIR')
      +    if self.name == 'netCDF' and self.version == '4.9.2' and cpu_target == CPU_TARGET_NEOVERSE_V1:
      +        self.cfg['testopts'] = "|| echo ignoring failing tests"
      
       def pre_single_extension_hook(ext, *args, **kwargs):
           """Main pre-extension: trigger custom functions based on software name."""
      @@ -573,6 +586,7 @@ PRE_TEST_HOOKS = {
           'ESPResSo': pre_test_hook_ignore_failing_tests_ESPResSo,
           'FFTW.MPI': pre_test_hook_ignore_failing_tests_FFTWMPI,
           'SciPy-bundle': pre_test_hook_ignore_failing_tests_SciPybundle,
      +    'netCDF': pre_test_hook_ignore_failing_tests_netCDF,
       }
      
       PRE_SINGLE_EXTENSION_HOOKS = {
    • eessi-2023.06-known-issues.yml

      diff
      diff --git a/eessi-2023.06-known-issues.yml b/eessi-2023.06-known-issues.yml
      new file mode 100644
      index 0000000..475ee2c
      --- /dev/null
      +++ b/eessi-2023.06-known-issues.yml
      @@ -0,0 +1,28 @@
      +- aarch64/neoverse_v1:
      +  - ESPResSo-4.2.1-foss-2023a:
      +      - issue: https://github.com/EESSI/software-layer/issues/363
      +      - info: "ESPResSo tests failing due to timeouts"
      +  - FFTW.MPI-3.3.10-gompi-2023a:
      +      - issue: https://github.com/EESSI/software-layer/issues/325
      +      - info: "Flaky FFTW tests, random failures"
      +  - FFTW.MPI-3.3.10-gompi-2023b:
      +      - issue: https://github.com/EESSI/software-layer/issues/325
      +      - info: "Flaky FFTW tests, random failures"
      +  - netCDF-4.9.2-gompi-2023a.eb:
      +      - issue: https://github.com/EESSI/software-layer/issues/425
      +      - info: "netCDF intermittent test failures"
      +  - netCDF-4.9.2-gompi-2023b.eb:
      +      - issue: https://github.com/EESSI/software-layer/issues/425
      +      - info: "netCDF intermittent test failures"
      +  - OpenBLAS-0.3.21-GCC-12.2.0:
      +    - issue: https://github.com/EESSI/software-layer/issues/314
      +    - info: "Increased number of numerical errors in OpenBLAS test suite (344 vs max. 150 on x86_64/*)"
      +  - SciPy-bundle-2023.02-gfbf-2022b:
      +    - issue: https://github.com/EESSI/software-layer/issues/318
      +    - info: "numpy built with -march=armv8.4-a instead of -mcpu=native (no SVE) + 2 failing tests (vs 50005 passed) in scipy test suite"
      +  - SciPy-bundle-2023.07-gfbf-2023a:
      +    - issue: https://github.com/EESSI/software-layer/issues/318
      +    - info: "2 failing tests (vs 54409 passed) in scipy test suite"
      +  - SciPy-bundle-2023.11-gfbf-2023b:
      +    - issue: https://github.com/EESSI/software-layer/issues/318
      +    - info: "2 failing tests (vs 54876 passed) in scipy test suite"
    • eessi_container.sh

      diff
      diff --git a/eessi_container.sh b/eessi_container.sh
      index bfc5963..d6e9558 100755
      --- a/eessi_container.sh
      +++ b/eessi_container.sh
      @@ -30,8 +30,8 @@
       # -. initial settings & exit codes
       TOPDIR=$(dirname $(realpath $0))
      
      -source ${TOPDIR}/scripts/utils.sh
      -source ${TOPDIR}/scripts/cfg_files.sh
      +source "${TOPDIR}"/scripts/utils.sh
      +source "${TOPDIR}"/scripts/cfg_files.sh
      
       # exit codes: bitwise shift codes to allow for combination of exit codes
       # ANY_ERROR_EXITCODE is sourced from ${TOPDIR}/scripts/utils.sh
      @@ -83,7 +83,7 @@ display_help() {
         echo "                           MODE==run (run a script or command) [default: shell]"
         echo "  -n | --nvidia MODE     - configure the container to work with NVIDIA GPUs,"
         echo "                           MODE==install for a CUDA installation, MODE==run to"
      -  echo "                           attach a GPU, MODE==all for both [default: false]"
      +  echo "                           attach a GPU, MODE==all for both [default: false]"
         echo "  -r | --repository CFG  - configuration file or identifier defining the"
         echo "                           repository to use [default: EESSI via"
         echo "                           default container, see --container]"
      @@ -164,7 +164,7 @@ while [[ $# -gt 0 ]]; do
             SETUP_NVIDIA=1
             NVIDIA_MODE="$2"
             shift 2
      -      ;;
      +      ;;
           -r|--repository)
             REPOSITORY="$2"
             shift 2
      @@ -575,7 +575,7 @@ fi
       declare -a EESSI_FUSE_MOUNTS=()
      
       # always mount cvmfs-config repo (to get access to software.eessi.io)
      -# EESSI_FUSE_MOUNTS+=("--fusemount" "container:cvmfs2 cvmfs-config.cern.ch /cvmfs/cvmfs-config.cern.ch")
      +EESSI_FUSE_MOUNTS+=("--fusemount" "container:cvmfs2 cvmfs-config.cern.ch /cvmfs/cvmfs-config.cern.ch")
      
       if [[ "${ACCESS}" == "ro" ]]; then
         export EESSI_READONLY="container:cvmfs2 ${repo_name} /cvmfs/${repo_name}"
    • init/Magic_Castle/bash

      diff
      diff --git a/init/Magic_Castle/bash b/init/Magic_Castle/bash
      index 0fb2f67..dbe1772 100644
      --- a/init/Magic_Castle/bash
      +++ b/init/Magic_Castle/bash
      @@ -6,7 +6,7 @@ EESSI_SILENT=1
       source $(dirname "$BASH_SOURCE")/../eessi_environment_variables
      
       # Don't change the default prompt
      -# export PS1="{NESSI $EESSI_VERSION} $ "
      +# export PS1="[EESSI $EESSI_VERSION] $ "
      
       # Provide a clean MODULEPATH
       export MODULEPATH_ROOT=$EESSI_MODULEPATH
      @@ -36,4 +36,4 @@ else
          module reload
       fi
      
      -echo "Environment set up to use NESSI (${EESSI_VERSION}), have fun!"
      +echo "Environment set up to use EESSI (${EESSI_VERSION}), have fun!"
    • init/Magic_Castle/eessi_python3

      diff
      diff --git a/init/Magic_Castle/eessi_python3 b/init/Magic_Castle/eessi_python3
      index c76c7a2..b2f7fd4 100755
      --- a/init/Magic_Castle/eessi_python3
      +++ b/init/Magic_Castle/eessi_python3
      @@ -16,6 +16,6 @@ eessi_python=$(ls ${EESSI_SOFTWARE_PATH}/software/Python/3*GCCcore*/bin/python |
       if [ -f "$eessi_python" ]; then
         $eessi_python "$@"
       else
      -  echo "ERROR: No NESSI Python 3 available."
      +  echo "ERROR: No EESSI Python 3 available."
         false
       fi
    • init/README.md

      diff
      diff --git a/init/README.md b/init/README.md
      index 05baffd..c253dc4 100644
      --- a/init/README.md
      +++ b/init/README.md
      @@ -1,7 +1,7 @@
      -# Bash initialisation for NESSI
      +# Bash initialisation for EESSI
      
       This directory contains the default initialisation script for a bash shell used to
      -configure Lmod and use the NESSI software modules. The (bash)
      +configure Lmod and use the EESSI software modules. The (bash)
       file `eessi_environment_variables` is used to set and export the full set of EESSI
       environment variables:
      
      @@ -9,12 +9,12 @@ environment variables:
       - `EESSI_EPREFIX`: The location of Gentoo Prefix compatability layer (for the architecture).
       - `EESSI_EPREFIX_PYTHON`: Path to `python3` in the Gentoo Prefix layer.
       - `EESSI_SOFTWARE_SUBDIR`: Hardware specific software subdirectory.
      -- `EESSI_SOFTWARE_PATH`: Full path to NESSI software stack.
      +- `EESSI_SOFTWARE_PATH`: Full path to EESSI software stack.
       - `EESSI_MODULEPATH`: Path to be added to the `MODULEPATH`. This can be influenced by two
         externally defined environment varialbes:
           - `EESSI_CUSTOM_MODULEPATH`: defines a fully custom directory to be added to
             `MODULEPATH`, the end user is entirely responsible for what this directory contains.
      -    - `EESSI_MODULE_SUBDIR`: NESSI may ship with a number of possible module naming schemes.
      +    - `EESSI_MODULE_SUBDIR`: EESSI may ship with a number of possible module naming schemes.
             This variable can be used to point to a non-default module naming scheme.
      
       All scripts respect the environment variable `EESSI_SILENT` which, if defined to any
      @@ -25,5 +25,5 @@ value, will make them produce no (non-error) output.
       The `Magic_Castle` subdirectory is home to the bash initialisation that we use for
       [Magic Castle](https://github.com/ComputeCanada/magic_castle).
      
      -It also contains a wrapper for a hardware optimised NESSI Python 3 installation that is used
      +It also contains a wrapper for a hardware optimised EESSI Python 3 installation that is used
       by Magic Castle to properly configure JupyterHub.
    • init/arch_specs/eessi_arch_arm.spec

      diff
      diff --git a/init/arch_specs/eessi_arch_arm.spec b/init/arch_specs/eessi_arch_arm.spec
      index f8f21f9..b5c9275 100755
      --- a/init/arch_specs/eessi_arch_arm.spec
      +++ b/init/arch_specs/eessi_arch_arm.spec
      @@ -1,6 +1,6 @@
       # ARM CPU architecture specifications
       # Software path in EESSI 	| Vendor ID 	| List of defining CPU features
      -"aarch64/neoverse-n1"	"ARM"		"asimd"		# Ampere Altra
      -"aarch64/neoverse-n1"	""		"asimd"		# AWS Graviton2
      -"aarch64/neoverse-v1"	"ARM"		"asimd svei8mm"
      -"aarch64/neoverse-v1"	""		"asimd svei8mm"	# AWS Graviton3
      +"aarch64/neoverse_n1"	"ARM"		"asimd"		# Ampere Altra
      +"aarch64/neoverse_n1"	""		"asimd"		# AWS Graviton2
      +"aarch64/neoverse_v1"	"ARM"		"asimd svei8mm"
      +"aarch64/neoverse_v1"	""		"asimd svei8mm"	# AWS Graviton3
    • init/bash

      diff
      diff --git a/init/bash b/init/bash
      index 26598bb..0029f91 100644
      --- a/init/bash
      +++ b/init/bash
      @@ -5,6 +5,7 @@ function show_msg {
           echo "$msg"
         fi
       }
      +
       # The following method should be safe, but might break if file is a symlink
       # (could switch to $(dirname "$(readlink -f "$BASH_SOURCE")") in that case)
       source $(dirname "$BASH_SOURCE")/eessi_environment_variables
      @@ -12,7 +13,7 @@ source $(dirname "$BASH_SOURCE")/eessi_environment_variables
       # only continue if setting EESSI environment variables worked fine
       if [ $? -eq 0 ]; then
      
      -    export PS1="{NESSI $EESSI_VERSION} $PS1"
      +    export PS1="{EESSI $EESSI_VERSION} $PS1"
      
           # add location of commands provided by compat layer to $PATH;
           # see https://github.com/EESSI/software-layer/issues/52
      @@ -23,7 +24,7 @@ if [ $? -eq 0 ]; then
           source $EESSI_EPREFIX/usr/share/Lmod/init/bash
      
           # prepend location of modules for EESSI software stack to $MODULEPATH
      -    echo "Prepending $EESSI_MODULEPATH to \$MODULEPATH..." >> $output
      +    show_msg "Prepending $EESSI_MODULEPATH to \$MODULEPATH..."
           module use $EESSI_MODULEPATH
      
           #show_msg ""
      @@ -33,6 +34,6 @@ if [ $? -eq 0 ]; then
           #show_msg ""
           #show_msg ""
      
      -    echo "Environment set up to use NESSI (${EESSI_VERSION}), have fun!"
      +    echo "Environment set up to use EESSI (${EESSI_VERSION}), have fun!"
      
       fi
    • init/eessi_archdetect.sh

      diff
      diff --git a/init/eessi_archdetect.sh b/init/eessi_archdetect.sh
      index 236ac5a..58b79b0 100755
      --- a/init/eessi_archdetect.sh
      +++ b/init/eessi_archdetect.sh
      @@ -73,7 +73,7 @@ cpupath(){
           [ $EESSI_SOFTWARE_SUBDIR_OVERRIDE ] && echo ${EESSI_SOFTWARE_SUBDIR_OVERRIDE} && exit
      
           # Identify the best matching CPU architecture from a list of supported specifications for the host CPU
      -    # Return the path to the installation files in NESSI of the best matching architecture
      +    # Return the path to the installation files in EESSI of the best matching architecture
           local cpu_arch_spec=()
      
           # Identify the host CPU architecture
      @@ -118,7 +118,7 @@ cpupath(){
                   # each flag in this CPU specification must be found in the list of flags of the host
                   check_allinfirst "${cpu_flags[*]}" ${arch_spec[2]} && best_arch_match=${arch_spec[0]} && \
                       all_arch_matches="$best_arch_match:$all_arch_matches" && \
      -                log "DEBUG" "cpupath: host CPU best match updated to $best_arch_match"
      +                log "DEBUG" "cpupath: host CPU best match updated to $best_arch_match"
               fi
           done
      
    • init/eessi_defaults

      diff
      diff --git a/init/eessi_defaults b/init/eessi_defaults
      index da057fb..d1779a3 100644
      --- a/init/eessi_defaults
      +++ b/init/eessi_defaults
      @@ -8,9 +8,8 @@
       # license: GPLv2
       #
      
      -export EESSI_CVMFS_REPO="${EESSI_CVMFS_REPO_OVERRIDE:=/cvmfs/pilot.nessi.no}"
      +export EESSI_CVMFS_REPO="${EESSI_CVMFS_REPO_OVERRIDE:=/cvmfs/software.eessi.io}"
       export EESSI_VERSION="${EESSI_VERSION_OVERRIDE:=2023.06}"
       # use archdetect by default, unless otherwise specified
      -# 2024-01-16: switch back to archspec with NESSI for now
      -export EESSI_USE_ARCHDETECT="${EESSI_USE_ARCHDETECT:=0}"
      -export EESSI_USE_ARCHSPEC="${EESSI_USE_ARCHSPEC:=1}"
      +export EESSI_USE_ARCHDETECT="${EESSI_USE_ARCHDETECT:=1}"
      +export EESSI_USE_ARCHSPEC="${EESSI_USE_ARCHSPEC:=0}"
    • init/eessi_environment_variables

      diff
      diff --git a/init/eessi_environment_variables b/init/eessi_environment_variables
      index d4a2e72..3584bfa 100644
      --- a/init/eessi_environment_variables
      +++ b/init/eessi_environment_variables
      @@ -2,6 +2,11 @@
       # $BASH_SOURCE points to correct path, see also http://mywiki.wooledge.org/BashFAQ/028
       EESSI_INIT_DIR_PATH=$(dirname $(realpath $BASH_SOURCE))
      
      +function error() {
      +    echo -e "\e[31mERROR: $1\e[0m" >&2
      +    false
      +}
      +
       function show_msg {
         # only echo msg if EESSI_SILENT is unset
         msg=$1
      @@ -10,16 +15,11 @@ function show_msg {
         fi
       }
      
      -function error() {
      -    echo -e "\e[31mERROR: $1\e[0m" >&2
      -    false
      -}
      -
       # set up minimal environment: $EESSI_PREFIX, $EESSI_VERSION, $EESSI_OS_TYPE, $EESSI_CPU_FAMILY, $EPREFIX
       source $EESSI_INIT_DIR_PATH/minimal_eessi_env
      
       if [ -d $EESSI_PREFIX ]; then
      -  show_msg "Found NESSI repo @ $EESSI_PREFIX!"
      +  show_msg "Found EESSI repo @ $EESSI_PREFIX!"
      
         export EESSI_EPREFIX=$EPREFIX
         if [ -d $EESSI_EPREFIX ]; then
      @@ -63,7 +63,7 @@ if [ -d $EESSI_PREFIX ]; then
                   export EESSI_MODULEPATH=$EESSI_MODULEPATH
                   show_msg "Using ${EESSI_MODULEPATH} as the directory to be added to MODULEPATH."
                 else
      -            error "NESSI module path at $EESSI_MODULEPATH not found!"
      +            error "EESSI module path at $EESSI_MODULEPATH not found!"
                   false
                 fi
      
      @@ -75,7 +75,7 @@ if [ -d $EESSI_PREFIX ]; then
                 fi
      
               else
      -          error "NESSI software layer at $EESSI_SOFTWARE_PATH not found!"
      +          error "EESSI software layer at $EESSI_SOFTWARE_PATH not found!"
               fi
           else
               error "no value set for \$EESSI_SOFTWARE_SUBDIR"
      @@ -84,5 +84,5 @@ if [ -d $EESSI_PREFIX ]; then
           error "Compatibility layer directory $EESSI_EPREFIX not found!"
         fi
       else
      -  error "NESSI repository at $EESSI_PREFIX not found!"
      +  error "EESSI repository at $EESSI_PREFIX not found!"
       fi
    • init/eessi_software_subdir_for_host.py

      diff
      diff --git a/init/eessi_software_subdir_for_host.py b/init/eessi_software_subdir_for_host.py
      index e8f1a71..58e9cfd 100755
      --- a/init/eessi_software_subdir_for_host.py
      +++ b/init/eessi_software_subdir_for_host.py
      @@ -1,6 +1,6 @@
       #!/usr/bin/env python3
       #
      -# Determine NESSI software subdirectory to use for current build host, using archspec
      +# Determine EESSI software subdirectory to use for current build host, using archspec
       #
       import glob
       import os
    • init/minimal_eessi_env

      diff
      diff --git a/init/minimal_eessi_env b/init/minimal_eessi_env
      index f938d1d..5b35620 100644
      --- a/init/minimal_eessi_env
      +++ b/init/minimal_eessi_env
      @@ -1,4 +1,4 @@
      -# define minimal NESSI environment, without relying on external scripts
      +# define minimal EESSI environment, without relying on external scripts
       #
       # this script is *sourced*, not executed, so can't rely on $0 to determine path to self
       # $BASH_SOURCE points to correct path, see also http://mywiki.wooledge.org/BashFAQ/028
  • sync scripts and init directories of /cvmfs/pilot.nessi.no with changes in /cvmfs/software.eessi.io

    • CVMFS repo pilot.nessi.no synced with EESSI:
      • removed symlink /cvmfs/pilot.nessi.no/latest
      • added /cvmfs/pilot.nessi.no/README.nessi
      • synced/populated /cvmfs/pilot.nessi.no/versions/2023.06/{init,scripts}
      • should be available soon on CVMFS clients
  • check Broadwell supported via archdetect (for building packages)

  • check archdetect is used correctly via source /cvmfs/.../init/bash (for using packages)

    • works for Broadwell
    [thomarob@login-2.FRAM ~]$ source /cvmfs/pilot.nessi.no/versions/2023.06/init/bash
    Found NESSI repo @ /cvmfs/pilot.nessi.no/versions/2023.06!
    archdetect says x86_64/intel/broadwell
    Using x86_64/intel/broadwell as software subdirectory.
    Using /cvmfs/pilot.nessi.no/versions/2023.06/software/linux/x86_64/intel/broadwell/modules/all as the directory to be added to MODULEPATH.
    Found Lmod configuration file at /cvmfs/pilot.nessi.no/versions/2023.06/software/linux/x86_64/intel/broadwell/.lmod/lmodrc.lua
    Initializing Lmod...
    Prepending /cvmfs/pilot.nessi.no/versions/2023.06/software/linux/x86_64/intel/broadwell/modules/all to $MODULEPATH...
    Environment set up to use NESSI (2023.06), have fun!
    
  • fix failing CI workflows

@TopRichard
Copy link
Author

TopRichard commented Jan 22, 2024

First complete pass by (except for some easystack files) is done. Remaining todos:

  • do second pass by Richard on current differences
    When only differences due to repo naming, repo config, supported CPU architectures --> ✅ is set.

    git diff nessi_2023.06 2023.06-software.eessi.io
    • .github/workflows/test-pilot.nessi.no.yml

      diff
      diff --git a/.github/workflows/test-pilot.nessi.no.yml b/.github/workflows/test-software.eessi.io.yml
      similarity index 76%
      rename from .github/workflows/test-pilot.nessi.no.yml
      rename to .github/workflows/test-software.eessi.io.yml
      index 6342d5d..3ac341a 100644
      --- a/.github/workflows/test-pilot.nessi.no.yml
      +++ b/.github/workflows/test-software.eessi.io.yml
      @@ -1,5 +1,5 @@
       # documentation: https://help.github.com/en/articles/workflow-syntax-for-github-actions
      -name: Check for missing software installations in pilot.nessi.no
      +name: Check for missing software installations in software.eessi.io
       on: [push, pull_request, workflow_dispatch]
       permissions:
         contents: read # to fetch code (actions/checkout)
      @@ -12,37 +12,40 @@ jobs:
               EESSI_VERSION:
               - 2023.06
               EESSI_SOFTWARE_SUBDIR_OVERRIDE:
      -#        - aarch64/generic
      +        - aarch64/generic
      +        - aarch64/neoverse_n1
      +        - aarch64/neoverse_v1
               - x86_64/amd/zen2
      -        - x86_64/intel/broadwell
      +        - x86_64/amd/zen3
      +        - x86_64/intel/haswell
               - x86_64/intel/skylake_avx512
               - x86_64/generic
           steps:
               - name: Check out software-layer repository
                 uses: actions/checkout@93ea575cb5d8a053eaa0ac8fa3b40d7e05a33cc8 # v3.1.0
      
      -        - name: Mount NESSI CernVM-FS repository
      +        - name: Mount EESSI CernVM-FS pilot repository
                 uses: cvmfs-contrib/github-action-cvmfs@d4641d0d591c9a5c3be23835ced2fb648b44c04b # v3.1
                 with:
      -              cvmfs_config_package: https://github.com/NorESSI/filesystem-layer/releases/download/latest/cvmfs-config-nessi_latest_all.deb
      +              cvmfs_config_package: https://github.com/EESSI/filesystem-layer/releases/download/latest/cvmfs-config-eessi_latest_all.deb
                     cvmfs_http_proxy: DIRECT
      -              cvmfs_repositories: pilot.nessi.no
      +              cvmfs_repositories: software.eessi.io
      
               - name: Test check_missing_installations.sh script
                 run: |
                     export EESSI_SOFTWARE_SUBDIR_OVERRIDE=${{matrix.EESSI_SOFTWARE_SUBDIR_OVERRIDE}}
      -              source /cvmfs/pilot.nessi.no/versions/${{matrix.EESSI_VERSION}}/init/bash
      +              source /cvmfs/software.eessi.io/versions/${{matrix.EESSI_VERSION}}/init/bash
                     # set $EESSI_CPU_FAMILY to the CPU architecture that corresponds to $EESSI_SOFTWARE_SUBDIR_OVERRIDE (part before the first slash),
                     # to prevent issues with checks in the Easybuild configuration that use this variable
                     export EESSI_CPU_FAMILY=${EESSI_SOFTWARE_SUBDIR_OVERRIDE%%/*}
                     module load EasyBuild
                     which eb
                     eb --version
      -              export EESSI_PREFIX=/cvmfs/pilot.nessi.no/versions/${{matrix.EESSI_VERSION}}
      +              export EESSI_PREFIX=/cvmfs/software.eessi.io/versions/${{matrix.EESSI_VERSION}}
                     export EESSI_OS_TYPE=linux
                     env | grep ^EESSI | sort
      -              echo "just run check_missing_installations.sh (should use easystacks/pilot.nessi.no/${{matrix.EESSI_VERSION}}/eessi-${{matrix.EESSI_VERSION}}-*.yml)"
      -              for easystack_file in $(ls easystacks/pilot.nessi.no/${{matrix.EESSI_VERSION}}/eessi-${{matrix.EESSI_VERSION}}-eb-*.yml); do
      +              echo "just run check_missing_installations.sh (should use easystacks/software.eessi.io/${{matrix.EESSI_VERSION}}/eessi-${{matrix.EESSI_VERSION}}-*.yml)"
      +              for easystack_file in $(ls easystacks/software.eessi.io/${{matrix.EESSI_VERSION}}/eessi-${{matrix.EESSI_VERSION}}-eb-*.yml); do
                         echo "check missing installations for ${easystack_file}..."
                         ./check_missing_installations.sh ${easystack_file}
                         ec=$?
      @@ -52,14 +55,14 @@ jobs:
               - name: Test check_missing_installations.sh with missing package (GCC/8.3.0)
                 run: |
                     export EESSI_SOFTWARE_SUBDIR_OVERRIDE=${{matrix.EESSI_SOFTWARE_SUBDIR_OVERRIDE}}
      -              source /cvmfs/pilot.nessi.no/versions/${{matrix.EESSI_VERSION}}/init/bash
      +              source /cvmfs/software.eessi.io/versions/${{matrix.EESSI_VERSION}}/init/bash
                     # set $EESSI_CPU_FAMILY to the CPU architecture that corresponds to $EESSI_SOFTWARE_SUBDIR_OVERRIDE (part before the first slash),
                     # to prevent issues with checks in the Easybuild configuration that use this variable
                     export EESSI_CPU_FAMILY=${EESSI_SOFTWARE_SUBDIR_OVERRIDE%%/*}
                     module load EasyBuild
                     which eb
                     eb --version
      -              export EESSI_PREFIX=/cvmfs/pilot.nessi.no/versions/${{matrix.EESSI_VERSION}}
      +              export EESSI_PREFIX=/cvmfs/software.eessi.io/versions/${{matrix.EESSI_VERSION}}
                     export EESSI_OS_TYPE=linux
                     env | grep ^EESSI | sort
                     # create dummy easystack file with a single entry (something that is not installed in EESSI)
    • .github/workflows/test_eessi_container_script.yml

      diff
      diff --git a/.github/workflows/test_eessi_container_script.yml b/.github/workflows/test_eessi_container_script.yml
      index 3bb67b4..33122e6 100644
      --- a/.github/workflows/test_eessi_container_script.yml
      +++ b/.github/workflows/test_eessi_container_script.yml
      @@ -90,15 +90,15 @@ jobs:
                     elif [[ ${{matrix.SCRIPT_TEST}} == 'readwrite' ]]; then
                       outfile=out_readwrite.txt
                       fn="test_${RANDOM}.txt"
      -                echo "touch /cvmfs/pilot.nessi.no/${fn}" > test_script.sh
      +                echo "touch /cvmfs/software.eessi.io/${fn}" > test_script.sh
                       chmod u+x test_script.sh
                       export SINGULARITY_BIND="$PWD:/test"
                       ./eessi_container.sh --verbose --access rw --mode run /test/test_script.sh > ${outfile}
      
                       tmpdir=$(grep "\-\-resume" ${outfile} | sed "s/.*--resume \([^']*\).*/\1/g")
                       # note: must use '--access rw' again here, since touched file is in overlay upper dir
      -                ./eessi_container.sh --verbose --resume ${tmpdir} --access rw --mode shell <<< "ls -l /cvmfs/pilot.nessi.no/${fn}" > ${outfile}
      -                grep "/cvmfs/pilot.nessi.no/${fn}$" $outfile
      +                ./eessi_container.sh --verbose --resume ${tmpdir} --access rw --mode shell <<< "ls -l /cvmfs/software.eessi.io/${fn}" > ${outfile}
      +                grep "/cvmfs/software.eessi.io/${fn}$" $outfile
      
                     # test use of --resume
                     elif [[ ${{matrix.SCRIPT_TEST}} == 'resume' ]]; then
      @@ -120,12 +120,12 @@ jobs:
                     elif [[ ${{matrix.SCRIPT_TEST}} == 'save' ]]; then
                       outfile=out_save.txt
                       fn="test_${RANDOM}.txt"
      -                test_cmd="touch /cvmfs/pilot.nessi.no/${fn}"
      +                test_cmd="touch /cvmfs/software.eessi.io/${fn}"
                       ./eessi_container.sh --verbose --mode shell --access rw --save test-save.tar <<< "${test_cmd}" 2>&1 | tee ${outfile}
                       rm -f ${outfile}
      
      -                ./eessi_container.sh --verbose --mode shell --access rw --resume test-save.tar <<< "ls -l /cvmfs/pilot.nessi.no/${fn}" > ${outfile}
      -                grep "/cvmfs/pilot.nessi.no/${fn}$" $outfile
      +                ./eessi_container.sh --verbose --mode shell --access rw --resume test-save.tar <<< "ls -l /cvmfs/software.eessi.io/${fn}" > ${outfile}
      +                grep "/cvmfs/software.eessi.io/${fn}$" $outfile
      
                       tar tfv test-save.tar | grep "overlay-upper/${fn}"
      
    • .github/workflows/tests_archdetect.yml

      diff
      diff --git a/.github/workflows/tests_archdetect.yml b/.github/workflows/tests_archdetect.yml
      index 74dbf03..1e8b830 100644
      --- a/.github/workflows/tests_archdetect.yml
      +++ b/.github/workflows/tests_archdetect.yml
      @@ -13,23 +13,23 @@ jobs:
                 - x86_64/intel/skylake_avx512/archspec-linux-6132
                 - x86_64/amd/zen2/Azure-CentOS7-7V12
                 - x86_64/amd/zen3/Azure-CentOS7-7V73X
      -          # commented out since these targets are currently not supported in pilot.nessi.no repo
      +          - aarch64/neoverse_n1/Azure-Ubuntu20-Altra
      +          - aarch64/neoverse_n1/AWS-awslinux-graviton2
      +          - aarch64/neoverse_v1/AWS-awslinux-graviton3
      +          # commented out since these targets are currently not supported in software.eessi.io repo
                 # (and some tests assume that the corresponding subdirectory in software layer is there)
                 # - ppc64le/power9le/unknown-power9le
      -          # - aarch64/neoverse-n1/Azure-Ubuntu20-Altra
      -          # - aarch64/neoverse-n1/AWS-awslinux-graviton2
      -          # - aarch64/neoverse-v1/AWS-awslinux-graviton3
             fail-fast: false
           steps:
           - name: checkout
             uses: actions/checkout@93ea575cb5d8a053eaa0ac8fa3b40d7e05a33cc8 # v3.1.0
      
      -    - name: Mount NESSI CernVM-FS repository
      +    - name: Mount EESSI CernVM-FS pilot repository
             uses: cvmfs-contrib/github-action-cvmfs@d4641d0d591c9a5c3be23835ced2fb648b44c04b # v3.1
             with:
      -          cvmfs_config_package: https://github.com/NorESSI/filesystem-layer/releases/download/latest/cvmfs-config-nessi_latest_all.deb
      +          cvmfs_config_package: https://github.com/EESSI/filesystem-layer/releases/download/latest/cvmfs-config-eessi_latest_all.deb
                 cvmfs_http_proxy: DIRECT
      -          cvmfs_repositories: pilot.nessi.no
      +          cvmfs_repositories: software.eessi.io
      
           - name: test eessi_archdetect.sh
             run: |
    • EESSI-install-software.sh

      diff
      diff --git a/EESSI-install-software.sh b/EESSI-install-software.sh
      index eb616b1..69de9d1 100755
      --- a/EESSI-install-software.sh
      +++ b/EESSI-install-software.sh
      @@ -1,6 +1,6 @@
       #!/bin/bash
       #
      -# Script to install NESSI software stack (version set through init/eessi_defaults)
      +# Script to install EESSI software stack (version set through init/eessi_defaults)
      
       # see example parsing of command line arguments at
       #   https://wiki.bash-hackers.org/scripting/posparams#using_a_while_loop
    • README.md

      diff
      diff --git a/README.md b/README.md
      index 4f06139..ab73ad7 100644
      --- a/README.md
      +++ b/README.md
      @@ -1,25 +1,25 @@
       # Software layer
      
      -The software layer of the NESSI project uses [EasyBuild](https://docs.easybuild.io), [Lmod](https://lmod.readthedocs.io) and [archspec](https://archspec.readthedocs.io).
      +The software layer of the EESSI project uses [EasyBuild](https://docs.easybuild.io), [Lmod](https://lmod.readthedocs.io) and [archspec](https://archspec.readthedocs.io).
      
       See also https://www.eessi.io/docs/software_layer .
      
      -## Accessing the software stack
      +## Pilot software stack
      
       You can set up your environment by sourcing the init script:
      

      -$ source /cvmfs/pilot.nessi.no/versions/2023.06/init/bash
      -Found NESSI repo @ /cvmfs/pilot.nessi.no/versions/2023.06!
      -Derived subdirectory for software layer: x86_64/intel/broadwell
      +$ source /cvmfs/software.eessi.io/versions/2023.06/init/bash
      +Found EESSI repo @ /cvmfs/software.eessi.io/versions/2023.06!
      +Derived subdirectory for software layer: x86_64/intel/haswell
      Using x86_64/intel/haswell subdirectory for software layer
      Initializing Lmod...
      -Prepending /cvmfs/pilot.nessi.no/versions/2023.06/software/x86_64/intel/broadwell/modules/all to $MODULEPATH...
      -Environment set up to use NESSI software stack, have fun!
      -{NESSI 2023.06} $
      +Prepending /cvmfs/software.eessi.io/versions/2023.06/software/x86_64/intel/haswell/modules/all to $MODULEPATH...
      +Environment set up to use EESSI (2023.06), have fun!
      +[EESSI 2023.06] $

      
      -### Accessing NESSI via a container
      +### Accessing EESSI via a container
      
      You need Singularity version 3.7 or newer. Then, simply run
      
      @@ -40,6 +40,6 @@ Note, not all features/arguments listed via `./eessi_container.sh --help` are im
      The software in this repository is distributed under the terms of the
      [GNU General Public License v2.0](https://opensource.org/licenses/GPL-2.0).
      
      -See [LICENSE](https://github.com/NorESSI/software-layer/blob/main/LICENSE) for more information.
      +See [LICENSE](https://github.com/EESSI/software-layer/blob/main/LICENSE) for more information.
      
      SPDX-License-Identifier: GPL-2.0-only
      
    • bot/build.sh

      diff
      diff --git a/bot/build.sh b/bot/build.sh
      index 7eb15f3..1622e75 100755
      --- a/bot/build.sh
      +++ b/bot/build.sh
      @@ -128,7 +128,7 @@ EESSI_REPOS_CFG_DIR_OVERRIDE=$(cfg_get_value "repository" "repos_cfg_dir")
       export EESSI_REPOS_CFG_DIR_OVERRIDE=${EESSI_REPOS_CFG_DIR_OVERRIDE:-${PWD}/cfg}
       echo "bot/build.sh: EESSI_REPOS_CFG_DIR_OVERRIDE='${EESSI_REPOS_CFG_DIR_OVERRIDE}'"
      
      -# determine NESSI version to be used from .repository.repo_version in ${JOB_CFG_FILE}
      +# determine EESSI version to be used from .repository.repo_version in ${JOB_CFG_FILE}
       # here, just set & export EESSI_VERSION_OVERRIDE
       # next script (eessi_container.sh) makes use of it via sourcing init scripts
       # (e.g., init/eessi_defaults or init/minimal_eessi_env)
    • build_container.sh 👎

      diff
      diff --git a/build_container.sh b/build_container.sh
      new file mode 100755
      index 0000000..23a9e66
      --- /dev/null
      +++ b/build_container.sh
      @@ -0,0 +1,69 @@
      +#!/bin/bash
      +
      +base_dir=$(dirname $(realpath $0))
      +
      +BUILD_CONTAINER="docker://ghcr.io/eessi/build-node:debian11"
      +
      +if [ $# -lt 2 ]; then
      +    echo "Usage: $0 <shell|run> <path for temporary directories>" >&2
      +    exit 1
      +fi
      +SHELL_OR_RUN=$1
      +EESSI_TMPDIR=$2
      +shift 2
      +
      +if [ "$SHELL_OR_RUN" == "run" ] && [ $# -eq 0 ]; then
      +    echo "ERROR: No command specified to run?!" >&2
      +    exit 1
      +fi
      +
      +# make sure specified temporary directory exists
      +mkdir -p $EESSI_TMPDIR
      +
      +echo "Using $EESSI_TMPDIR as parent for temporary directories..."
      +
      +# create temporary directories
      +mkdir -p $EESSI_TMPDIR/{home,overlay-upper,overlay-work}
      +mkdir -p $EESSI_TMPDIR/{var-lib-cvmfs,var-run-cvmfs}
      +# configure Singularity
      +export SINGULARITY_CACHEDIR=$EESSI_TMPDIR/singularity_cache
      +
      +# take into account that $SINGULARITY_BIND may be defined already, to bind additional paths into the build container
      +BIND_PATHS="$EESSI_TMPDIR/var-run-cvmfs:/var/run/cvmfs,$EESSI_TMPDIR/var-lib-cvmfs:/var/lib/cvmfs,$EESSI_TMPDIR"
      +if [ -z $SINGULARITY_BIND ]; then
      +    export SINGULARITY_BIND="$BIND_PATHS"
      +else
      +    export SINGULARITY_BIND="$SINGULARITY_BIND,$BIND_PATHS"
      +fi
      +
      +# allow that SINGULARITY_HOME is defined before script is run
      +if [ -z $SINGULARITY_HOME ]; then
      +    export SINGULARITY_HOME="$EESSI_TMPDIR/home:/home/$USER"
      +fi
      +
      +source ${base_dir}/init/eessi_defaults
      +# strip "/cvmfs/" from default setting
      +repo_name=${EESSI_CVMFS_REPO/\/cvmfs\//}
      +
      +# set environment variables for fuse mounts in Singularity container
      +export EESSI_PILOT_READONLY="container:cvmfs2 ${repo_name} /cvmfs_ro/${repo_name}"
      +export EESSI_PILOT_WRITABLE_OVERLAY="container:fuse-overlayfs -o lowerdir=/cvmfs_ro/${repo_name} -o upperdir=$EESSI_TMPDIR/overlay-upper -o workdir=$EESSI_TMPDIR/overlay-work ${EESSI_CVMFS_REPO}"
      +
      +# pass $EESSI_SOFTWARE_SUBDIR_OVERRIDE into build container (if set)
      +if [ ! -z ${EESSI_SOFTWARE_SUBDIR_OVERRIDE} ]; then
      +    export SINGULARITYENV_EESSI_SOFTWARE_SUBDIR_OVERRIDE=${EESSI_SOFTWARE_SUBDIR_OVERRIDE}
      +    # also specify via $APPTAINERENV_* (future proof, cfr. https://apptainer.org/docs/user/latest/singularity_compatibility.html#singularity-environment-variable-compatibility)
      +    export APPTAINERENV_EESSI_SOFTWARE_SUBDIR_OVERRIDE=${EESSI_SOFTWARE_SUBDIR_OVERRIDE}
      +fi
      +
      +if [ "$SHELL_OR_RUN" == "shell" ]; then
      +    # start shell in Singularity container, with EESSI repository mounted with writable overlay
      +    echo "Starting Singularity build container..."
      +    singularity shell --fusemount "$EESSI_PILOT_READONLY" --fusemount "$EESSI_PILOT_WRITABLE_OVERLAY" $BUILD_CONTAINER
      +elif [ "$SHELL_OR_RUN" == "run" ]; then
      +    echo "Running '$@' in Singularity build container..."
      +    singularity exec --fusemount "$EESSI_PILOT_READONLY" --fusemount "$EESSI_PILOT_WRITABLE_OVERLAY" $BUILD_CONTAINER "$@"
      +else
      +    echo "ERROR: Unknown action specified: $SHELL_OR_RUN (should be either 'shell' or 'run')" >&2
      +    exit 1
      +fi

      NOTE:The file is removed intentionally by Thomas

    • check_missing_installations.sh

      diff
      diff --git a/check_missing_installations.sh b/check_missing_installations.sh
      index dd292e4..5ea7c5a 100755
      --- a/check_missing_installations.sh
      +++ b/check_missing_installations.sh
      @@ -1,6 +1,6 @@
       #!/bin/bash
       #
      -# Script to check for missing installations in NESSI software stack (version 2023.06)
      +# Script to check for missing installations in EESSI software stack
       #
       # author: Kenneth Hoste (@boegel)
       # author: Thomas Roeblitz (@trz42)
    • configure_easybuild

      diff
      diff --git a/configure_easybuild b/configure_easybuild
      index c1bd1d3..c67b879 100644
      --- a/configure_easybuild
      +++ b/configure_easybuild
      @@ -25,7 +25,6 @@ fi
      
       # note: filtering Bison may break some installations, like Qt5 (see https://github.com/EESSI/software-layer/issues/49)
       # filtering pkg-config breaks R-bundle-Bioconductor installation (see also https://github.com/easybuilders/easybuild-easyconfigs/pull/11104)
      -# problems occur when filtering pkg-config with gnuplot too (picks up Lua 5.1 from $EPREFIX rather than from Lua 5.3 dependency)
       DEPS_TO_FILTER=Autoconf,Automake,Autotools,binutils,bzip2,DBus,flex,gettext,gperf,help2man,intltool,libreadline,libtool,M4,makeinfo,ncurses,util-linux,XZ,zlib
       # For aarch64 we need to also filter out Yasm.
       # See https://github.com/easybuilders/easybuild-easyconfigs/issues/11190
    • create_directory_tarballs.sh

      diff
      diff --git a/create_directory_tarballs.sh b/create_directory_tarballs.sh
      index a1f05b5..0270719 100755
      --- a/create_directory_tarballs.sh
      +++ b/create_directory_tarballs.sh
      @@ -9,7 +9,7 @@ fi
      
       version=$1
      
      -SOFTWARE_LAYER_TARBALL_URL=https://github.com/NorESSI/software-layer/tarball/nessi.no-2023.06
      +SOFTWARE_LAYER_TARBALL_URL="https://github.com/EESSI/software-layer/tarball/${version}-software.eessi.io"
      
       TOPDIR=$(dirname $(realpath $0))
      
    • create_tarball.sh 👎

      diff
      diff --git a/create_tarball.sh b/create_tarball.sh
      index f6239cf..a619df9 100755
      --- a/create_tarball.sh
      +++ b/create_tarball.sh
      @@ -43,7 +43,7 @@ module_files_list=${tmpdir}/module_files.list.txt
       if [ -d ${eessi_version}/software/${os}/${cpu_arch_subdir}/.lmod ]; then
           # include Lmod cache and configuration file (lmodrc.lua),
           # skip whiteout files and backup copies of Lmod cache (spiderT.old.*)
      -    find ${eessi_version}/software/${os}/${cpu_arch_subdir}/.lmod -type f | egrep -v '/\.wh\.|spiderT.old' > ${files_list}
      +    find ${eessi_version}/software/${os}/${cpu_arch_subdir}/.lmod -type f | egrep -v '/\.wh\.|spiderT.old' >> ${files_list}
       fi
      
       # include scripts that were copied by install_scripts.sh, which we want to ship in EESSI repository

      NOTE: Modify > --> >>

    • easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.7.2-2021a.yml

      diff
      diff --git a/easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.7.2-2021a.yml b/easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.7.2-2021a.yml
      deleted file mode 100644
      index 28612c5..0000000
      --- a/easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.7.2-2021a.yml
      +++ /dev/null
      @@ -1,27 +0,0 @@
      -easyconfigs:
      -  - OpenSSL-1.1.eb
      -  # - OpenSSL-1.1.eb:
      -  #     options:
      -  #       include-easyblocks-from-pr: 2922
      -  - git-2.32.0-GCCcore-10.3.0-nodocs.eb
      -  - GCC-10.3.0
      -  - CMake-3.20.1-GCCcore-10.3.0.eb
      -  # - CMake-3.20.1-GCCcore-10.3.0.eb:
      -  #     options:
      -  #       include-easyblocks-from-pr: 2248
      -  - Rust-1.52.1-GCCcore-10.3.0.eb
      -  - foss-2021a.eb
      -  - QuantumESPRESSO-6.7-foss-2021a.eb
      -  - GROMACS-2021.3-foss-2021a.eb:
      -       options:
      -         download-timeout: 1000
      -  - libGLU-9.0.1-GCCcore-10.3.0.eb
      -  # since the BAGEL ec file is new we have to keep the from-pr or eb fails
      -  - BAGEL-1.2.2-foss-2021a.eb:
      -        options:
      -          from-pr: 18446
      -  - WRF-4.3-foss-2021a-dmpar.eb
      -  - Yambo-5.1.1-foss-2021a.eb:
      -        options:
      -          from-pr: 18905
      -
    • easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.7.2-2021b.yml

      diff
      diff --git a/easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.7.2-2021b.yml b/easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.7.2-2021b.yml
      deleted file mode 100644
      index d838e15..0000000
      --- a/easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.7.2-2021b.yml
      +++ /dev/null
      @@ -1,18 +0,0 @@
      -easyconfigs:
      -  - GCC-11.2.0
      -  - git-2.33.1-GCCcore-11.2.0-nodocs.eb
      -  - CMake-3.21.1-GCCcore-11.2.0.eb
      -  # - CMake-3.21.1-GCCcore-11.2.0.eb:
      -  #     options:
      -  #       include-easyblocks-from-pr: 2248
      -  - OpenMPI-4.1.1-GCC-11.2.0.eb
      -  - FFTW-3.3.10-gompi-2021b.eb
      -  - BLIS-0.8.1-GCC-11.2.0.eb
      -  - foss-2021b.eb
      -  # note, the below CMake has never been installed, just put here to use the
      -  # same version as easy does
      -  # - CMake-3.22.1-GCCcore-11.2.0.eb:
      -  #     options:
      -  #       include-easyblocks-from-pr: 2248
      -  # - zlib-1.2.11-GCCcore-11.2.0
      -  - QuantumESPRESSO-6.8-foss-2021b.eb
    • easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.7.2-2022a.yml

      diff
      diff --git a/easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.7.2-2022a.yml b/easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.7.2-2022a.yml
      deleted file mode 100644
      index 9991de1..0000000
      --- a/easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.7.2-2022a.yml
      +++ /dev/null
      @@ -1,12 +0,0 @@
      -easyconfigs:
      -  - GCC-11.3.0
      -  - git-2.36.0-GCCcore-11.3.0-nodocs.eb
      -  - CMake-3.23.1-GCCcore-11.3.0.eb
      -  # - CMake-3.23.1-GCCcore-11.3.0.eb:
      -  #     options:
      -  #       include-easyblocks-from-pr: 2248
      -  - BLIS-0.9.0-GCC-11.3.0.eb
      -  - OpenMPI-4.1.4-GCC-11.3.0.eb
      -  - FFTW.MPI-3.3.10-gompi-2022a.eb
      -  - foss-2022a.eb
      -  - Python-3.10.4-GCCcore-11.3.0.eb
    • easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.7.2-2022b.yml

      diff
      diff --git a/easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.7.2-2022b.yml b/easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.7.2-2022b.yml
      deleted file mode 100644
      index 129b709..0000000
      --- a/easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.7.2-2022b.yml
      +++ /dev/null
      @@ -1,12 +0,0 @@
      -easyconfigs:
      -  - GCC-12.2.0
      -  - git-2.38.1-GCCcore-12.2.0-nodocs.eb
      -  - CMake-3.24.3-GCCcore-12.2.0.eb
      -  # - CMake-3.24.3-GCCcore-12.2.0.eb:
      -  #     options:
      -  #       include-easyblocks-from-pr: 2248
      -  - BLIS-0.9.0-GCC-12.2.0.eb
      -  - OpenMPI-4.1.4-GCC-12.2.0.eb
      -  - FFTW.MPI-3.3.10-gompi-2022b.eb
      -  - Python-3.10.8-GCCcore-12.2.0.eb
      -  - foss-2022b.eb
    • easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.7.2-system.yml

      diff
      diff --git a/easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.7.2-system.yml b/easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.7.2-system.yml
      deleted file mode 100644
      index a5f84c7..0000000
      --- a/easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.7.2-system.yml
      +++ /dev/null
      @@ -1,4 +0,0 @@
      -easyconfigs:
      -  - EasyBuild-4.8.0.eb:
      -      options:
      -        from-pr: 18282
    • easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.8.0-system.yml

      diff
      diff --git a/easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.8.0-system.yml b/easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.8.0-system.yml
      deleted file mode 100644
      index 96d79de..0000000
      --- a/easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.8.0-system.yml
      +++ /dev/null
      @@ -1,4 +0,0 @@
      -easyconfigs:
      -  - EasyBuild-4.8.1.eb:
      -      options:
      -        from-pr: 18761
    • easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.8.1-2022a.yml

      diff
      diff --git a/easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.8.1-2022a.yml b/easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.8.1-2022a.yml
      deleted file mode 100644
      index 14f936b..0000000
      --- a/easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.8.1-2022a.yml
      +++ /dev/null
      @@ -1,66 +0,0 @@
      -easyconfigs:
      -  - BAMM-2.5.0-foss-2022a.eb
      -  - BamTools-2.5.2-GCC-11.3.0.eb
      -  - BCFtools-1.15.1-GCC-11.3.0.eb
      -  - Bowtie2-2.4.5-GCC-11.3.0.eb
      -  - FastTree-2.1.11-GCCcore-11.3.0.eb
      -  - SAMtools-1.16.1-GCC-11.3.0.eb
      -  - gzip-1.12-GCCcore-11.3.0.eb
      -  - lz4-1.9.3-GCCcore-11.3.0.eb
      -  - zstd-1.5.2-GCCcore-11.3.0.eb
      -  - Pillow-9.1.1-GCCcore-11.3.0.eb:
      -      # avoid that hardcoded paths like /usr/include are used in build commands
      -        options:
      -          from-pr: 18881
      -  - Boost.Python-1.79.0-GCC-11.3.0.eb
      -  - netCDF-4.9.0-gompi-2022a.eb:
      -      # use updated CMakeMake easyblock to avoid that -DCMAKE_SKIP_RPATH=ON is used, which breaks the netCDF test step
      -      # see https://github.com/easybuilders/easybuild-easyblocks/pull/3012
      -      options:
      -        include-easyblocks-from-pr: 3012
      -  - NCO-5.1.0-foss-2022a.eb
      -  - AdapterRemoval-2.3.3-GCC-11.3.0.eb
      -  - BEDTools-2.30.0-GCC-11.3.0.eb
      -  - Pysam-0.20.0-GCC-11.3.0.eb
      -  - TransDecoder-5.5.0-GCC-11.3.0.eb
      -  - VCFtools-0.1.16-GCC-11.3.0.eb
      -  - XML-LibXML-2.0207-GCCcore-11.3.0.eb
      -  - CMake-3.24.3-GCCcore-11.3.0.eb
      -  - elfutils-0.187-GCCcore-11.3.0.eb
      -  - Ninja-1.10.2-GCCcore-11.3.0.eb
      -  - Z3-4.10.2-GCCcore-11.3.0.eb
      -  - SciPy-bundle-2022.05-foss-2022a.eb
      -  - Xvfb-21.1.3-GCCcore-11.3.0.eb:
      -      # enable exec permissions for xvfb-run after copying;
      -      # need to also enable user write permissions on xvfb-run to ensure that copying with preserved permissions works
      -      options:
      -        from-pr: 18834
      -  - tbb-2021.5.0-GCCcore-11.3.0.eb
      -  - CMSeq-1.0.4-foss-2022a.eb
      -  - Pango-1.50.7-GCCcore-11.3.0.eb
      -  - nodejs-16.15.1-GCCcore-11.3.0.eb:
      -      options:
      -        download-timeout: 1000
      -  - ecCodes-2.27.0-gompi-2022a.eb:
      -        options:
      -          download-timeout: 1000
      -  - GLPK-5.0-GCCcore-11.3.0.eb
      -  - libgit2-1.4.3-GCCcore-11.3.0.eb
      -  - libGLU-9.0.2-GCCcore-11.3.0.eb
      -  - libsndfile-1.1.0-GCCcore-11.3.0.eb
      -  - LibTIFF-4.3.0-GCCcore-11.3.0.eb
      -  - MPFR-4.1.0-GCCcore-11.3.0.eb
      -  - NLopt-2.7.1-GCCcore-11.3.0.eb
      -  - PCRE2-10.40-GCCcore-11.3.0.eb
      -  - Tk-8.6.12-GCCcore-11.3.0.eb
      -  - GROMACS-2023.1-foss-2022a.eb
      -  - MUMPS-5.5.1-foss-2022a-metis.eb
      -  - Pillow-9.1.1-GCCcore-11.3.0.eb:
      -      # Uses a custom hook since has zlib as dependency which has hard coded header and library path within Pillow code.
      -      options:
      -        from-pr: 18881
      -  - ParaView-5.10.1-foss-2022a-mpi.eb:
      -        options:
      -          download-timeout: 1000
      -  - ASE-3.22.1-foss-2022a.eb
      -  - HISAT2-2.2.1-gompi-2022a.eb
    • easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.8.2-2022a.yml

      diff
      diff --git a/easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.8.2-2022a.yml b/easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.8.2-2022a.yml
      deleted file mode 100644
      index f191e76..0000000
      --- a/easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.8.2-2022a.yml
      +++ /dev/null
      @@ -1,3 +0,0 @@
      -easyconfigs:
      -  - CDO-2.0.6-gompi-2022a.eb
      -  - OpenBabel-3.1.1-gompi-2022a.eb
    • easystacks/pilot.nessi.no/2023.06/README.md

      diff
      diff --git a/easystacks/pilot.nessi.no/2023.06/README.md b/easystacks/software.eessi.io/2023.06/README.md
      similarity index 100%
      rename from easystacks/pilot.nessi.no/2023.06/README.md
      rename to easystacks/software.eessi.io/2023.06/README.md
    • easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.8.1-system.yml

      diff
      diff --git a/easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.8.1-system.yml b/easystacks/software.eessi.io/2023.06/eessi-2023.06-eb-4.8.2-001-system.yml
      similarity index 54%
      rename from easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.8.1-system.yml
      rename to easystacks/software.eessi.io/2023.06/eessi-2023.06-eb-4.8.2-001-system.yml
      index 7418c0a..f02b9f2 100644
      --- a/easystacks/pilot.nessi.no/2023.06/eessi-2023.06-eb-4.8.1-system.yml
      +++ b/easystacks/software.eessi.io/2023.06/eessi-2023.06-eb-4.8.2-001-system.yml
      @@ -1,6 +1,7 @@
       easyconfigs:
      -  # wraps around Java/11.0.20
      -  - Java-11.eb
         - EasyBuild-4.8.2.eb:
             options:
               from-pr: 19105
      +  - Nextflow-23.10.0.eb:
      +      options:
      +        from-pr: 19172
    • easystacks/software.eessi.io/2023.06/eessi-2023.06-eb-4.8.2-010-CUDA.yml

      diff
      diff --git a/easystacks/software.eessi.io/2023.06/eessi-2023.06-eb-4.8.2-010-CUDA.yml b/easystacks/software.eessi.io/2023.06/eessi-2023.06-eb-4.8.2-010-CUDA.yml
      new file mode 100644
      index 0000000..dda274b
      --- /dev/null
      +++ b/easystacks/software.eessi.io/2023.06/eessi-2023.06-eb-4.8.2-010-CUDA.yml
      @@ -0,0 +1,5 @@
      +easyconfigs:
      +  - CUDA-12.1.1.eb:
      +      options:
      +        include-easyblocks-from-pr: 3045
      +        accept-eula-for: CUDA
    • easystacks/software.eessi.io/2023.06/eessi-2023.06-eb-4.8.2-2022b.yml

      diff
      diff --git a/easystacks/software.eessi.io/2023.06/eessi-2023.06-eb-4.8.2-2022b.yml b/easystacks/software.eessi.io/2023.06/eessi-2023.06-eb-4.8.2-2022b.yml
      new file mode 100644
      index 0000000..69ffb75
      --- /dev/null
      +++ b/easystacks/software.eessi.io/2023.06/eessi-2023.06-eb-4.8.2-2022b.yml
      @@ -0,0 +1,6 @@
      +easyconfigs:
      +  - foss-2022b.eb
      +  - HarfBuzz-5.3.1-GCCcore-12.2.0.eb:
      +      options:
      +        from-pr: 19339
      +  - Qt5-5.15.7-GCCcore-12.2.0.eb
    • easystacks/software.eessi.io/2023.06/eessi-2023.06-eb-4.8.2-2023a.yml

      diff
      diff --git a/easystacks/software.eessi.io/2023.06/eessi-2023.06-eb-4.8.2-2023a.yml b/easystacks/software.eessi.io/2023.06/eessi-2023.06-eb-4.8.2-2023a.yml
      new file mode 100644
      index 0000000..aab910d
      --- /dev/null
      +++ b/easystacks/software.eessi.io/2023.06/eessi-2023.06-eb-4.8.2-2023a.yml
      @@ -0,0 +1,48 @@
      +easyconfigs:
      +  - GCC-12.3.0.eb
      +  - Rust-1.70.0-GCCcore-12.3.0.eb:
      +      # fix build of Rust 1.70.0 by disabling download of pre-built LLVM;
      +      # see https://github.com/easybuilders/easybuild-easyblocks/pull/3038
      +      options:
      +        include-easyblocks-from-pr: 3038
      +  - foss-2023a.eb
      +  - pybind11-2.11.1-GCCcore-12.3.0.eb:
      +      # avoid indirect dependency on old CMake version built with GCCcore/10.2.0 via Catch2 build dependency;
      +      # see https://github.com/easybuilders/easybuild-easyconfigs/pull/19270
      +      options:
      +        from-pr: 19270
      +  - SciPy-bundle-2023.07-gfbf-2023a.eb
      +  - TensorFlow-2.13.0-foss-2023a.eb:
      +      # patch setup.py for grpcio extension in TensorFlow 2.13.0 easyconfigs to take into account alternate sysroot;
      +      # see https://github.com/easybuilders/easybuild-easyconfigs/pull/19268
      +      options:
      +        from-pr: 19268
      +  - X11-20230603-GCCcore-12.3.0.eb
      +  - HarfBuzz-5.3.1-GCCcore-12.3.0.eb:
      +      options:
      +        from-pr: 19339
      +  - Qt5-5.15.10-GCCcore-12.3.0.eb
      +  - OSU-Micro-Benchmarks-7.1-1-gompi-2023a.eb
      +  - LHAPDF-6.5.4-GCC-12.3.0.eb:
      +      options:
      +        from-pr: 19363
      +  - LoopTools-2.15-GCC-12.3.0.eb:
      +      options:
      +        from-pr: 19397
      +  - R-4.3.2-gfbf-2023a.eb:
      +      options:
      +        from-pr: 19185
      +  - Boost-1.82.0-GCC-12.3.0.eb
      +  - netCDF-4.9.2-gompi-2023a.eb
      +  - FFmpeg-6.0-GCCcore-12.3.0.eb
      +  - CUDA-Samples-12.1-GCC-12.3.0-CUDA-12.1.1.eb:
      +      # use easyconfig that only install subset of CUDA samples,
      +      # to circumvent problem with nvcc linking to glibc of host OS,
      +      # see https://github.com/easybuilders/easybuild-easyconfigs/pull/19189;
      +      # and where additional samples are excluded because they fail to build on aarch64,
      +      # see https://github.com/easybuilders/easybuild-easyconfigs/pull/19451;
      +      options:
      +        from-pr: 19451
      +  - ALL-0.9.2-foss-2023a.eb:
      +      options:
      +        from-pr: 19455
    • easystacks/software.eessi.io/2023.06/eessi-2023.06-eb-4.9.0-001-system.yml

      diff
      diff --git a/easystacks/software.eessi.io/2023.06/eessi-2023.06-eb-4.9.0-001-system.yml b/easystacks/software.eessi.io/2023.06/eessi-2023.06-eb-4.9.0-001-system.yml
      new file mode 100644
      index 0000000..25c13e4
      --- /dev/null
      +++ b/easystacks/software.eessi.io/2023.06/eessi-2023.06-eb-4.9.0-001-system.yml
      @@ -0,0 +1,5 @@
      +easyconfigs:
      +  - EasyBuild-4.9.0.eb:
      +      options:
      +        from-pr: 19464
      +  - ReFrame-4.3.3.eb
    • easystacks/software.eessi.io/2023.06/eessi-2023.06-eb-4.9.0-2022b.yml

      diff
      diff --git a/easystacks/software.eessi.io/2023.06/eessi-2023.06-eb-4.9.0-2022b.yml b/easystacks/software.eessi.io/2023.06/eessi-2023.06-eb-4.9.0-2022b.yml
      new file mode 100644
      index 0000000..fdacd95
      --- /dev/null
      +++ b/easystacks/software.eessi.io/2023.06/eessi-2023.06-eb-4.9.0-2022b.yml
      @@ -0,0 +1,2 @@
      +easyconfigs:
      +  - SciPy-bundle-2023.02-gfbf-2022b.eb
    • easystacks/software.eessi.io/2023.06/eessi-2023.06-eb-4.9.0-2023a.yml

      diff
      diff --git a/easystacks/software.eessi.io/2023.06/eessi-2023.06-eb-4.9.0-2023a.yml b/easystacks/software.eessi.io/2023.06/eessi-2023.06-eb-4.9.0-2023a.yml
      new file mode 100644
      index 0000000..eba776a
      --- /dev/null
      +++ b/easystacks/software.eessi.io/2023.06/eessi-2023.06-eb-4.9.0-2023a.yml
      @@ -0,0 +1,10 @@
      +easyconfigs:
      +  - OpenFOAM-11-foss-2023a.eb:
      +      # see https://github.com/easybuilders/easybuild-easyconfigs/pull/19545
      +      options:
      +        from-pr: 19545
      +  - at-spi2-core-2.49.91-GCCcore-12.3.0.eb
      +  - ESPResSo-4.2.1-foss-2023a.eb:
      +      # see https://github.com/easybuilders/easybuild-easyconfigs/pull/19592
      +      options:
      +        from-pr: 19592
    • easystacks/software.eessi.io/2023.06/eessi-2023.06-eb-4.9.0-2023b.yml

      diff
      diff --git a/easystacks/software.eessi.io/2023.06/eessi-2023.06-eb-4.9.0-2023b.yml b/easystacks/software.eessi.io/2023.06/eessi-2023.06-eb-4.9.0-2023b.yml
      new file mode 100644
      index 0000000..9a3b8ef
      --- /dev/null
      +++ b/easystacks/software.eessi.io/2023.06/eessi-2023.06-eb-4.9.0-2023b.yml
      @@ -0,0 +1,7 @@
      +easyconfigs:
      +  - GCC-13.2.0.eb
      +  - foss-2023b.eb
      +  - SciPy-bundle-2023.11-gfbf-2023b.eb
      +  - netCDF-4.9.2-gompi-2023b.eb:
      +      options:
      +        from-pr: 19534
    • eb_hooks.py 👎

      diff
      diff --git a/eb_hooks.py b/eb_hooks.py
      index b3cbc4b..2bc2fde 100644
      --- a/eb_hooks.py
      +++ b/eb_hooks.py
      @@ -185,20 +185,21 @@ def parse_hook_fontconfig_add_fonts(ec, eprefix):
      
      
       def parse_hook_openblas_relax_lapack_tests_num_errors(ec, eprefix):
      -    """Relax number of failing numerical LAPACK tests for aarch64/neoverse_v1 CPU target."""
      +    """Relax number of failing numerical LAPACK tests for aarch64/neoverse_v1 CPU target for OpenBLAS < 0.3.23"""
           cpu_target = get_eessi_envvar('EESSI_SOFTWARE_SUBDIR')
           if ec.name == 'OpenBLAS':
      -        # relax maximum number of failed numerical LAPACK tests for aarch64/neoverse_v1 CPU target
      -        # since the default setting of 150 that works well on other aarch64 targets and x86_64 is a bit too strict
      -        # See https://github.com/EESSI/software-layer/issues/314
      -        cfg_option = 'max_failing_lapack_tests_num_errors'
      -        if cpu_target == CPU_TARGET_NEOVERSE_V1:
      -            orig_value = ec[cfg_option]
      -            ec[cfg_option] = 400
      -            print_msg("Maximum number of failing LAPACK tests with numerical errors for %s relaxed to %s (was %s)",
      -                      ec.name, ec[cfg_option], orig_value)
      -        else:
      -            print_msg("Not changing option %s for %s on non-AARCH64", cfg_option, ec.name)
      +        if LooseVersion(ec.version) < LooseVersion('0.3.23'):
      +            # relax maximum number of failed numerical LAPACK tests for aarch64/neoverse_v1 CPU target
      +            # since the default setting of 150 that works well on other aarch64 targets and x86_64 is a bit too strict
      +            # See https://github.com/EESSI/software-layer/issues/314
      +            cfg_option = 'max_failing_lapack_tests_num_errors'
      +            if cpu_target == CPU_TARGET_NEOVERSE_V1:
      +                orig_value = ec[cfg_option]
      +                ec[cfg_option] = 400
      +                print_msg("Maximum number of failing LAPACK tests with numerical errors for %s relaxed to %s (was %s)",
      +                          ec.name, ec[cfg_option], orig_value)
      +            else:
      +                print_msg("Not changing option %s for %s on non-AARCH64", cfg_option, ec.name)
           else:
               raise EasyBuildError("OpenBLAS-specific hook triggered for non-OpenBLAS easyconfig?!")
      
      @@ -393,6 +394,18 @@ def pre_test_hook_ignore_failing_tests_SciPybundle(self, *args, **kwargs):
           if self.name == 'SciPy-bundle' and self.version in scipy_bundle_versions and cpu_target == CPU_TARGET_NEOVERSE_V1:
               self.cfg['testopts'] = "|| echo ignoring failing tests"
      
      +def pre_test_hook_ignore_failing_tests_netCDF(self, *args, **kwargs):
      +    """
      +    Pre-test hook for netCDF: skip failing tests for selected netCDF versions on neoverse_v1
      +    cfr. https://github.com/EESSI/software-layer/issues/425
      +    The following tests are problematic:
      +        163 - nc_test4_run_par_test (Timeout)
      +        190 - h5_test_run_par_tests (Timeout)
      +    A few other tests are skipped in the easyconfig and patches for similar issues, see above issue for details.
      +    """
      +    cpu_target = get_eessi_envvar('EESSI_SOFTWARE_SUBDIR')
      +    if self.name == 'netCDF' and self.version == '4.9.2' and cpu_target == CPU_TARGET_NEOVERSE_V1:
      +        self.cfg['testopts'] = "|| echo ignoring failing tests"
      
       def pre_single_extension_hook(ext, *args, **kwargs):
           """Main pre-extension: trigger custom functions based on software name."""
      @@ -573,6 +586,7 @@ PRE_TEST_HOOKS = {
           'ESPResSo': pre_test_hook_ignore_failing_tests_ESPResSo,
           'FFTW.MPI': pre_test_hook_ignore_failing_tests_FFTWMPI,
           'SciPy-bundle': pre_test_hook_ignore_failing_tests_SciPybundle,
      +    'netCDF': pre_test_hook_ignore_failing_tests_netCDF,
       }
      
       PRE_SINGLE_EXTENSION_HOOKS = {

      NOTE:Newly added changes

    • eessi-2023.06-known-issues.yml 👎

      diff
      diff --git a/eessi-2023.06-known-issues.yml b/eessi-2023.06-known-issues.yml
      new file mode 100644
      index 0000000..475ee2c
      --- /dev/null
      +++ b/eessi-2023.06-known-issues.yml
      @@ -0,0 +1,28 @@
      +- aarch64/neoverse_v1:
      +  - ESPResSo-4.2.1-foss-2023a:
      +      - issue: https://github.com/EESSI/software-layer/issues/363
      +      - info: "ESPResSo tests failing due to timeouts"
      +  - FFTW.MPI-3.3.10-gompi-2023a:
      +      - issue: https://github.com/EESSI/software-layer/issues/325
      +      - info: "Flaky FFTW tests, random failures"
      +  - FFTW.MPI-3.3.10-gompi-2023b:
      +      - issue: https://github.com/EESSI/software-layer/issues/325
      +      - info: "Flaky FFTW tests, random failures"
      +  - netCDF-4.9.2-gompi-2023a.eb:
      +      - issue: https://github.com/EESSI/software-layer/issues/425
      +      - info: "netCDF intermittent test failures"
      +  - netCDF-4.9.2-gompi-2023b.eb:
      +      - issue: https://github.com/EESSI/software-layer/issues/425
      +      - info: "netCDF intermittent test failures"
      +  - OpenBLAS-0.3.21-GCC-12.2.0:
      +    - issue: https://github.com/EESSI/software-layer/issues/314
      +    - info: "Increased number of numerical errors in OpenBLAS test suite (344 vs max. 150 on x86_64/*)"
      +  - SciPy-bundle-2023.02-gfbf-2022b:
      +    - issue: https://github.com/EESSI/software-layer/issues/318
      +    - info: "numpy built with -march=armv8.4-a instead of -mcpu=native (no SVE) + 2 failing tests (vs 50005 passed) in scipy test suite"
      +  - SciPy-bundle-2023.07-gfbf-2023a:
      +    - issue: https://github.com/EESSI/software-layer/issues/318
      +    - info: "2 failing tests (vs 54409 passed) in scipy test suite"
      +  - SciPy-bundle-2023.11-gfbf-2023b:
      +    - issue: https://github.com/EESSI/software-layer/issues/318
      +    - info: "2 failing tests (vs 54876 passed) in scipy test suite"

      NOTE: Newly added changes

    • eessi_container.sh 👎

      diff
      diff --git a/eessi_container.sh b/eessi_container.sh
      index bfc5963..d6e9558 100755
      --- a/eessi_container.sh
      +++ b/eessi_container.sh
      @@ -30,8 +30,8 @@
       # -. initial settings & exit codes
       TOPDIR=$(dirname $(realpath $0))
      
      -source ${TOPDIR}/scripts/utils.sh
      -source ${TOPDIR}/scripts/cfg_files.sh
      +source "${TOPDIR}"/scripts/utils.sh
      +source "${TOPDIR}"/scripts/cfg_files.sh
      
       # exit codes: bitwise shift codes to allow for combination of exit codes
       # ANY_ERROR_EXITCODE is sourced from ${TOPDIR}/scripts/utils.sh
      @@ -83,7 +83,7 @@ display_help() {
         echo "                           MODE==run (run a script or command) [default: shell]"
         echo "  -n | --nvidia MODE     - configure the container to work with NVIDIA GPUs,"
         echo "                           MODE==install for a CUDA installation, MODE==run to"
      -  echo "                           attach a GPU, MODE==all for both [default: false]"
      +  echo "                           attach a GPU, MODE==all for both [default: false]"
         echo "  -r | --repository CFG  - configuration file or identifier defining the"
         echo "                           repository to use [default: EESSI via"
         echo "                           default container, see --container]"
      @@ -164,7 +164,7 @@ while [[ $# -gt 0 ]]; do
             SETUP_NVIDIA=1
             NVIDIA_MODE="$2"
             shift 2
      -      ;;
      +      ;;
           -r|--repository)
             REPOSITORY="$2"
             shift 2
      @@ -575,7 +575,7 @@ fi
       declare -a EESSI_FUSE_MOUNTS=()
      
       # always mount cvmfs-config repo (to get access to software.eessi.io)
      -# EESSI_FUSE_MOUNTS+=("--fusemount" "container:cvmfs2 cvmfs-config.cern.ch /cvmfs/cvmfs-config.cern.ch")
      +EESSI_FUSE_MOUNTS+=("--fusemount" "container:cvmfs2 cvmfs-config.cern.ch /cvmfs/cvmfs-config.cern.ch")
      
       if [[ "${ACCESS}" == "ro" ]]; then
         export EESSI_READONLY="container:cvmfs2 ${repo_name} /cvmfs/${repo_name}"

      NOTE: Add quotes + commented out intentionally

    • init/Magic_Castle/bash

      diff
      diff --git a/init/Magic_Castle/bash b/init/Magic_Castle/bash
      index 0fb2f67..dbe1772 100644
      --- a/init/Magic_Castle/bash
      +++ b/init/Magic_Castle/bash
      @@ -6,7 +6,7 @@ EESSI_SILENT=1
       source $(dirname "$BASH_SOURCE")/../eessi_environment_variables
      
       # Don't change the default prompt
      -# export PS1="{NESSI $EESSI_VERSION} $ "
      +# export PS1="[EESSI $EESSI_VERSION] $ "
      
       # Provide a clean MODULEPATH
       export MODULEPATH_ROOT=$EESSI_MODULEPATH
      @@ -36,4 +36,4 @@ else
          module reload
       fi
      
      -echo "Environment set up to use NESSI (${EESSI_VERSION}), have fun!"
      +echo "Environment set up to use EESSI (${EESSI_VERSION}), have fun!"
    • init/Magic_Castle/eessi_python3

      diff
      diff --git a/init/Magic_Castle/eessi_python3 b/init/Magic_Castle/eessi_python3
      index c76c7a2..b2f7fd4 100755
      --- a/init/Magic_Castle/eessi_python3
      +++ b/init/Magic_Castle/eessi_python3
      @@ -16,6 +16,6 @@ eessi_python=$(ls ${EESSI_SOFTWARE_PATH}/software/Python/3*GCCcore*/bin/python |
       if [ -f "$eessi_python" ]; then
         $eessi_python "$@"
       else
      -  echo "ERROR: No NESSI Python 3 available."
      +  echo "ERROR: No EESSI Python 3 available."
         false
       fi
    • init/README.md

      diff
      diff --git a/init/README.md b/init/README.md
      index 05baffd..c253dc4 100644
      --- a/init/README.md
      +++ b/init/README.md
      @@ -1,7 +1,7 @@
      -# Bash initialisation for NESSI
      +# Bash initialisation for EESSI
      
       This directory contains the default initialisation script for a bash shell used to
      -configure Lmod and use the NESSI software modules. The (bash)
      +configure Lmod and use the EESSI software modules. The (bash)
       file `eessi_environment_variables` is used to set and export the full set of EESSI
       environment variables:
      
      @@ -9,12 +9,12 @@ environment variables:
       - `EESSI_EPREFIX`: The location of Gentoo Prefix compatability layer (for the architecture).
       - `EESSI_EPREFIX_PYTHON`: Path to `python3` in the Gentoo Prefix layer.
       - `EESSI_SOFTWARE_SUBDIR`: Hardware specific software subdirectory.
      -- `EESSI_SOFTWARE_PATH`: Full path to NESSI software stack.
      +- `EESSI_SOFTWARE_PATH`: Full path to EESSI software stack.
       - `EESSI_MODULEPATH`: Path to be added to the `MODULEPATH`. This can be influenced by two
         externally defined environment varialbes:
           - `EESSI_CUSTOM_MODULEPATH`: defines a fully custom directory to be added to
             `MODULEPATH`, the end user is entirely responsible for what this directory contains.
      -    - `EESSI_MODULE_SUBDIR`: NESSI may ship with a number of possible module naming schemes.
      +    - `EESSI_MODULE_SUBDIR`: EESSI may ship with a number of possible module naming schemes.
             This variable can be used to point to a non-default module naming scheme.
      
       All scripts respect the environment variable `EESSI_SILENT` which, if defined to any
      @@ -25,5 +25,5 @@ value, will make them produce no (non-error) output.
       The `Magic_Castle` subdirectory is home to the bash initialisation that we use for
       [Magic Castle](https://github.com/ComputeCanada/magic_castle).
      
      -It also contains a wrapper for a hardware optimised NESSI Python 3 installation that is used
      +It also contains a wrapper for a hardware optimised EESSI Python 3 installation that is used
       by Magic Castle to properly configure JupyterHub.
    • init/arch_specs/eessi_arch_arm.spec

      diff
      diff --git a/init/arch_specs/eessi_arch_arm.spec b/init/arch_specs/eessi_arch_arm.spec
      index f8f21f9..b5c9275 100755
      --- a/init/arch_specs/eessi_arch_arm.spec
      +++ b/init/arch_specs/eessi_arch_arm.spec
      @@ -1,6 +1,6 @@
       # ARM CPU architecture specifications
       # Software path in EESSI 	| Vendor ID 	| List of defining CPU features
      -"aarch64/neoverse-n1"	"ARM"		"asimd"		# Ampere Altra
      -"aarch64/neoverse-n1"	""		"asimd"		# AWS Graviton2
      -"aarch64/neoverse-v1"	"ARM"		"asimd svei8mm"
      -"aarch64/neoverse-v1"	""		"asimd svei8mm"	# AWS Graviton3
      +"aarch64/neoverse_n1"	"ARM"		"asimd"		# Ampere Altra
      +"aarch64/neoverse_n1"	""		"asimd"		# AWS Graviton2
      +"aarch64/neoverse_v1"	"ARM"		"asimd svei8mm"
      +"aarch64/neoverse_v1"	""		"asimd svei8mm"	# AWS Graviton3
    • init/bash 👎

      diff
      diff --git a/init/bash b/init/bash
      index 26598bb..0029f91 100644
      --- a/init/bash
      +++ b/init/bash
      @@ -5,6 +5,7 @@ function show_msg {
           echo "$msg"
         fi
       }
      +
       # The following method should be safe, but might break if file is a symlink
       # (could switch to $(dirname "$(readlink -f "$BASH_SOURCE")") in that case)
       source $(dirname "$BASH_SOURCE")/eessi_environment_variables
      @@ -12,7 +13,7 @@ source $(dirname "$BASH_SOURCE")/eessi_environment_variables
       # only continue if setting EESSI environment variables worked fine
       if [ $? -eq 0 ]; then
      
      -    export PS1="{NESSI $EESSI_VERSION} $PS1"
      +    export PS1="{EESSI $EESSI_VERSION} $PS1"
      
           # add location of commands provided by compat layer to $PATH;
           # see https://github.com/EESSI/software-layer/issues/52
      @@ -23,7 +24,7 @@ if [ $? -eq 0 ]; then
           source $EESSI_EPREFIX/usr/share/Lmod/init/bash
      
           # prepend location of modules for EESSI software stack to $MODULEPATH
      -    echo "Prepending $EESSI_MODULEPATH to \$MODULEPATH..." >> $output
      +    show_msg "Prepending $EESSI_MODULEPATH to \$MODULEPATH..."
           module use $EESSI_MODULEPATH
      
           #show_msg ""
      @@ -33,6 +34,6 @@ if [ $? -eq 0 ]; then
           #show_msg ""
           #show_msg ""
      
      -    echo "Environment set up to use NESSI (${EESSI_VERSION}), have fun!"
      +    echo "Environment set up to use EESSI (${EESSI_VERSION}), have fun!"
      
       fi

      NOTE: need to change echo -->show_msg.

    • init/eessi_archdetect.sh 👎

      diff
      diff --git a/init/eessi_archdetect.sh b/init/eessi_archdetect.sh
      index 236ac5a..58b79b0 100755
      --- a/init/eessi_archdetect.sh
      +++ b/init/eessi_archdetect.sh
      @@ -73,7 +73,7 @@ cpupath(){
           [ $EESSI_SOFTWARE_SUBDIR_OVERRIDE ] && echo ${EESSI_SOFTWARE_SUBDIR_OVERRIDE} && exit
      
           # Identify the best matching CPU architecture from a list of supported specifications for the host CPU
      -    # Return the path to the installation files in NESSI of the best matching architecture
      +    # Return the path to the installation files in EESSI of the best matching architecture
           local cpu_arch_spec=()
      
           # Identify the host CPU architecture
      @@ -118,7 +118,7 @@ cpupath(){
                   # each flag in this CPU specification must be found in the list of flags of the host
                   check_allinfirst "${cpu_flags[*]}" ${arch_spec[2]} && best_arch_match=${arch_spec[0]} && \
                       all_arch_matches="$best_arch_match:$all_arch_matches" && \
      -                log "DEBUG" "cpupath: host CPU best match updated to $best_arch_match"
      +                log "DEBUG" "cpupath: host CPU best match updated to $best_arch_match"
               fi
           done
      

      NOTE: Extra space character in NESSI.

    • init/eessi_defaults

      diff
      diff --git a/init/eessi_defaults b/init/eessi_defaults
      index da057fb..d1779a3 100644
      --- a/init/eessi_defaults
      +++ b/init/eessi_defaults
      @@ -8,9 +8,8 @@
       # license: GPLv2
       #
      
      -export EESSI_CVMFS_REPO="${EESSI_CVMFS_REPO_OVERRIDE:=/cvmfs/pilot.nessi.no}"
      +export EESSI_CVMFS_REPO="${EESSI_CVMFS_REPO_OVERRIDE:=/cvmfs/software.eessi.io}"
       export EESSI_VERSION="${EESSI_VERSION_OVERRIDE:=2023.06}"
       # use archdetect by default, unless otherwise specified
      -# 2024-01-16: switch back to archspec with NESSI for now
      -export EESSI_USE_ARCHDETECT="${EESSI_USE_ARCHDETECT:=0}"
      -export EESSI_USE_ARCHSPEC="${EESSI_USE_ARCHSPEC:=1}"
      +export EESSI_USE_ARCHDETECT="${EESSI_USE_ARCHDETECT:=1}"
      +export EESSI_USE_ARCHSPEC="${EESSI_USE_ARCHSPEC:=0}"
    • init/eessi_environment_variables 👎

      diff
      diff --git a/init/eessi_environment_variables b/init/eessi_environment_variables
      index d4a2e72..3584bfa 100644
      --- a/init/eessi_environment_variables
      +++ b/init/eessi_environment_variables
      @@ -2,6 +2,11 @@
       # $BASH_SOURCE points to correct path, see also http://mywiki.wooledge.org/BashFAQ/028
       EESSI_INIT_DIR_PATH=$(dirname $(realpath $BASH_SOURCE))
      
      +function error() {
      +    echo -e "\e[31mERROR: $1\e[0m" >&2
      +    false
      +}
      +
       function show_msg {
         # only echo msg if EESSI_SILENT is unset
         msg=$1
      @@ -10,16 +15,11 @@ function show_msg {
         fi
       }
      
      -function error() {
      -    echo -e "\e[31mERROR: $1\e[0m" >&2
      -    false
      -}
      -
       # set up minimal environment: $EESSI_PREFIX, $EESSI_VERSION, $EESSI_OS_TYPE, $EESSI_CPU_FAMILY, $EPREFIX
       source $EESSI_INIT_DIR_PATH/minimal_eessi_env
      
       if [ -d $EESSI_PREFIX ]; then
      -  show_msg "Found NESSI repo @ $EESSI_PREFIX!"
      +  show_msg "Found EESSI repo @ $EESSI_PREFIX!"
      
         export EESSI_EPREFIX=$EPREFIX
         if [ -d $EESSI_EPREFIX ]; then
      @@ -63,7 +63,7 @@ if [ -d $EESSI_PREFIX ]; then
                   export EESSI_MODULEPATH=$EESSI_MODULEPATH
                   show_msg "Using ${EESSI_MODULEPATH} as the directory to be added to MODULEPATH."
                 else
      -            error "NESSI module path at $EESSI_MODULEPATH not found!"
      +            error "EESSI module path at $EESSI_MODULEPATH not found!"
                   false
                 fi
      
      @@ -75,7 +75,7 @@ if [ -d $EESSI_PREFIX ]; then
                 fi
      
               else
      -          error "NESSI software layer at $EESSI_SOFTWARE_PATH not found!"
      +          error "EESSI software layer at $EESSI_SOFTWARE_PATH not found!"
               fi
           else
               error "no value set for \$EESSI_SOFTWARE_SUBDIR"
      @@ -84,5 +84,5 @@ if [ -d $EESSI_PREFIX ]; then
           error "Compatibility layer directory $EESSI_EPREFIX not found!"
         fi
       else
      -  error "NESSI repository at $EESSI_PREFIX not found!"
      +  error "EESSI repository at $EESSI_PREFIX not found!"
       fi

      NOTE: code line reordering .

    • init/eessi_software_subdir_for_host.py

      diff
      diff --git a/init/eessi_software_subdir_for_host.py b/init/eessi_software_subdir_for_host.py
      index e8f1a71..58e9cfd 100755
      --- a/init/eessi_software_subdir_for_host.py
      +++ b/init/eessi_software_subdir_for_host.py
      @@ -1,6 +1,6 @@
       #!/usr/bin/env python3
       #
      -# Determine NESSI software subdirectory to use for current build host, using archspec
      +# Determine EESSI software subdirectory to use for current build host, using archspec
       #
       import glob
       import os
    • init/minimal_eessi_env

      diff
      diff --git a/init/minimal_eessi_env b/init/minimal_eessi_env
      index f938d1d..5b35620 100644
      --- a/init/minimal_eessi_env
      +++ b/init/minimal_eessi_env
      @@ -1,4 +1,4 @@
      -# define minimal NESSI environment, without relying on external scripts
      +# define minimal EESSI environment, without relying on external scripts
       #
       # this script is *sourced*, not executed, so can't rely on $0 to determine path to self
       # $BASH_SOURCE points to correct path, see also http://mywiki.wooledge.org/BashFAQ/028

@trz42 trz42 changed the title Syncing with EESSI. Syncing code-base and CVMFS repository with EESSI (January 2024) Apr 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants