Sync meeting on EESSI Software layer (2023 09 05)

EESSI software layer sync meeting

planning

• dow we want/need to do this more often?
  • Tue 12 Sept at 09:00 CEST (same time slot as bi-weekly bot sync meetings) - Thomas will update meeting invite

Meeting (2023-09-05)

Attending: Lara, Kenneth, Alan, Thomas, Richard, Bob, Caspar, Maksim, Satish

Status

• Easybuild 4.8.0 now installed PR #319
• eessi-2023.06-eb-4.7.2-2021a.yml
  • Gromacs (PR #288)
  • QuantumEspresso (PR #281)
• eessi-2023.06-eb-4.7.2-2021b.yml
  • Gromacs (PR #304)
  • QuantumESPRESSO (PR #289)
• eessi-2023.06-eb-4.7.2-2022b.yml
  • ready to start adding more software
    • maybe Xin can look into GROMACS with foss/2022b?
      • she's currently looking into walBERla

Open PRs and Issues

• Non optimal CPU detection with archspec in EESSI 2023.06

  • issue #322: zen3
  • issue #318: neoverse_v1
    • archspec picks neoverse_n1 while it should be picking neoverse_v1
  • we can fix this by using archdetect instead of archspec
    • see also PR #264

• Failing tests

  • issue #318: SciPy-bundle
    • ignoring failing numpy test suite on neoverse_v1 via hook for specific SciPy-bundle version
  • issue #314: OpenBLAS
    • more failing numerical tests on neoverse_v1, workaround in place by allowing more failing tests via hook
  • issue #325: FFTW
    • [Caspar] should ignore (flaky) failing tests on neoverse_v1 (like we do for numpy, see hook changes in PR #306)
    • sometimes FFTW test suite hands (but only when running interactively?)
  • policy to deal with handful of failing tests for a specific CPU target is "documented" in https://github.com/EESSI/software-layer/pull/309#issuecomment-1666064083
  • we should add a "Known issues" page in EESSI docs with pointers to issues with more info

• [Kenneth] Failing to rebuild ReFrame

  • issue #312
  • issue #311

• Open software build PRs

  • PR #290: WRF-dmpar (foss/2021a)
    • [Kenneth] needs fix in our configure hooks for more recent WRF versions
  • PR #310: foss-2022a
    • [Caspar] needs workaround for FFTW test suite failures
  • PR #316: Arrow (foss/2021a)
    • [Richard?] need to figure out what the problem is there
  • [Lara] filtering of cURL issue (PR #320)
    • PR #299: RStudio-server (foss/2021a)
    • PR #321: Tensorflow (foss/2021b)
      • filtering selectively (using hook)

• other stuff

  • Richard is looking into installing intel toolchain (primarily for NESSI)
    • end user have to agree with EULA, so we can't ship Intel compilers in EESSI (only the runtime parts)
  • pre-fetching source tarballs would make it more robust and avoid that each build job needs to re-download the sources
    • to be robust against flaky issues with GitHub (or other places we download from)
  • Maksim will look into adding Espresso to EESSI 2023.06 (via easyconfigs PR #18486)
  • what are the goals for EESSI pilot 2023.06?
    • same software as in EESSI 2021.12
    • • Espresso & walBERla (for MultiXscale)
    • • more software like AlphaFold, etc. which helps with exposure of EESSI
  • an EESSI pilot that is ready-for-use?
    • ideally under eessi.io
    • EESSI pilot that can be used and will be supported early 2024 for 3 years (until end of MultiXscale)
      • Need to define well what we mean by support
    • Bob will chase colleagues to get new Stratum-0 ready to go
    • Thomas can look into building compat layer for EESSI 2023.09 under eessi.io
      • same gentoo commit
      • repo name?
        • pilot.eessi.io? (do we really want another pilot)
        • should go with software.eessi.io (time to go production...)
      • focus on a single (sufficiently recent) toolchain?
        • foss/2022a is probably a good pick there
        • will need more synchronisation to avoid stepping on each other's toes
        • list of missing easyconfigs can already be obtained easily from the CI checks
  • support policy
    • see draft in https://github.com/EESSI/docs/pull/109
    • do we need to discriminate between end users and HPC sites who adopt EESSI?