meeting June 3 2021

Notes for 20210603 meeting

• date & time: Thu June 3rd 2021 - 2pm CEST (12:00 UTC)
  • (every first Thursday of the month)
• venue: (online, see mail for meeting link, or ask in Slack)
• agenda:
  • Quick introduction by new people
  • EESSI-related meetings in last month
  • Progress update per EESSI layer
  • 2021.03 version of pilot repository: status
  • Testing with ReFrame
  • AWS/Azure update + infrastructure in AWS
  • EESSI paper
  • Upcoming events
  • Q&A

Slides

• see https://github.com/EESSI/meetings/blob/main/meetings/EESSI_meeting_20210603.pdf

Meeting notes

(by Bob, Kenneth)

Introduction by new people

• Guilherme Amadio (CERN), Gentoo Prefix developer
  • presentation about use of Gentoo Prefix to replace current CERN approach
  • at CHEP conference in Sofia, see https://indico.cern.ch/event/587955/contributions/2938043/
  • An older paper, written by Benda Xu and Guilherme, also about Gentoo prefix and its potential usage in HPC environments: https://arxiv.org/abs/1610.02742
• Lucy (UiO), colleague of Terje

EESSI-related meetings in last month

• hoping that a better solution is possible for the CernVM-FS corruption issue when doing in-place updates
  • the symlinks workaround is going to be a PITA, just symlinkin libc.so is not enough
  • Guilherme: clients could stick to a particular version/snapshot of the repo
  • Guilherme is physically located in the same building as Jakob Blomer (CernVM-FS lead dev), could help with making it clear this problem is important for the HPC community (Compute Canada + EESSI)

Progress update per EESSI layer

• (see slides)

2021.03 version of pilot repository: status

• this version will probably not be extended anymore, instead we aim for a new version (2021.06?)

Outlook to next pilot version

• main focus would be to automate the ingestion of software installations from build node to Stratum 0

Testing with ReFrame

• Plan is start setting up tests using ReFrame library approach
  • Caspar is planning to rework his GROMACS/TensorFlow tests

Sponsorship AWS/Azure

• AWS credtis are being put to good use
• things have started moving w.r.t. Azure sponsorship, more info soon...

AWS infrastructure

• S3 bucket available to facilitate ingestion of software install tarballs from build node to Stratum 1

EESSI paper

• submitted paper available for request, contact Thomas Röblitz
• topics covered:
  • context & motivation
  • high-level overview
  • instructions for getting access to EESSI pilot repo
  • use cases
  • performance evaluation
  • related work
  • future work
• notification expected by end of July'21
• final revision due a couple of weeks later

NESSI update

• EESSI test setup is live on Saga, coming soon on Fram...

Q&A

• Guilherme: For software installed with Portage, have you tried out using Portage's FEATURES=test to run the tests?
  • extra function available to provide additional tests that can be run during the installation
  • https://packages.gentoo.org/useflags/test
• Guilherme: worth evaluating GROMACS installation provided via Gentoo Prefix?
• Guilherme: using busybox as an OS in build host to prevent picking up stuff from host
• Guilherme: EESSI approach could be interesting for CERN as well
  • they're currently rebuilding the software stack for a bunch of different operating systems...
  • strong focus on Spack currently, ongoing effort for years...
  • interesting applications: ROOT, Geant4