Brainstorm meeting (July 7th 2020) Syncing software to Stratum 0

Syncing software into Stratum-0 from build node

• Compute Canada would use gateway-publisher procedure if they would start over

• at least one build machine per CPU arch

• getting started

  # request publishing rights from gateway
  # (can be done for specific subdirectory)
  cvmfs_server mkfs
  # gives you mount of /cvmfs/pilot.eessi-hpc.org/
  # (how does this work for specific subdir?)
  

• overlapping transaction and installation is not a good idea

  • installation make take a long time, may fail
  • ask Maxime why?

• rsync to /build, bind /build into container as /cvmfs

  • use minimal container also to build in isolated environment
  • rsync could be limited if needed
    • only pull in specific paths (ask EB which deps/toolchain)

• when installation is done:

  • start transaction
  • rsync from /build to /cvmfs
  • publish transaction

• other options

  • overlay through container
    • pull in deps through CVMFS during installation (delay)
  • transaction on specific installation dirs
    • allows for multiple build hosts for same CPU arch

• starting point

  • evaluate overlapping transaction + installation
  • benchmark delay compared to installation on local filesystem
  • test with GCC/9.3.0 + deps?
  • try to use publisher container image (only CVMFS)
    • based on container image for CVMFS client: https://github.com/EESSI/filesystem-layer/issues/10#issuecomment-652985462
  • report back Thu Aug 6th 2020 at next EESSI meeting?

• optimizations in CVMFS (https://github.com/EESSI/filesystem-layer/issues/29)

  • separate catalogs
  • one catalog per installation directory?
  • basically touching a .cvmfs_catalog file (through hook)
  • finer grained for very large installation directories?
    • for example one catalog per extension
  • rule of thumb
    • in between 1k & 200k files
  • "catch-all" catalog for installations that are too small (<100)?
  • one catalog for all versions of a software package?
  • can also be done automatically (see CVMFS manual)
    • can still intervene for exceptional cases, if necessary
  • see:
  • https://cvmfs.readthedocs.io/en/stable/cpt-repo.html#recommendations-for-nested-catalogs
  • https://cvmfs.readthedocs.io/en/stable/cpt-repo.html#managing-nested-catalogs-with-cvmfsdirtab

• granularity of transactions?

  • each installation as a separate transaction?

• Goal: container that takes private key + easyconfig and eventually results in installation being added to Stratum-0?

• EasyBuild configuration:

  • for now (in container recipe): use CLI options
  • later: driver script will configure EB (https://github.com/EESSI/software-layer/pull/5)
    • through set_up_configuration function

• AP:

  • Bob + Maxim: look into container recipe for CVMFS publisher
    • Maxim: report during next EESSI meeting (Thu Aug 6th)
  • Caspar: EB driver script (maybe)
  • Alan: testing, feedback
  • Kenneth: well deserved vacation