Skip to content

Conference call notes 20200513

Jump to bottom
Kenneth Hoste edited this page May 20, 2020 · 8 revisions

(back to Conference calls)

Notes on the 147th EasyBuild conference call, Wednesday May 13th 2020 (8:00 UTC - 10:00 CEST)

Attendees

Alphabetical list of attendees (9):

  • Kenneth Hoste (HPC-UGent, Belgium)
  • Åke Sandgren (Umeå University, Sweden)
  • Mikael Öhman (Chalmers University of Technology, Sweden)
  • Henrik Nagel (NTNU, Norway)
  • Alexandre Strube (JSC, Germany)
  • Alan O'Cais (JSC, Germany)
  • Lars Viklund (Umeå University, Sweden)
  • Thekla Loizou (Cyprus Institute)
  • Adam Huffman (Big Data Institute, Oxford, UK)

Agenda

  • overview of recent changes to be included with upcoming EasyBuild v4.2.1
  • update on progress on 2020a generation of easyconfigs
  • Q&A

Overview of recent changes

framework: merged PRs in 4.2.1 milestone
  • recently merged PRs:
    • automatic changing of directory by extract_file is deprecated (PR #3292)
    • proper fix for PyPI download problem, parse page served by PyPI's /simple API as HTML (not XML) (PR #3329)
      • it looks like the cause of the problems in EasyBuild will be fixed on the PyPI side too (see here)
    • cleaning up locks on signals (PR #3291)
      • this is not enough to auto-cleanup locks for EasyBuild sessions run in Slurm jobs that get killed though... (issue #3280)
      • requires looking into fcntl style looks; looks doable, maybe even should be the default locking mechanism?
    • stop using distutils.dir_util in copy_dir (PR #3310)
    • make ModulesTool.exist more robust w.r.t. module wrappers & co (PR #3216)
    • fix problems with parsing easyconfigs using a Cray toolchain when there are no actual external modules (PR #3319)
      • this fixed --list-installed-software crash when using HMNS
    • add constants for common OS dependencies (PR #3309, PR #3334)
      • OS_PKG_IBVERBS_DEV, OS_PKG_OPENSSL_DEV, OS_PKG_PAM_DEV, ...
      • Åke: OS_PKG_IBVERBS_DEV should be renamed?
    • bug fixes for --try-update-deps experimental feature (PR #3325), (PR #3326), (PR #3330),
    • cleaned up output for --list-toolchains --output-format=rst (PR #3246)
    • add --disable-job in 'eb' command used in jobs, to prevent infinite job cycle (PR #3328)
    • avoid empty entries in $LD_LIBRARY_PATH and other path-like environment variables (PR #3333)
  • open PRs/issues that should be merged/fixed for EasyBuild 4.2.1 release:
    • bugs to fix:
      • Java/11 module wrapper is not seen as installed when using HierarchicalMNS (issue #3335, introduced by PR #3216)
    • bugfix PRs to review/merge:
      • expand glob paths when checking for multiple inclusion of easyblocks (bugfix for --include-easyblocks*) (PR #3311)
    • new features:
      • allow copying of tweaked easyconfigs when using --try-* (PR #3332)
easyblocks: merged PRs in 4.2.1 milestone
  • recently merged PRs:
    • fixes/enhancements for PETSc easyblock (PR #2048)
    • use config.guess for GCC builds (PR #2033)
    • impi: execute minimal test in sanity check commands (PR #2045)
    • fix for ROOT easyblock (PR #2047)
    • include own lib dir in RPATH before system lib dirs in binutils easyblock (PR #2044)
  • open PRs/issues that should be merged/fixed for EasyBuild 4.2.1 release:
    • correct download url for test data in WPS easyblock (PR #2055)
    • use updated config.guess for installing R packages (PR #1949)
    • fix for CMakeMake easyblock w.r.t. switch to building outside source tree (PR#2032)
    • rewrite GROMACS easyblock to build all four variations in the same build (PR #1991)
    • generically find the arch folder and add top-level bin/lib symlinks in PDT & TAU easyblocks (PR #2029)
    • new easyblock to install XALT (PR #1942)
    • fix problem with forced reinstallation of ROOT (issue #2023)
easyconfigs merged PRs in 4.2.1 milestone
  • recently merged PRs:
    • patch for h5py with 2020a toolchain to avoid workaround for MPI_Init at import h5py (PR #10626, original workaround in PR #10246)
      • will need more work, current approach of disabling auto-init by mpi4py isn't working out well
      • see also https://github.com/h5py/h5py/pull/1544
      • Alan: god way forward could be to roll back patch for h5py with foss/2020a and pull out h5py with intel/2020a from develop, to avoid blocking 4.2.1 release over this
    • fix for R 3.6.2 + R 3.6.3 on POWER (PR #9830, PR #10531)
  • open issues/PRs:
    • fix Mako easyconfigs using a 2016 & 2017 toolchain (PR #10627)
    • TensorFlow 2.2.0 w/ fosscuda/2019b (PR #10600)
      • Mikael: discussion with Jack Perdue in this PR w.r.t. handling of CUDA compute capabilities
      • silly typo to fix in TensorFlow easyblock: "may not optional" -> "may not be optimal"
    • updated/more extensions for R 4.0.0 & Bioconductor 3.11 (PR #10614, PR #10621)
    • bugfix for Arlequin (PR #10620)

Update on 2020a common toolchains

Q&A

  • Henrik: Java/11 wrapper doesn't seem to work for us in HMNS context
    • could be an issue with old Lmod version not picking up .modulerc.lua?
      • Lmod 7.8
    • fix could be to install Java/11 wrapper with Tcl module syntax (because older Lmod versions don't pick up on .modulerc.lua, only .modulerc)
  • Alexandre: issues with updating PyTorch to 1.5.0 with fosscuda toolchain, CMake build dep is not being found?
    • easier to discuss in Slack, CMake using GCCcore easyconfig should be picked up when using fosscuda
  • Alexandre: starting point for implementing --new-draft-pr?
    • see easybuild/tools/github.py
    • add option to new_pr_from_branch function, act in it after figuring out magic incantation via GitHub PR, expose it via new --new-draft-pr CLI option
Clone this wiki locally