add custom easyblock to support installation of new Intel compilers (v2021.x, oneAPI)

[boegel]

• enhance IntelBase to implement installation procedure for 2021.x versions of Intel tools
• add new custom easyblock for intel-compilers, to be used with Intel oneAPI HPC Base toolkit v2021 & newer
• • small bug fix in Bundle that I hit when working on this

requires

• add support for --accept-eula configuration option + 'accept_eula' easyconfig parameter easybuild-framework#3535 (support for --accept-eula)
• add support for specifying custom name to use when checking accepted EULA easybuild-framework#3546 to allow using custom name when calling check_accepted_eula

[boegel]

@rscohn2 Any feedback on this?

[rscohn2]

It looks good. Where is the code that constructs the URL for downloads?

[ocaisa]

@rscohn2 You can find the URL for downloading in https://github.com/easybuilders/easybuild-easyconfigs/pull/11982/files

[scroogie]

Can I do anything to help this getting forward?

[boegel]

Can I do anything to help this getting forward?

If you're up for it, you could look into the necessary changes w.r.t. redefining $HOME and rolling that back in time.

One way could be to adding 'HOME=%s' % self.builddir to the cmd being constructed in install_step_oneapi (rather than setting $HOME globally via os.environ), and then check whether that works as expected.

You can test using eb --from-pr 11982 (see easybuilders/easybuild-easyconfigs#11982), but that PR also needs a little bit of updating.

I hope to pick up on this again myself soon, but finding time for it has not been easy in recent weeks... 😅

[scroogie]

Okay, not sure I have the necessary knowledge yet, but I'll have a look.

Regarding the variable, subprocess.Popen() actually allows passing an env, which we could just copy from os.environ. Then we dont run any risk of modifying the live env. Maybe that would be a sensible addition to run_cmd?

I'll try prefixing the command as well though. What I don't know, does that actually work in all shells, i.e. tscsh or ksh, or are we always in bash (didnt find it directly)?

[scroogie]

@boegel Just a short ping to ask which direction you think makes more sense.

[boegel]

@scroogie run_cmd always runs commands in a bash subshell I think, so just prefixing the command with HOME=... should work

[ocaisa]

@boegel The issues I saw with this are addressed in boegel#37

[boegel]

Don't merge this just yet, may need some more love since to get the 2021.1.2 version of the compilers you need to install the HPC base kit + additional updates for C/C++ and Fortran compilers; see https://community.intel.com/t5/Intel-Fortran-Compiler/Announcing-Intel-Fortran-Compiler-Patch-Release-2021-1-2-Answers/m-p/1240262#M153503...

[boegel]

@ocaisa I think this is good to go now...

Works fine with easybuilders/easybuild-easyconfigs#11982, and I'll also upload a test report with older versions of the Intel tools, to confirm nothing got broken with the changes being made here...

[boegel]

Test report by @boegel

Overview of tested easyconfigs (in order)

• SUCCESS icc-2016.3.210-GCC-5.4.0-2.26.eb
• SUCCESS ifort-2016.3.210-GCC-5.4.0-2.26.eb
• SUCCESS impi-5.1.3.181-iccifort-2016.3.210-GCC-5.4.0-2.26.eb
• SUCCESS imkl-11.3.3.210-iimpi-2016b.eb
• SUCCESS intel-2016b.eb

Build succeeded for 5 out of 5 (5 easyconfigs in total)
node2610.swalot.os - Linux centos linux 7.9.2009, x86_64, Intel(R) Xeon(R) CPU E5-2660 v3 @ 2.60GHz (haswell), Python 3.6.8
See https://gist.github.com/69a657eb1ad0f2da5b0a4029325d4e0c for a full test report.

[ocaisa]

LGTM

[boegel]

Test report by @boegel

Overview of tested easyconfigs (in order)

• SUCCESS iccifort-2020.4.304.eb
• SUCCESS impi-2019.9.304-iccifort-2020.4.304.eb
• SUCCESS imkl-2020.4.304-iimpi-2020b.eb
• SUCCESS intel-2020b.eb

Build succeeded for 4 out of 4 (4 easyconfigs in total)
node3300.joltik.os - Linux centos linux 7.9.2009, x86_64, Intel(R) Xeon(R) Gold 6242 CPU @ 2.80GHz (cascadelake), Python 3.6.8
See https://gist.github.com/2d4bfdab8cf88cda0b58fdb7871e6c0a for a full test report.