Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

{numlib}[GCC/11.3.0] FFTW v3.3.10 #15406

Merged
merged 2 commits into from
May 29, 2022
Merged

{numlib}[GCC/11.3.0] FFTW v3.3.10 #15406

merged 2 commits into from
May 29, 2022

Conversation

bartoldeman
Copy link
Contributor

using --new-pr but manually adjusted

Note that since FFTW puts MPI functionality in separate libraries (such as libfftw3f_mpi.so.3.5.8) the concerns raised by @Micket in easybuilders/easybuild#649 (comment) that apply to HDF5 and netCDF don't apply for FFTW (or Boost).

@bartoldeman
Copy link
Contributor Author

@boegelbot please test @ generoso

@boegelbot
Copy link
Collaborator

@bartoldeman: Request for testing this PR well received on login1

PR test command 'EB_PR=15406 EB_ARGS= /opt/software/slurm/bin/sbatch --job-name test_PR_15406 --ntasks=4 ~/boegelbot/eb_from_pr_upload_generoso.sh' executed!

  • exit code: 0
  • output:
Submitted batch job 8468

Test results coming soon (I hope)...

- notification for comment with ID 1114904311 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

@boegelbot
Copy link
Collaborator

Test report by @boegelbot
SUCCESS
Build succeeded for 1 out of 1 (1 easyconfigs in total)
cns1 - Linux Rocky Linux 8.5, x86_64, Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz (haswell), Python 3.6.8
See https://gist.github.com/7096733d43cb8dc3a2192fe32d67b1da for a full test report.

@SebastianAchilles
Copy link
Member

Test report by @SebastianAchilles
SUCCESS
Build succeeded for 1 out of 1 (1 easyconfigs in total)
zen2-ubuntu-eb - Linux Ubuntu 22.04, x86_64, AMD EPYC 7452 32-Core Processor (zen2), Python 3.10.4
See https://gist.github.com/67b7705e79bb1fd6fb64355daf36b712 for a full test report.

@branfosj
Copy link
Member

branfosj commented May 3, 2022

Test report by @branfosj
FAILED
Build succeeded for 0 out of 1 (1 easyconfigs in total)
bear-pg0306u07a.bear.cluster - Linux RHEL 8.5, POWER, 8335-GTX (power9le), 4 x NVIDIA Tesla V100-SXM2-16GB, 470.57.02, Python 3.6.8
See https://gist.github.com/e83785097e987ba586e070bdac102b2c for a full test report.

@branfosj
Copy link
Member

branfosj commented May 3, 2022

I'm putting #15406 (comment) here so we know about the failure. However, because I expect we'll have issues creating a foss toolchain (see #12968), I am not going to investigate this failure.

@branfosj
Copy link
Member

branfosj commented May 3, 2022

Test report by @branfosj
SUCCESS
Build succeeded for 1 out of 1 (1 easyconfigs in total)
bear-pg0105u36b.bear.cluster - Linux RHEL 8.5, x86_64, Intel(R) Xeon(R) Platinum 8360Y CPU @ 2.40GHz (icelake), Python 3.6.8
See https://gist.github.com/9bddc062142f236dcd24a46e3a684a09 for a full test report.

@branfosj
Copy link
Member

branfosj commented May 3, 2022

Test report by @branfosj
SUCCESS
Build succeeded for 1 out of 1 (1 easyconfigs in total)
bear-pg0211u03a.bear.cluster - Linux RHEL 8.5, x86_64, Intel(R) Xeon(R) Gold 6248 CPU @ 2.50GHz (cascadelake), Python 3.6.8
See https://gist.github.com/e4044c1e0a36f66ea56b82cb3ca4445b for a full test report.

@boegel boegel added this to the 4.x milestone May 3, 2022
@boegel
Copy link
Member

boegel commented May 3, 2022

@bartoldeman Can you clarify the intended use of this exactly?

Do you want to split off the FFTW MPI parts into a separate module for foss/2022a?
If so, will that require changes to EasyBuild framework too in order to avoid trouble with setting up the build environment?

@bartoldeman
Copy link
Contributor Author

Do you want to split off the FFTW MPI parts into a separate module for foss/2022a?
yes

It will need a few framework changes, in fftw.py to include both library paths.

Also if we stick to FFTW.serial (GCC toolchain) / FFTW (gompi toolchain)
one can have a new gfbf toolchain (GCC-FlexiBlas-FFTW, like golf) with
FFT_MODULE_NAME = ['FFTW.serial']
and that works without any changes in fftw.py.

If we use instead rename this PR from FFTW.serial to FFTW (GCC toolchain) and use FFTW.MPI (gompi toolchain), as more people on slack seem to be in favour of (3 vs 1 now), then gfbf doesn't need to change FFT_MODULE_NAME but
fftw.py will need to account for the existence of the FFTW.MPI module and if it's loaded, adjust the library paths and other env vars, instead of relying on the usempi toolchainopt.

@verdurin
Copy link
Member

@boegel would you prefer to pause this one for more discussion?

@boegel
Copy link
Member

boegel commented May 10, 2022

@bartoldeman I think building FFTW with MPI support is the exception, so that should be reflected in the name, so FFTW + FFTW.MPI (or FFTW-MPI) makes more sense to me...

@bartoldeman bartoldeman changed the title {numlib}[GCC/11.3.0] FFTW.serial v3.3.10 {numlib}[GCC/11.3.0] FFTW v3.3.10 May 11, 2022
@SebastianAchilles
Copy link
Member

@boegelbot please test @ generoso

@boegelbot
Copy link
Collaborator

@SebastianAchilles: Request for testing this PR well received on login1

PR test command 'EB_PR=15406 EB_ARGS= /opt/software/slurm/bin/sbatch --job-name test_PR_15406 --ntasks=4 ~/boegelbot/eb_from_pr_upload_generoso.sh' executed!

  • exit code: 0
  • output:
Submitted batch job 8537

Test results coming soon (I hope)...

- notification for comment with ID 1124220872 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

@SebastianAchilles
Copy link
Member

@boegelbot please test @ jsc-zen2

@boegelbot
Copy link
Collaborator

@SebastianAchilles: Request for testing this PR well received on jsczen2l1.int.jsc-zen2.easybuild-test.cluster

PR test command 'EB_PR=15406 EB_ARGS= /opt/software/slurm/bin/sbatch --job-name test_PR_15406 --ntasks=8 ~/boegelbot/eb_from_pr_upload_jsc-zen2.sh' executed!

  • exit code: 0
  • output:
Submitted batch job 1194

Test results coming soon (I hope)...

- notification for comment with ID 1124221707 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

@boegelbot
Copy link
Collaborator

Test report by @boegelbot
SUCCESS
Build succeeded for 1 out of 1 (1 easyconfigs in total)
cns1 - Linux Rocky Linux 8.5, x86_64, Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz (haswell), Python 3.6.8
See https://gist.github.com/110656578ef47b7d820b3c44727465a8 for a full test report.

@SebastianAchilles
Copy link
Member

Test report by @SebastianAchilles
SUCCESS
Build succeeded for 1 out of 1 (1 easyconfigs in total)
eb-centosstream9.novalocal - Linux CentOS Stream 9, x86_64, Intel Xeon Processor (Skylake, IBRS) (skylake_avx512), Python 3.9.10
See https://gist.github.com/22615f6b7f1f2e80c77213e99a978805 for a full test report.

@boegelbot
Copy link
Collaborator

Test report by @boegelbot
SUCCESS
Build succeeded for 1 out of 1 (1 easyconfigs in total)
jsczen2c1.int.jsc-zen2.easybuild-test.cluster - Linux Rocky Linux 8.5, x86_64, AMD EPYC 7742 64-Core Processor (zen2), Python 3.6.8
See https://gist.github.com/c0b1ba295f2efabd2531938c566b86ea for a full test report.

@SebastianAchilles
Copy link
Member

Test report by @SebastianAchilles
SUCCESS
Build succeeded for 1 out of 1 (1 easyconfigs in total)
zen2-ubuntu-eb - Linux Ubuntu 22.04, x86_64, AMD EPYC 7452 32-Core Processor (zen2), Python 3.10.4
See https://gist.github.com/4652b8adb94f02ac5f82070d0b4993a0 for a full test report.

@jfgrimm
Copy link
Member

jfgrimm commented May 12, 2022

Test report by @jfgrimm
SUCCESS
Build succeeded for 1 out of 1 (1 easyconfigs in total)
node134.pri.viking.alces.network - Linux CentOS Linux 7.9.2009, x86_64, Intel(R) Xeon(R) Gold 6138 CPU @ 2.00GHz (skylake_avx512), Python 3.6.8
See https://gist.github.com/b00a12196bbd3671cbfb1c5d1f3c13e7 for a full test report.

@verdurin
Copy link
Member

Test report by @verdurin
SUCCESS
Build succeeded for 1 out of 1 (1 easyconfigs in total)
easybuild-c7.novalocal - Linux CentOS Linux 7.9.2009, x86_64, Intel Xeon Processor (Skylake, IBRS), Python 3.6.8
See https://gist.github.com/9c7ddb45d2f6aafbd3c401d36c64b924 for a full test report.

@boegel
Copy link
Member

boegel commented May 25, 2022

Do you want to split off the FFTW MPI parts into a separate module for foss/2022a?
yes

It will need a few framework changes, in fftw.py to include both library paths.

@bartoldeman Any updates on the necessary changes to framework that are required for this split FFTW approach?

I would like to squeeze in those changes for the EasyBuild v4.5.5 release, which I would like to get out there ASAP...

@bartoldeman
Copy link
Contributor Author

@boegel I was waiting for Open MPI 4.1.4 to be released but that doesn't seem to be in the cards, so we can't have FFTW-MPI yet since gompi/2022a can't be there yet.
Will try with the rc1 PR then.

@boegel
Copy link
Member

boegel commented May 25, 2022

@bartoldeman It would be nice if we can get the necessary framework changes already included, since that'll make playing with FFTW-MPI and the candidate for foss/2022a significantly easier for a lot of people...

@boegel
Copy link
Member

boegel commented May 29, 2022

Test report by @boegel
SUCCESS
Build succeeded for 1 out of 1 (1 easyconfigs in total)
node3141.skitty.os - Linux RHEL 8.4, x86_64, Intel(R) Xeon(R) Gold 6140 CPU @ 2.30GHz, Python 3.6.8
See https://gist.github.com/0f8d478ca1c38cbb44913001ec45d451 for a full test report.

@boegel
Copy link
Member

boegel commented May 29, 2022

Test report by @boegel
SUCCESS
Build succeeded for 1 out of 1 (1 easyconfigs in total)
node3539.doduo.os - Linux RHEL 8.4, x86_64, AMD EPYC 7552 48-Core Processor (zen2), Python 3.6.8
See https://gist.github.com/c22162d7a8fbde0de6cb7a76f591438b for a full test report.

@boegel
Copy link
Member

boegel commented May 29, 2022

@boegel I was waiting for Open MPI 4.1.4 to be released but that doesn't seem to be in the cards, so we can't have FFTW-MPI yet since gompi/2022a can't be there yet. Will try with the rc1 PR then.

@bartoldeman Open MPI 4.1.4 was released, see also merged PR #15426

Is there any reason not to merge this PR yet?

@bartoldeman
Copy link
Contributor Author

This PR can be merged as is. The changes are needed to support FFTW.MPI only.

@bartoldeman
Copy link
Contributor Author

I'll send a PR for FFTW.MPI later today

@boegel boegel modified the milestones: 4.x, next release (4.5.5?) May 29, 2022
boegel
boegel approved these changes May 29, 2022
View reviewed changes
Copy link
Member

@boegel boegel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@boegel
Copy link
Member

boegel commented May 29, 2022

Going in, thanks @bartoldeman!

@boegel boegel merged commit 8ce2958 into easybuilders:develop May 29, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@boegel boegel boegel approved these changes

Assignees
No one assigned
Labels
new update
Projects
None yet
Milestone
4.5.5
Development

Successfully merging this pull request may close these issues.
7 participants
@bartoldeman @boegelbot @SebastianAchilles @branfosj @boegel @verdurin @jfgrimm