Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

enhance OpenMPI easyblock to catch any --with-ucx* configure options #2230

Merged
merged 1 commit into from
Dec 1, 2020

Conversation

lexming
Copy link
Contributor

@lexming lexming commented Nov 11, 2020

This little change covers two additional cases where it is necessary to disable verbs:

  • configopts with --with-ucx not having any explicit path
  • configopts with --with-ucx-libdir

@lexming lexming added this to the 4.x milestone Nov 11, 2020
@lexming
Copy link
Contributor Author

lexming commented Nov 11, 2020

Positive test with a custom OpenMPI easyconfig using --with-ucx:
https://gist.github.com/68eaf4245f6b4ce2899b9c2f9a2bef48

@lexming
Copy link
Contributor Author

lexming commented Nov 11, 2020

Positive with standard OpenMPI-4.0.3-gcccuda-2020a.eb:
https://gist.github.com/aa873272523739afd3d7f588ab84b6be

@boegel
Copy link
Member

boegel commented Nov 11, 2020

@lexming You can upload test reports in easyblock PRs now, using --include-easyblocks-from-pr 2230 --upload-test-report (as long as --from-pr isn't used, the test report will go into the easyblock PR)

@boegel boegel changed the title catch any UCX configopts in openmpi enhance OpenMPI easyblock to catch any --with-ucx* configure options Nov 11, 2020
@boegel boegel modified the milestones: 4.x, 4.3.2 Nov 11, 2020
@bartoldeman
Copy link
Contributor

I'm having one issue with this hard check: we sometimes need verbs at runtime for DDT, see this bug:
open-mpi/ompi#6464

The way we deal with this is having the OpenMPI module set this (in Lua code):

setenv("OMPI_MCA_btl","^openib")

so that it's possible to use --mca pml ob1 --mca btl openib if needed.

Would it still be possible to somehow force both UCX and Verbs without modifying the easyblock?

@lexming
Copy link
Contributor Author

lexming commented Nov 12, 2020

@bartoldeman these checks are under if config_opt_unused('verbs'):, so you can manually overwrite this behaviour by manually adding --with-verbs to the configopts of the easyconfig.

@boegel
Copy link
Member

boegel commented Dec 1, 2020

Test report by @boegel

Overview of tested easyconfigs (in order)

  • SUCCESS OpenMPI-4.0.3-GCC-9.3.0.eb
  • SUCCESS OpenMPI-3.1.1-GCC-7.3.0-2.30.eb
  • SUCCESS OpenMPI-3.1.4-GCC-8.3.0.eb

Build succeeded for 3 out of 3 (3 easyconfigs in total)
generoso - Linux centos linux 8.2.2004, x86_64, Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz (haswell), Python 3.6.8
See https://gist.github.com/94774804509d837ebe98e6af6464a5a9 for a full test report.

Copy link
Member

@boegel boegel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@boegel boegel merged commit 35099c2 into easybuilders:develop Dec 1, 2020
@lexming lexming deleted the ompi-ucx-verbs branch December 1, 2020 18:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants