Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

use regular configure instead of wrapper script for recent UCX easyconfigs #20428

Merged
merged 2 commits into from
May 3, 2024

Conversation

bedroge
Copy link
Contributor

@bedroge bedroge commented Apr 26, 2024

The UCX easyconfigs for version 1.9.0 and newer are currently using configure_cmd = contrib/configure-release; this is a wrapper script around ./configure which passed some predefined flags (these haven't changed for many years), see:
https://github.com/openucx/ucx/blob/master/contrib/configure-release

While trying to install UCX on RISC-V, I ran into issues with the outdated config.guess included with UCX. The ConfigureMake easyblock should download a newer version and use that one, but it turns out that it only does that when the configure_cmd script includes a certain string (Generated by GNU Autoconf). Though the configure script does include it, this is not the case for the contrib/configure-release wrapper script, hence that step is skipped. By appending the options that this wrapper uses to configopts, we can just call ./configure itself, and then EB does make sure that an updated config.guess is being used. Additionally, it's more explicit in the easyconfig which flags are being passed.

@bedroge bedroge added enhancement EESSI Related to EESSI project riscv labels Apr 26, 2024
@bedroge
Copy link
Contributor Author

bedroge commented Apr 26, 2024

@boegelbot please test @ generoso

@boegelbot
Copy link
Collaborator

@bedroge: Request for testing this PR well received on login1

PR test command 'EB_PR=20428 EB_ARGS= EB_CONTAINER= EB_REPO=easybuild-easyconfigs /opt/software/slurm/bin/sbatch --job-name test_PR_20428 --ntasks=4 ~/boegelbot/eb_from_pr_upload_generoso.sh' executed!

  • exit code: 0
  • output:
Submitted batch job 13364

Test results coming soon (I hope)...

- notification for comment with ID 2079607477 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

@bedroge
Copy link
Contributor Author

bedroge commented Apr 26, 2024

@boegelbot please test @ jsc-zen3

@bedroge bedroge changed the title don't use configure wrapper script for recent UCX easyconfigs use regular configure instead of wrapper script for recent UCX easyconfigs Apr 26, 2024
@boegelbot
Copy link
Collaborator

@bedroge: Request for testing this PR well received on jsczen3l1.int.jsc-zen3.fz-juelich.de

PR test command 'if [[ develop != 'develop' ]]; then EB_BRANCH=develop ./easybuild_develop.sh 2> /dev/null 1>&2; EB_PREFIX=/home/boegelbot/easybuild/develop source init_env_easybuild_develop.sh; fi; EB_PR=20428 EB_ARGS= EB_CONTAINER= EB_REPO=easybuild-easyconfigs EB_BRANCH=develop /opt/software/slurm/bin/sbatch --job-name test_PR_20428 --ntasks=8 ~/boegelbot/eb_from_pr_upload_jsc-zen3.sh' executed!

  • exit code: 0
  • output:
Submitted batch job 4036

Test results coming soon (I hope)...

- notification for comment with ID 2079612506 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

@boegelbot
Copy link
Collaborator

Test report by @boegelbot
SUCCESS
Build succeeded for 9 out of 9 (9 easyconfigs in total)
cns1 - Linux Rocky Linux 8.9, x86_64, Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz (haswell), Python 3.6.8
See https://gist.github.com/boegelbot/755c932646bac0d833b12738b981ac26 for a full test report.

@boegelbot
Copy link
Collaborator

Test report by @boegelbot
SUCCESS
Build succeeded for 9 out of 9 (9 easyconfigs in total)
jsczen3c1.int.jsc-zen3.fz-juelich.de - Linux Rocky Linux 9.3, x86_64, AMD EPYC-Milan Processor (zen3), Python 3.9.18
See https://gist.github.com/boegelbot/c47e7ca8c6ba8d9e8e6a90fc030c2709 for a full test report.

@SebastianAchilles SebastianAchilles added this to the release after 4.9.1 milestone Apr 26, 2024
@branfosj
Copy link
Member

We also use contrib/configure-release in ucx_plugins.py. Do we want to look at changing that as well?

@bedroge
Copy link
Contributor Author

bedroge commented Apr 27, 2024

We also use contrib/configure-release in ucx_plugins.py. Do we want to look at changing that as well?

Ah, good catch, didn't know it was used in other places. I suppose it makes sense to use the same approach everywhere, so I can make a PR for that easyblock as well.

However, @SebastianAchilles found out that this patch is actually not really required anymore to solve the RISC-V issue (since the autoreconf command added in #20429 apparently updates the included config.guess, and then it does recognize RISC-V), so in that sense we don't really need this PR anymore. On the other hand, this approach may still be a bit cleaner anyway, since we don't depend on that wrapper script anymore?

edit: and it may still be useful for UCX 1.16.0. Even though that version does support RISC-V out of the box (and hence doesn't need the patch from #20429), they still ship a very old config.guess (which doesn't detect RISC-V...) in their tarball. This means that this version will either need to do an autoreconf on RISC-V or use the regular configure script (to make sure that EB's config.guess is used).
I can verify this by making a PR for UCX 1.16.0 as well.

bedroge added a commit to bedroge/easybuild-easyconfigs that referenced this pull request Apr 28, 2024
@bedroge
Copy link
Contributor Author

bedroge commented Apr 28, 2024

I've applied the same change to the UCX plugins easyblock in easybuilders/easybuild-easyblocks#3315. I've also opened a PR for UCX 1.16.0 (#20431), which demonstrates that this change is also still useful for RISC-V builds.

Copy link
Member

@SebastianAchilles SebastianAchilles left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this approach may still be a bit cleaner anyway

Yes, I share your opinion, using the regular configure seems a bit cleaner.

I've applied the same change to the UCX plugins easyblock in easybuilders/easybuild-easyblocks#3315. I've also opened a PR for UCX 1.16.0 (#20431), which demonstrates that this change is also still useful for RISC-V builds.

Since this change will be useful for future UCX easyconfigs for RISC-V I think it makes sense to merge this PR.

@branfosj Do you agree?

Copy link
Member

@boegel boegel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

make sense to circumvent the contrib/configure-release script, since it doesn't add much value, and causes trouble since updated config.guess is not picked up

@boegel
Copy link
Member

boegel commented May 3, 2024

Test report by @boegel
SUCCESS
Build succeeded for 9 out of 9 (9 easyconfigs in total)
node3147.skitty.os - Linux RHEL 8.8, x86_64, Intel(R) Xeon(R) Gold 6140 CPU @ 2.30GHz, Python 3.6.8
See https://gist.github.com/boegel/17a637b6b2616e41c6cafbcf3d01f8e9 for a full test report.

@boegel
Copy link
Member

boegel commented May 3, 2024

Going in, thanks @bedroge!

@boegel boegel merged commit a080c2f into easybuilders:develop May 3, 2024
9 checks passed
@bedroge bedroge deleted the ucx_dont_use_configure_wrapper branch May 3, 2024 17:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
EESSI Related to EESSI project enhancement riscv
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants