add wrapper for Julia with linking safeguards and delegate environment setup to JuliaPackage #20103

lexming · 2024-03-13T10:29:48Z

(created using eb --new-pr)

Fixes #17692
Alternative fix for #19650

Depends on:

make a project environment and manage LOAD_PATH for JuliaPackage easybuild-easyblocks#3239

Changelog:

remove modextravars, the depot path will be set by the JuliaPackage easyblock with make a project environment and manage LOAD_PATH for JuliaPackage easybuild-easyblocks#3239
execute julia through a wrapper to ensure that the libraries bundled in Julia have precedence

…t setup to JuliaPackage

lexming · 2024-03-13T15:12:47Z

@boegelbot: please test @ generoso

boegelbot · 2024-03-13T15:15:09Z

@lexming: Request for testing this PR well received on login1

PR test command 'EB_PR=20103 EB_ARGS= EB_CONTAINER= EB_REPO=easybuild-easyconfigs /opt/software/slurm/bin/sbatch --job-name test_PR_20103 --ntasks=4 ~/boegelbot/eb_from_pr_upload_generoso.sh' executed!

exit code: 0
output:

Submitted batch job 13092

Test results coming soon (I hope)...

- notification for comment with ID 1994631082 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

boegelbot · 2024-03-13T15:23:11Z

Test report by @boegelbot
FAILED
Build succeeded for 14 out of 15 (15 easyconfigs in total)
cns2 - Linux Rocky Linux 8.9, x86_64, Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz (haswell), Python 3.6.8
See https://gist.github.com/boegelbot/c1e1c468ed754bb3f8efaa3c6a2ede97 for a full test report.

lexming · 2024-03-13T15:27:04Z

@boegelbot: please test @ generoso
EB_ARGS="Julia-1.10.0-linux-x86_64.eb Julia-1.6.5-linux-x86_64.eb Julia-1.6.6-linux-x86_64.eb Julia-1.6.7-linux-x86_64.eb Julia-1.7.0-linux-x86_64.eb Julia-1.7.1-linux-x86_64.eb Julia-1.7.2-linux-x86_64.eb Julia-1.7.3-linux-x86_64.eb Julia-1.8.0-linux-x86_64.eb Julia-1.8.2-linux-x86_64.eb Julia-1.8.5-linux-x86_64.eb Julia-1.9.0-linux-x86_64.eb Julia-1.9.2-linux-x86_64.eb Julia-1.9.3-linux-x86_64.eb"

boegelbot · 2024-03-13T15:30:06Z

@lexming: Request for testing this PR well received on login1

PR test command 'EB_PR=20103 EB_ARGS="Julia-1.10.0-linux-x86_64.eb Julia-1.6.5-linux-x86_64.eb Julia-1.6.6-linux-x86_64.eb Julia-1.6.7-linux-x86_64.eb Julia-1.7.0-linux-x86_64.eb Julia-1.7.1-linux-x86_64.eb Julia-1.7.2-linux-x86_64.eb Julia-1.7.3-linux-x86_64.eb Julia-1.8.0-linux-x86_64.eb Julia-1.8.2-linux-x86_64.eb Julia-1.8.5-linux-x86_64.eb Julia-1.9.0-linux-x86_64.eb Julia-1.9.2-linux-x86_64.eb Julia-1.9.3-linux-x86_64.eb" EB_CONTAINER= EB_REPO=easybuild-easyconfigs /opt/software/slurm/bin/sbatch --job-name test_PR_20103 --ntasks=4 ~/boegelbot/eb_from_pr_upload_generoso.sh' executed!

exit code: 0
output:

Submitted batch job 13093

Test results coming soon (I hope)...

- notification for comment with ID 1994663398 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

boegelbot · 2024-03-13T15:37:04Z

Test report by @boegelbot
SUCCESS
Build succeeded for 14 out of 14 (14 easyconfigs in total)
cns2 - Linux Rocky Linux 8.9, x86_64, Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz (haswell), Python 3.6.8
See https://gist.github.com/boegelbot/9ef90dbf705e2642e958f984db2deca0 for a full test report.

lexming · 2024-03-13T15:47:27Z

@boegelbot: please test @ jsc-zen3
EB_ARGS="Julia-1.10.0-linux-x86_64.eb Julia-1.6.5-linux-x86_64.eb Julia-1.6.6-linux-x86_64.eb Julia-1.6.7-linux-x86_64.eb Julia-1.7.0-linux-x86_64.eb Julia-1.7.1-linux-x86_64.eb Julia-1.7.2-linux-x86_64.eb Julia-1.7.3-linux-x86_64.eb Julia-1.8.0-linux-x86_64.eb Julia-1.8.2-linux-x86_64.eb Julia-1.8.5-linux-x86_64.eb Julia-1.9.0-linux-x86_64.eb Julia-1.9.2-linux-x86_64.eb Julia-1.9.3-linux-x86_64.eb"

boegelbot · 2024-03-13T15:50:08Z

@lexming: Request for testing this PR well received on jsczen3l1.int.jsc-zen3.fz-juelich.de

PR test command 'if [[ develop != 'develop' ]]; then EB_BRANCH=develop ./easybuild_develop.sh 2> /dev/null 1>&2; EB_PREFIX=/home/boegelbot/easybuild/develop source init_env_easybuild_develop.sh; fi; EB_PR=20103 EB_ARGS="Julia-1.10.0-linux-x86_64.eb Julia-1.6.5-linux-x86_64.eb Julia-1.6.6-linux-x86_64.eb Julia-1.6.7-linux-x86_64.eb Julia-1.7.0-linux-x86_64.eb Julia-1.7.1-linux-x86_64.eb Julia-1.7.2-linux-x86_64.eb Julia-1.7.3-linux-x86_64.eb Julia-1.8.0-linux-x86_64.eb Julia-1.8.2-linux-x86_64.eb Julia-1.8.5-linux-x86_64.eb Julia-1.9.0-linux-x86_64.eb Julia-1.9.2-linux-x86_64.eb Julia-1.9.3-linux-x86_64.eb" EB_CONTAINER= EB_REPO=easybuild-easyconfigs EB_BRANCH=develop /opt/software/slurm/bin/sbatch --job-name test_PR_20103 --ntasks=8 ~/boegelbot/eb_from_pr_upload_jsc-zen3.sh' executed!

exit code: 0
output:

Submitted batch job 3761

Test results coming soon (I hope)...

- notification for comment with ID 1994731844 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

boegelbot · 2024-03-13T15:55:35Z

Test report by @boegelbot
SUCCESS
Build succeeded for 14 out of 14 (14 easyconfigs in total)
jsczen3c1.int.jsc-zen3.fz-juelich.de - Linux Rocky Linux 9.3, x86_64, AMD EPYC-Milan Processor (zen3), Python 3.9.18
See https://gist.github.com/boegelbot/9ab99dcc62c3d8f6d499f02652b9d5f3 for a full test report.

jfgrimm · 2024-03-27T09:46:35Z

an alternative solution would be to replace the RUNPATH sections of all binaries/libraries in the Julia install dir with RPATH instead (so it takes precedence over LD_LIBRARY_PATH)

I played around with this, and something like this would do the trick:

find . -type f -exec sh -c "readelf -d {} | grep -Po1 '(?<=Library runpath: \[).*(?=\])' | xargs -r -I rpth patchelf --set-rpath \'rpth\' --force-rpath {}" \;

lexming · 2024-07-02T23:18:30Z

@boegelbot: please test @ generoso
EB_ARGS="Julia-1.10.0-linux-x86_64.eb Julia-1.6.5-linux-x86_64.eb Julia-1.6.6-linux-x86_64.eb Julia-1.6.7-linux-x86_64.eb Julia-1.7.0-linux-x86_64.eb Julia-1.7.1-linux-x86_64.eb Julia-1.7.2-linux-x86_64.eb Julia-1.7.3-linux-x86_64.eb Julia-1.8.0-linux-x86_64.eb Julia-1.8.2-linux-x86_64.eb Julia-1.8.5-linux-x86_64.eb Julia-1.9.0-linux-x86_64.eb Julia-1.9.2-linux-x86_64.eb Julia-1.9.3-linux-x86_64.eb"

boegelbot · 2024-07-02T23:20:05Z

@lexming: Request for testing this PR well received on login1

PR test command 'EB_PR=20103 EB_ARGS="Julia-1.10.0-linux-x86_64.eb Julia-1.6.5-linux-x86_64.eb Julia-1.6.6-linux-x86_64.eb Julia-1.6.7-linux-x86_64.eb Julia-1.7.0-linux-x86_64.eb Julia-1.7.1-linux-x86_64.eb Julia-1.7.2-linux-x86_64.eb Julia-1.7.3-linux-x86_64.eb Julia-1.8.0-linux-x86_64.eb Julia-1.8.2-linux-x86_64.eb Julia-1.8.5-linux-x86_64.eb Julia-1.9.0-linux-x86_64.eb Julia-1.9.2-linux-x86_64.eb Julia-1.9.3-linux-x86_64.eb" EB_CONTAINER= EB_REPO=easybuild-easyconfigs /opt/software/slurm/bin/sbatch --job-name test_PR_20103 --ntasks=4 ~/boegelbot/eb_from_pr_upload_generoso.sh' executed!

exit code: 0
output:

Submitted batch job 13863

Test results coming soon (I hope)...

- notification for comment with ID 2204662208 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

boegelbot · 2024-07-02T23:28:08Z

Test report by @boegelbot
SUCCESS
Build succeeded for 14 out of 14 (14 easyconfigs in total)
cns1 - Linux Rocky Linux 8.9, x86_64, Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz (haswell), Python 3.6.8
See https://gist.github.com/boegelbot/b9743efa5224349b20610377f2c62e82 for a full test report.

branfosj · 2024-07-03T10:53:26Z

Test report by @branfosj
SUCCESS
Build succeeded for 14 out of 14 (14 easyconfigs in total)
bear-pg0105u03a - Linux RHEL 8.6, x86_64, Intel(R) Xeon(R) Platinum 8360Y CPU @ 2.40GHz (icelake), Python 3.6.8
See https://gist.github.com/branfosj/ce0202cec3746c5093b9a991f0e087e6 for a full test report.

branfosj · 2024-07-03T11:42:44Z

Going in, thanks @lexming!

lexming added 2 commits March 13, 2024 11:29

add wrapper for Julia with linking safeguards and delegate environmen…

9a12aa2

…t setup to JuliaPackage

adding file julia.wrapper

36fdc53

lexming mentioned this pull request Mar 13, 2024

strip Suppressor out of ResistanceGA #20106

Merged

2 tasks

lexming mentioned this pull request Mar 13, 2024

Segmentation fault when trying to test ResistanceGA #17692

Closed

lexming added the bug fix label Mar 13, 2024

lexming added 3 commits May 8, 2024 10:17

use generic posix shell in julia wrapper

72a2299

Merge branch 'develop' into 20240313112943_new_pr_Julia1100

45a5f1b

update checksum of julia.wrapper

116ab0c

lexming mentioned this pull request Jun 10, 2024

{lang,lib}[system/system] Julia v1.10.3, IJulia v1.24.2 w/ Julia 1.10.3, linux x86_64 #20662

Merged

1 task

branfosj added this to the release after 4.9.2 milestone Jul 3, 2024

branfosj approved these changes Jul 3, 2024

View reviewed changes

branfosj merged commit 8f931cb into easybuilders:develop Jul 3, 2024
9 checks passed

lexming deleted the 20240313112943_new_pr_Julia1100 branch July 4, 2024 09:10

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add wrapper for Julia with linking safeguards and delegate environment setup to JuliaPackage #20103

add wrapper for Julia with linking safeguards and delegate environment setup to JuliaPackage #20103

lexming commented Mar 13, 2024 •

edited by branfosj

Loading

lexming commented Mar 13, 2024

boegelbot commented Mar 13, 2024

boegelbot commented Mar 13, 2024

lexming commented Mar 13, 2024

boegelbot commented Mar 13, 2024

boegelbot commented Mar 13, 2024

lexming commented Mar 13, 2024

boegelbot commented Mar 13, 2024

boegelbot commented Mar 13, 2024

jfgrimm commented Mar 27, 2024

lexming commented Jul 2, 2024

boegelbot commented Jul 2, 2024

boegelbot commented Jul 2, 2024

branfosj commented Jul 3, 2024

branfosj commented Jul 3, 2024

add wrapper for Julia with linking safeguards and delegate environment setup to JuliaPackage #20103

add wrapper for Julia with linking safeguards and delegate environment setup to JuliaPackage #20103

Conversation

lexming commented Mar 13, 2024 • edited by branfosj Loading

lexming commented Mar 13, 2024

boegelbot commented Mar 13, 2024

boegelbot commented Mar 13, 2024

lexming commented Mar 13, 2024

boegelbot commented Mar 13, 2024

boegelbot commented Mar 13, 2024

lexming commented Mar 13, 2024

boegelbot commented Mar 13, 2024

boegelbot commented Mar 13, 2024

jfgrimm commented Mar 27, 2024

lexming commented Jul 2, 2024

boegelbot commented Jul 2, 2024

boegelbot commented Jul 2, 2024

branfosj commented Jul 3, 2024

branfosj commented Jul 3, 2024

lexming commented Mar 13, 2024 •

edited by branfosj

Loading