{data}[fosscuda/2019b] JAX v0.1.77 w/ Python 3.7.4 #11595

edmondac · 2020-10-30T13:21:48Z

(created using eb --new-pr)

Requires easybuilders/easybuild-easyblocks#2262

edmondac · 2020-10-30T14:50:30Z

Test report by @bear-rsg
SUCCESS
Build succeeded for 1 out of 1 (1 easyconfigs in this PR)
bear-pg0212u27a.bear.cluster - Linux centos linux 7.8.2003, x86_64, Intel(R) Xeon(R) CPU E5-2640 v4 @ 2.40GHz, Python 2.7.5
See https://gist.github.com/0e2556e18868fdd34d432a294b74aba2 for a full test report.

edmondac · 2020-10-30T16:13:23Z

Note bear-rsg#371 - which might need a Bazel fix pushing upstream

surak · 2020-10-30T16:29:31Z

I have added the static setting to Bazel, and in fact I get a bazel without the dynamic link to libstdc++:

 ldd /p/software/juwels/stages/Devel-2020/software/Bazel/3.6.0-GCCcore-9.3.0/bin/bazel 
	linux-vdso.so.1 (0x00007ffd4aaf2000)
	librt.so.1 => /usr/lib64/librt.so.1 (0x0000153a2b185000)
	libdl.so.2 => /usr/lib64/libdl.so.2 (0x0000153a2af81000)
	libpthread.so.0 => /usr/lib64/libpthread.so.0 (0x0000153a2ad61000)
	libm.so.6 => /usr/lib64/libm.so.6 (0x0000153a2a9df000)
	libc.so.6 => /usr/lib64/libc.so.6 (0x0000153a2a61d000)
	/lib64/ld-linux-x86-64.so.2 (0x0000153a2b38e000)

And I hit the libstdc++ problem further down the road. A more complete log here:

jax-ml/jax#4743

edmondac · 2020-10-30T16:48:39Z

I'll try uploading some EL8 test reports on haswell and POWER9...

surak · 2020-10-30T18:47:12Z

I'll try uploading some EL8 test reports on haswell and POWER9...

Ok. It was my 0.2.5. I went with 0.1.77 and it installs

edmondac · 2020-10-30T21:04:51Z

Test report by @bear-rsg
SUCCESS
Build succeeded for 51 out of 51 (1 easyconfigs in this PR)
bear-pg0212u15b.bear.cluster - Linux centos linux 8.2.2004, x86_64, Intel(R) Xeon(R) CPU E5-2640 v4 @ 2.40GHz, Python 3.6.8
See https://gist.github.com/1db6f62ad2e51995c055202d4d5696c8 for a full test report.

easybuild/easyconfigs/j/JAX/JAX-0.1.77-fosscuda-2019b-Python-3.7.4.eb

….7.4.eb Avoid it using the system `ld.gold`

surak · 2020-11-13T17:08:23Z

My suggestion comes from @Flamefire having had the same problem in tensorflow on centos 7. I didn't see this issue before because on Centos8, the ld.gold is roughly the same version of that on our toolchain. tensorflow/tensorflow#39263

boegelbot · 2020-12-02T22:23:21Z

@edmondac: Tests failed in GitHub Actions, see https://github.com/easybuilders/easybuild-easyconfigs/actions/runs/397069336
Output from first failing test suite run:

ERROR: test__parse_easyconfig_JAX-0.1.77-fosscuda-2019b-Python-3.7.4.eb (test.easyconfigs.easyconfigs.EasyConfigTest)
Test for parsing of easyconfig JAX-0.1.77-fosscuda-2019b-Python-3.7.4.eb
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/easyconfigs/easyconfigs.py", line 1041, in innertest
    template_easyconfig_test(self, spec_path)
  File "test/easyconfigs/easyconfigs.py", line 837, in template_easyconfig_test
    ecs = process_easyconfig(spec)
  File "/opt/hostedtoolcache/Python/2.7.18/x64/lib/python2.7/site-packages/easybuild/framework/easyconfig/easyconfig.py", line 1977, in process_easyconfig
    raise EasyBuildError("Failed to process easyconfig %s: %s", spec, err.msg)
EasyBuildError: "Failed to process easyconfig /home/runner/work/easybuild-easyconfigs/easybuild-easyconfigs/easybuild/easyconfigs/j/JAX/JAX-0.1.77-fosscuda-2019b-Python-3.7.4.eb: No software-specific easyblock 'EB_JAX' found for JAX"

======================================================================
FAIL: test_changed_files_pull_request (test.easyconfigs.easyconfigs.EasyConfigTest)
Specific checks only done for the (easyconfig) files that were changed in a pull request.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/easyconfigs/easyconfigs.py", line 805, in test_changed_files_pull_request
    self.assertTrue(False, error_msg)
AssertionError: Failed to find parsed easyconfig for JAX-0.1.77-fosscuda-2019b-Python-3.7.4.eb (and could not isolate it in easyconfigs archive either)

----------------------------------------------------------------------
Ran 10993 tests in 874.636s

FAILED (failures=1, errors=1)
ERROR: Not all tests were successful.

bleep, bloop, I'm just a bot (boegelbot v20200716.01)
Please talk to my owner @boegel if you notice you me acting stupid),
or submit a pull request to https://github.com/boegel/boegelbot fix the problem.

easybuild/easyconfigs/b/Bazel/Bazel-2.0.0-GCCcore-8.3.0.eb

edmondac · 2020-12-03T10:11:55Z

I think easybuilders/easybuild-easyblocks#2262 is required before the CI will pass

boegelbot · 2020-12-03T10:23:10Z

@edmondac: Tests failed in GitHub Actions, see https://github.com/easybuilders/easybuild-easyconfigs/actions/runs/398095163
Output from first failing test suite run:

ERROR: test__parse_easyconfig_Bazel-2.0.0-GCCcore-8.3.0.eb (test.easyconfigs.easyconfigs.EasyConfigTest)
Test for parsing of easyconfig Bazel-2.0.0-GCCcore-8.3.0.eb
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/easyconfigs/easyconfigs.py", line 1041, in innertest
    template_easyconfig_test(self, spec_path)
  File "test/easyconfigs/easyconfigs.py", line 837, in template_easyconfig_test
    ecs = process_easyconfig(spec)
  File "/opt/hostedtoolcache/Python/2.7.18/x64/lib/python2.7/site-packages/easybuild/framework/easyconfig/easyconfig.py", line 1977, in process_easyconfig
    raise EasyBuildError("Failed to process easyconfig %s: %s", spec, err.msg)
EasyBuildError: "Failed to process easyconfig /home/runner/work/easybuild-easyconfigs/easybuild-easyconfigs/easybuild/easyconfigs/b/Bazel/Bazel-2.0.0-GCCcore-8.3.0.eb: Use of 1 unknown easyconfig parameters detected in Bazel-2.0.0-GCCcore-8.3.0.eb: static\nIf these are just local variables please rename them to start with 'local_', or try using --fix-deprecated-easyconfigs to do this automatically.\nFor more information, see https://easybuild.readthedocs.io/en/latest/Easyconfig-files-local-variables.html ."

======================================================================
ERROR: test__parse_easyconfig_JAX-0.1.77-fosscuda-2019b-Python-3.7.4.eb (test.easyconfigs.easyconfigs.EasyConfigTest)
Test for parsing of easyconfig JAX-0.1.77-fosscuda-2019b-Python-3.7.4.eb
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/easyconfigs/easyconfigs.py", line 1041, in innertest
    template_easyconfig_test(self, spec_path)
  File "test/easyconfigs/easyconfigs.py", line 837, in template_easyconfig_test
    ecs = process_easyconfig(spec)
  File "/opt/hostedtoolcache/Python/2.7.18/x64/lib/python2.7/site-packages/easybuild/framework/easyconfig/easyconfig.py", line 1977, in process_easyconfig
    raise EasyBuildError("Failed to process easyconfig %s: %s", spec, err.msg)
EasyBuildError: "Failed to process easyconfig /home/runner/work/easybuild-easyconfigs/easybuild-easyconfigs/easybuild/easyconfigs/j/JAX/JAX-0.1.77-fosscuda-2019b-Python-3.7.4.eb: No software-specific easyblock 'EB_JAX' found for JAX"

======================================================================
ERROR: test_dep_versions_per_toolchain_generation (test.easyconfigs.easyconfigs.EasyConfigTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/easyconfigs/easyconfigs.py", line 441, in test_dep_versions_per_toolchain_generation
    self.process_all_easyconfigs()
  File "test/easyconfigs/easyconfigs.py", line 121, in process_all_easyconfigs
    EasyConfigTest.parsed_easyconfigs, modules_tool(), retain_all_deps=True)
  File "/opt/hostedtoolcache/Python/2.7.18/x64/lib/python2.7/site-packages/easybuild/tools/robot.py", line 436, in resolve_dependencies
    processed_ecs = process_easyconfig(path, validate=not retain_all_deps, hidden=hidden)
  File "/opt/hostedtoolcache/Python/2.7.18/x64/lib/python2.7/site-packages/easybuild/framework/easyconfig/easyconfig.py", line 1977, in process_easyconfig
    raise EasyBuildError("Failed to process easyconfig %s: %s", spec, err.msg)
EasyBuildError: "Failed to process easyconfig /home/runner/work/easybuild-easyconfigs/easybuild-easyconfigs/easybuild/easyconfigs/b/Bazel/Bazel-2.0.0-GCCcore-8.3.0.eb: Use of 1 unknown easyconfig parameters detected in Bazel-2.0.0-GCCcore-8.3.0.eb: static\nIf these are just local variables please rename them to start with 'local_', or try using --fix-deprecated-easyconfigs to do this automatically.\nFor more information, see https://easybuild.readthedocs.io/en/latest/Easyconfig-files-local-variables.html ."

======================================================================
ERROR: test_sanity_check_paths (test.easyconfigs.easyconfigs.EasyConfigTest)
Make sure specified sanity check paths adher to the requirements.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/easyconfigs/easyconfigs.py", line 504, in test_sanity_check_paths
    self.process_all_easyconfigs()
  File "test/easyconfigs/easyconfigs.py", line 121, in process_all_easyconfigs
    EasyConfigTest.parsed_easyconfigs, modules_tool(), retain_all_deps=True)
  File "/opt/hostedtoolcache/Python/2.7.18/x64/lib/python2.7/site-packages/easybuild/tools/robot.py", line 436, in resolve_dependencies
    processed_ecs = process_easyconfig(path, validate=not retain_all_deps, hidden=hidden)
  File "/opt/hostedtoolcache/Python/2.7.18/x64/lib/python2.7/site-packages/easybuild/framework/easyconfig/easyconfig.py", line 1977, in process_easyconfig
    raise EasyBuildError("Failed to process easyconfig %s: %s", spec, err.msg)
EasyBuildError: "Failed to process easyconfig /home/runner/work/easybuild-easyconfigs/easybuild-easyconfigs/easybuild/easyconfigs/b/Bazel/Bazel-2.0.0-GCCcore-8.3.0.eb: Use of 1 unknown easyconfig parameters detected in Bazel-2.0.0-GCCcore-8.3.0.eb: static\nIf these are just local variables please rename them to start with 'local_', or try using --fix-deprecated-easyconfigs to do this automatically.\nFor more information, see https://easybuild.readthedocs.io/en/latest/Easyconfig-files-local-variables.html ."

======================================================================
FAIL: test_changed_files_pull_request (test.easyconfigs.easyconfigs.EasyConfigTest)
Specific checks only done for the (easyconfig) files that were changed in a pull request.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/easyconfigs/easyconfigs.py", line 805, in test_changed_files_pull_request
    self.assertTrue(False, error_msg)
AssertionError: Failed to find parsed easyconfig for Bazel-2.0.0-GCCcore-8.3.0.eb (and could not isolate it in easyconfigs archive either)

----------------------------------------------------------------------
Ran 11003 tests in 368.157s

FAILED (failures=1, errors=4)
ERROR: Not all tests were successful.

bleep, bloop, I'm just a bot (boegelbot v20200716.01)
Please talk to my owner @boegel if you notice you me acting stupid),
or submit a pull request to https://github.com/boegel/boegelbot fix the problem.

edmondac · 2020-12-03T11:03:46Z

Test report by @bear-rsg
SUCCESS
Build succeeded for 2 out of 2 (2 easyconfigs in total)
bear-pg0306u19a - Linux RHEL 8.2, POWER, 8335-GTX, Python 3.6.8
See https://gist.github.com/7cd7ec68c6b92f45843b7ca27c26c5e8 for a full test report.

edmondac · 2020-12-03T11:08:42Z

Test report by @bear-rsg
SUCCESS
Build succeeded for 2 out of 2 (2 easyconfigs in total)
bear-pg0305u05a.bear.cluster - Linux RHEL 7.6, POWER, 8335-GTX, Python 3.6.8
See https://gist.github.com/ee41677edc98dee8648002cb546cb508 for a full test report.

edmondac · 2020-12-03T11:14:56Z

Test report by @bear-rsg
SUCCESS
Build succeeded for 2 out of 2 (2 easyconfigs in total)
bear-pg0212u15b.bear.cluster - Linux centos linux 8.2.2004, x86_64, Intel(R) Xeon(R) CPU E5-2640 v4 @ 2.40GHz, Python 3.6.8
See https://gist.github.com/a678a7a38383fdb33f8f9be192b396da for a full test report.

branfosj · 2021-08-12T12:04:39Z

closing - we'll go with #13622 instead

edmondac and others added 3 commits October 30, 2020 13:21

adding easyconfigs: JAX-0.1.77-fosscuda-2019b-Python-3.7.4.eb

85b0fde

Update JAX-0.1.77-fosscuda-2019b-Python-3.7.4.eb

ec0a3ca

Bazel is just a build dep

ba8b3e4

edmondac mentioned this pull request Oct 30, 2020

The same problem of libc on jax appears for flatbuffers during jax install bear-rsg/easybuild-easyconfigs#371

Closed

boegel requested changes Nov 2, 2020

View reviewed changes

boegel added the new label Nov 2, 2020

boegel added this to the 4.x milestone Nov 2, 2020

edmondac commented Nov 13, 2020

View reviewed changes

easybuild/easyconfigs/j/JAX/JAX-0.1.77-fosscuda-2019b-Python-3.7.4.eb Outdated Show resolved Hide resolved

Update easybuild/easyconfigs/j/JAX/JAX-0.1.77-fosscuda-2019b-Python-3…

fbde4aa

….7.4.eb Avoid it using the system `ld.gold`

edmondac mentioned this pull request Dec 2, 2020

add custom easyblock for JAX easybuilders/easybuild-easyblocks#2262

Closed

edmondac added 3 commits December 2, 2020 15:58

Use the easyblock

7e2a6b7

Removing everything that's moved to the easyblock

724f5f9

Bazel on EL8 has glibc conflicts

fa80d2b

Flamefire reviewed Dec 3, 2020

View reviewed changes

easybuild/easyconfigs/b/Bazel/Bazel-2.0.0-GCCcore-8.3.0.eb Outdated Show resolved Hide resolved

Move Bazel static linking into easyblock

5e0e4cb

branfosj closed this Aug 12, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

{data}[fosscuda/2019b] JAX v0.1.77 w/ Python 3.7.4 #11595

{data}[fosscuda/2019b] JAX v0.1.77 w/ Python 3.7.4 #11595

edmondac commented Oct 30, 2020 •

edited

Loading

edmondac commented Oct 30, 2020

edmondac commented Oct 30, 2020

surak commented Oct 30, 2020 •

edited

Loading

edmondac commented Oct 30, 2020

surak commented Oct 30, 2020

edmondac commented Oct 30, 2020

surak commented Nov 13, 2020 •

edited

Loading

boegelbot commented Dec 2, 2020

edmondac commented Dec 3, 2020

boegelbot commented Dec 3, 2020

edmondac commented Dec 3, 2020

edmondac commented Dec 3, 2020

edmondac commented Dec 3, 2020

branfosj commented Aug 12, 2021

{data}[fosscuda/2019b] JAX v0.1.77 w/ Python 3.7.4 #11595

{data}[fosscuda/2019b] JAX v0.1.77 w/ Python 3.7.4 #11595

Conversation

edmondac commented Oct 30, 2020 • edited Loading

edmondac commented Oct 30, 2020

edmondac commented Oct 30, 2020

surak commented Oct 30, 2020 • edited Loading

edmondac commented Oct 30, 2020

surak commented Oct 30, 2020

edmondac commented Oct 30, 2020

surak commented Nov 13, 2020 • edited Loading

boegelbot commented Dec 2, 2020

edmondac commented Dec 3, 2020

boegelbot commented Dec 3, 2020

edmondac commented Dec 3, 2020

edmondac commented Dec 3, 2020

edmondac commented Dec 3, 2020

branfosj commented Aug 12, 2021

edmondac commented Oct 30, 2020 •

edited

Loading

surak commented Oct 30, 2020 •

edited

Loading

surak commented Nov 13, 2020 •

edited

Loading