Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

{devel,vis}[fosscuda/2019b] torchvision v0.8.1, PyTorch v1.7.0 w/ Python 3.7.4, with MKL #11786

Conversation

terjekv
Copy link
Collaborator

@terjekv terjekv commented Nov 30, 2020

(created using eb --new-pr)

…orch-1.7.0-imkl.eb, PyTorch-1.7.0-fosscuda-2019b-Python-3.7.4-imkl.eb
@terjekv terjekv added the update label Nov 30, 2020
@terjekv terjekv changed the title {devel,vis}[fosscuda/2019b] torchvision v0.8.1, PyTorch v1.7.0 w/ Python 3.7.4 {devel,vis}[fosscuda/2019b] torchvision v0.8.1, PyTorch v1.7.0 w/ Python 3.7.4, with MKL Nov 30, 2020
@terjekv
Copy link
Collaborator Author

terjekv commented Nov 30, 2020

Requires #11636 for patches and fixes.

builddependencies = [
('CMake', '3.15.3'),
('hypothesis', '4.44.2', local_pyversionsuffix),
('libarchive', '3.4.2'),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This shouldn't be required. The error message from #11636 (comment) hints at cmake not pytorch so if at all it should be added to the cmake EC but first it should be checked why it uses the system cmake

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to check, is it libarchive or cmake that shouldn't be required in builddependencies?

Copy link
Contributor

@Flamefire Flamefire Nov 30, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

libarchive. The CMake 3.15 shouldn't need it. I see CMake-3.18.4-GCCcore-10.2.0.eb added it so I guess your system CMake is newer than 3.15 and something is wrong so the system cmake gets picked up

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, ick. No idea why this happens. My familiarity with CMake isn't great. :-(

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing with CMake here, only with the CMake module. As you posted the PyTorch build somehow uses /usr/bin/cmake3 which is clearly wrong. So try to load the CMake/3.15.3 module and run which cmake and which cmake3 to check if the module is broken.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And I checked the PyTorch code: CMAKE = find_executable('cmake3') or find_executable('cmake')
Looks like we need an update to the CMake easyblock

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, good catch. I doubt I have time to sort it right now. Make an issue of it and first one gets a cookie?

Copy link
Contributor

@Flamefire Flamefire Nov 30, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can I haz my cookie please: easybuilders/easybuild-easyblocks#2259

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🍪 !

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated in 117a9bd to not need libarchive, which I didn't include in the PR anyway... I was clearly ahead of myself this morning.

@terjekv
Copy link
Collaborator Author

terjekv commented Nov 30, 2020

This also requires the imkl modules from #11785.

@boegelbot
Copy link
Collaborator

@terjekv: Tests failed in GitHub Actions, see https://github.com/easybuilders/easybuild-easyconfigs/actions/runs/391563189
Output from first failing test suite run:

ERROR: test_dep_versions_per_toolchain_generation (test.easyconfigs.easyconfigs.EasyConfigTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/easyconfigs/easyconfigs.py", line 441, in test_dep_versions_per_toolchain_generation
    self.process_all_easyconfigs()
  File "test/easyconfigs/easyconfigs.py", line 121, in process_all_easyconfigs
    EasyConfigTest.parsed_easyconfigs, modules_tool(), retain_all_deps=True)
  File "/opt/hostedtoolcache/Python/2.7.18/x64/lib/python2.7/site-packages/easybuild/tools/robot.py", line 460, in resolve_dependencies
    raise_error_missing_deps(totally_missing, extra_msg="no easyconfig file or existing module found")
  File "/opt/hostedtoolcache/Python/2.7.18/x64/lib/python2.7/site-packages/easybuild/tools/robot.py", line 323, in raise_error_missing_deps
    raise EasyBuildError(error_msg)
EasyBuildError: 'Missing dependencies: typing-extensions/3.7.4.3-fosscuda-2019b-Python-3.7.4, imkl/2019.5.281-fosscuda-2019b (no easyconfig file or existing module found)'

======================================================================
ERROR: test_sanity_check_paths (test.easyconfigs.easyconfigs.EasyConfigTest)
Make sure specified sanity check paths adher to the requirements.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/easyconfigs/easyconfigs.py", line 504, in test_sanity_check_paths
    self.process_all_easyconfigs()
  File "test/easyconfigs/easyconfigs.py", line 121, in process_all_easyconfigs
    EasyConfigTest.parsed_easyconfigs, modules_tool(), retain_all_deps=True)
  File "/opt/hostedtoolcache/Python/2.7.18/x64/lib/python2.7/site-packages/easybuild/tools/robot.py", line 460, in resolve_dependencies
    raise_error_missing_deps(totally_missing, extra_msg="no easyconfig file or existing module found")
  File "/opt/hostedtoolcache/Python/2.7.18/x64/lib/python2.7/site-packages/easybuild/tools/robot.py", line 323, in raise_error_missing_deps
    raise EasyBuildError(error_msg)
EasyBuildError: 'Missing dependencies: typing-extensions/3.7.4.3-fosscuda-2019b-Python-3.7.4, imkl/2019.5.281-fosscuda-2019b (no easyconfig file or existing module found)'

======================================================================
FAIL: test__parse_easyconfig_PyTorch-1.7.0-fosscuda-2019b-Python-3.7.4-imkl.eb (test.easyconfigs.easyconfigs.EasyConfigTest)
Test for parsing of easyconfig PyTorch-1.7.0-fosscuda-2019b-Python-3.7.4-imkl.eb
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/easyconfigs/easyconfigs.py", line 1041, in innertest
    template_easyconfig_test(self, spec_path)
  File "test/easyconfigs/easyconfigs.py", line 939, in template_easyconfig_test
    self.assertTrue(os.path.isfile(patch_full), msg)
AssertionError: Patch file /home/runner/work/easybuild-easyconfigs/easybuild-easyconfigs/easybuild/easyconfigs/p/PyTorch/PyTorch-1.7.0_fix_altivec_defines.patch is available for PyTorch-1.7.0-fosscuda-2019b-Python-3.7.4-imkl.eb

======================================================================
FAIL: test_style_conformance (test.easyconfigs.styletests.StyleTest)
Check the easyconfigs for style
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/easyconfigs/styletests.py", line 57, in test_style_conformance
    self.assertEqual(result, 0, "Found code style errors (and/or warnings): %s" % result)
AssertionError: Found code style errors (and/or warnings): 5

----------------------------------------------------------------------
Ran 10987 tests in 339.573s

FAILED (failures=2, errors=2)
ERROR: Not all tests were successful.

bleep, bloop, I'm just a bot (boegelbot v20200716.01)
Please talk to my owner @boegel if you notice you me acting stupid),
or submit a pull request to https://github.com/boegel/boegelbot fix the problem.

@Flamefire
Copy link
Contributor

Can you add a ToDo to resync the 1.7 with the other PR once that is finished?

@terjekv
Copy link
Collaborator Author

terjekv commented Dec 1, 2020

@Flamefire
Copy link
Contributor

Flamefire commented Jun 10, 2021

@terjekv Ready to be synced and please update to use the easyblock as in #13102

@boegel boegel added this to the 4.x milestone Jan 13, 2024
@boegel
Copy link
Member

boegel commented Jan 13, 2024

closing this since fosscuda/2019b is no longer supported, see https://docs.easybuild.io/policies/toolchains

@boegel boegel closed this Jan 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants