Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

use DCPU_BASELINE=DETECT for OpenCV when default optarch compiler option is used #2954

Merged
merged 7 commits into from
Aug 30, 2023
Merged

use DCPU_BASELINE=DETECT for OpenCV when default optarch compiler option is used #2954

merged 7 commits into from
Aug 30, 2023

Conversation

smoors
Copy link
Contributor

@smoors smoors commented Jun 24, 2023 •
edited
Loading

(created using eb --new-pr)

sets -DCPU_BASELINE=DETECT also if

  • GCC is used and optarch is set to march=native (default for GCC)
  • intel is used and optarch is set to xHost (default for intel)

avoids build failure for example with GCC and
--optarch='Intel:march=core-avx2;GCC:march=native'

@boegel boegel added this to the release after 4.7.3 milestone Jul 5, 2023
boegel
boegel requested changes Jul 5, 2023
View reviewed changes
easybuild/easyblocks/o/opencv.py Outdated
if (
not optarch
or (get_software_root('GCC') and (optarch == 'march=native' or optarch.get('GCC') == 'march=native'))
or (get_software_root('intel-compilers') and (optarch == 'xHost' or optarch.get('Intel') == 'xHost'))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ideally we don't hardcode march=native and xHost here, since those values may change.

We should check whether we can extract the default from framework (and if not maybe we need to introduce a constant in framework so the default can be easily obtained without hardcoding)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And note that a manually set optarch may well be something more than march=native/xHost so a simple == doesn't suffice

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@boegel @akesandgren done in 7310de7

migueldiascosta
migueldiascosta reviewed Jul 18, 2023
View reviewed changes
easybuild/easyblocks/o/opencv.py Outdated
or (isinstance(optarch, str) and optimal_arch_option in optarch)
or (isinstance(optarch, dict) and (
(get_software_root('GCC') and optimal_arch_option in optarch.get('GCC', ''))
or (get_software_root('intel-compilers') and optimal_arch_option in optarch.get('Intel', ''))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for intel < 2021a we would need to look for iccifort instead of intel-compilers?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done in eb0ee64

@smoors
Copy link
Contributor Author

smoors commented Jul 19, 2023

Test report by @smoors

Overview of tested easyconfigs (in order)

  • SUCCESS OpenCV-4.6.0-foss-2022a-contrib.eb

Build succeeded for 1 out of 1 (1 easyconfigs in total)
node301.hydra.os - Linux CentOS Linux 7.9.2009, x86_64, Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz (skylake_avx512), Python 3.6.8
See https://gist.github.com/smoors/ac803c8f88a0203138902442ad5dd4e5 for a full test report.

@migueldiascosta
Copy link
Member

Test report by @migueldiascosta

Overview of tested easyconfigs (in order)

  • SUCCESS OpenCV-4.2.0-foss-2019b-Python-3.7.4.eb
  • SUCCESS OpenCV-4.2.0-foss-2020a-Python-3.8.2-contrib.eb
  • SUCCESS OpenCV-4.2.0-foss-2020a-Python-3.8.2.eb
  • SUCCESS OpenCV-4.5.1-foss-2020b-contrib.eb
  • SUCCESS OpenCV-4.5.3-foss-2021a-contrib.eb
  • SUCCESS OpenCV-4.5.5-foss-2021b-contrib.eb

Build succeeded for 6 out of 6 (6 easyconfigs in total)
sms - Linux CentOS Linux 7.6.1810, x86_64, AMD EPYC 7601 32-Core Processor, Python 3.6.8
See https://gist.github.com/migueldiascosta/a5f47dc26a901be29ee59c8e47f486a8 for a full test report.

@migueldiascosta
Copy link
Member

Test report by @migueldiascosta

Overview of tested easyconfigs (in order)

  • SUCCESS OpenCV-3.3.0-intel-2017b-Python-2.7.14.eb

Build succeeded for 1 out of 1 (1 easyconfigs in total)
sms - Linux CentOS Linux 7.6.1810, x86_64, AMD EPYC 7601 32-Core Processor, Python 3.6.8
See https://gist.github.com/migueldiascosta/32b52d01997d10a77188eb93c97f4f40 for a full test report.

@branfosj branfosj mentioned this pull request Jul 25, 2023
Merged
2 tasks
@boegel
Copy link
Member

boegel commented Aug 17, 2023

Test report by @boegel

Overview of tested easyconfigs (in order)

Build succeeded for 0 out of 1 (1 easyconfigs in total)
node3568.doduo.os - Linux RHEL 8.6, x86_64, AMD EPYC 7552 48-Core Processor (zen2), Python 3.6.8
See https://gist.github.com/boegel/fa275d7d035bd8b760e137e67d9f6bd1 for a full test report.

@boegel
Copy link
Member

boegel commented Aug 17, 2023

@smoors Hmm, I was hoping this would work with EasyBuild is configured with $EASYBUILD_OPTARCH set to "Intel:march=core-avx2", but it doesn't?

@smoors
Copy link
Contributor Author

smoors commented Aug 17, 2023

Test report by @smoors

Overview of tested easyconfigs (in order)

  • SUCCESS OpenCV-4.6.0-foss-2022a-contrib.eb

Build succeeded for 1 out of 1 (1 easyconfigs in total)
node364.hydra.os - Linux CentOS Linux 7.9.2009, x86_64, Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz (skylake_avx512), Python 3.6.8
See https://gist.github.com/smoors/5c6d35c3ef87428212dd77a4fdb72807 for a full test report.

@smoors
Copy link
Contributor Author

smoors commented Aug 17, 2023

@smoors Hmm, I was hoping this would work with EasyBuild is configured with $EASYBUILD_OPTARCH set to "Intel:march=core-avx2", but it doesn't?

that case was missing indeed, fixed in 25a66f0

@boegel
Copy link
Member

boegel commented Aug 30, 2023

Test report by @boegel

Overview of tested easyconfigs (in order)

  • SUCCESS OpenCV-4.6.0-foss-2022a-contrib.eb

Build succeeded for 1 out of 1 (1 easyconfigs in total)
node3537.doduo.os - Linux RHEL 8.6, x86_64, AMD EPYC 7552 48-Core Processor (zen2), Python 3.6.8
See https://gist.github.com/boegel/a4e0c77764dff04a5b6270657e38277f for a full test report.

@boegel
Copy link
Member

boegel commented Aug 30, 2023

Test report by @boegel

Overview of tested easyconfigs (in order)

  • SUCCESS OpenCV-3.4.1-intel-2018a-Python-3.6.4.eb

Build succeeded for 1 out of 1 (1 easyconfigs in total)
node3102.skitty.os - Linux CentOS Linux 7.9.2009, x86_64, Intel(R) Xeon(R) Gold 6140 CPU @ 2.30GHz (skylake_avx512), Python 2.7.5
See https://gist.github.com/boegel/3473dd29b8e7f07bd68a28f7211fc7d6 for a full test report.

@boegel boegel merged commit 6bcef45 into easybuilders:develop Aug 30, 2023
46 checks passed
@boegel boegel changed the title improve cpu-arch optimization settings for OpenCV use DCPU_BASELINE=DETECT for OpenCV when default optarch compiler option is used Sep 10, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@migueldiascosta migueldiascosta migueldiascosta left review comments

@akesandgren akesandgren akesandgren left review comments

@boegel boegel boegel approved these changes

Assignees
No one assigned
Labels
enhancement
Projects
None yet
Milestone
4.8.1
Development

Successfully merging this pull request may close these issues.
4 participants
@smoors @migueldiascosta @boegel @akesandgren