Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use wider-compatible LLVM-OpenMP, bundle libgfortran dylibs for macOS wheels #4092

Merged

Conversation

agriyakhetarpal
Copy link
Member

@agriyakhetarpal agriyakhetarpal commented May 15, 2024

Description

Closes #4091, binaries on macOS need to be built with OpenMP which is more compatible with older macOS versions, rather than Homebrew OpenMP, which is not built for use cases where one needs to set MACOSX_DEPLOYMENT_TARGET to a lower value than the version specifier of the host macOS machine, e.g., at the time of building wheels with cibuildwheel.

This change is required due to stricter requirements across macOS version compatibility when repairing the wheel with tools like delocate.

Type of change

Please add a line in the relevant section of CHANGELOG.md to document the change (include PR #) - note reverse order of PR #s. If necessary, also add to the list of breaking changes.

  • Bug fix (non-breaking change which fixes an issue)

Key checklist:

  • No style issues: $ pre-commit run (or $ nox -s pre-commit) (see CONTRIBUTING.md for how to set this up to run automatically when committing locally, in just two lines of code)
  • All tests pass: $ python run-tests.py --all (or $ nox -s tests)
  • The documentation builds: $ python run-tests.py --doctest (or $ nox -s doctests)

You can run integration tests, unit tests, and doctests together at once, using $ python run-tests.py --quick (or $ nox -s quick).

Further checks:

  • Code is commented, particularly in hard-to-understand areas
  • Tests added that prove fix is effective or that feature works

@agriyakhetarpal
Copy link
Member Author

Looks like matthew-brett/delocate#211 and matthew-brett/delocate#212 are some related threads.

https://github.com/agriyakhetarpal/PyBaMM/actions/runs/9096310836/job/25001433738 has some progress because the SuiteSparse and SUNDIALS dylibs are now exported correctly, but the build is still failing because we use gfortran and OpenMP headers from Homebrew, and Homebrew binaries are not necessarily good to publish Python wheels with. It is not recommended to downgrade the macOS runner version, since they shall be phased out soon... I'll look further into how we can replace these dependencies with other alternatives.

@agriyakhetarpal agriyakhetarpal marked this pull request as draft May 15, 2024 13:22
Copy link

codecov bot commented May 15, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 99.54%. Comparing base (33d9ea7) to head (11015cb).

Current head 11015cb differs from pull request most recent head 30a76b9

Please upload reports for the commit 30a76b9 to get more accurate results.

Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #4092      +/-   ##
===========================================
- Coverage    99.56%   99.54%   -0.02%     
===========================================
  Files          287      287              
  Lines        21744    21744              
===========================================
- Hits         21649    21646       -3     
- Misses          95       98       +3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@agriyakhetarpal
Copy link
Member Author

Another solution is to set them to 12.0 and 14.0 as requested – however, this would break compatibility for a big proportion of users

@agriyakhetarpal
Copy link
Member Author

Yet another option: set up C and C++ compilers with the LLVM-OpenMP variant from https://scikit-learn.org/dev/developers/advanced_installation.html#macos-compilers-from-conda-forge rather than using Homebrew

Point of breakage is pypa/cibuildwheel#1766, but this was just a documentation update, so our builds might have been incompatible silently: matthew-brett/delocate#198 is where all dylibs are being checked for compatibility, and matthew-brett/delocate#214 reveals that this was previously undetected across projects, so in any case we will need to migrate to using gcc which provides libgomp headers or use a different installation of libomp (LLVM variant) that has greater compatibility across macOS versions.

@agriyakhetarpal
Copy link
Member Author

See also: Homebrew/legacy-homebrew#48437

@agriyakhetarpal agriyakhetarpal changed the title Set MACOSX_DEPLOYMENT_TARGET when building SuiteSparse and SUNDIALS Use wider-compatible LLVM-OpenMP when building SuiteSparse and SUNDIALS May 15, 2024
@agriyakhetarpal
Copy link
Member Author

agriyakhetarpal commented May 15, 2024

I was able to handle OpenMP, the other remaining task is to handle gfortran dynamic libraries. We are still getting them from Homebrew, so they have to either be replaced with a different installation. Using gfortran and gcc as compilers doesn't bring issues, but we have a tricky dependency from suitesparseconfig.dylib on libgfortran.dylib which is why the latter gets bundled into our wheels, and therefore breaks compatibility... A way out could be to load at runtime instead while excluding it from the wheels, but this should be used as a last resort. The macos-11 images have been deprecated and will be removed in Q2 2024 (this quarter), so we can't use those, and we will need to migrate from macos-12 to macos-13 sometime soon.

@agriyakhetarpal
Copy link
Member Author

agriyakhetarpal commented May 16, 2024

Almost all the way, here: https://github.com/agriyakhetarpal/PyBaMM/actions/runs/9116260227/job/25064471991

CasADi wheels are not repaired properly and it has a higher macOS minimum version requirement than expected – libcasadi.3.7.dylib requires 11.1, so I have opened an issue on their repository: casadi/casadi#3698

I'll simply rename the wheels to get the builds passing, since we cannot do much about it at this time. We're going to lose out on compatibility on macOS versions prior to 11.1 silently – let's hope we don't receive too many bug reports about it.

@agriyakhetarpal agriyakhetarpal changed the title Use wider-compatible LLVM-OpenMP when building SuiteSparse and SUNDIALS Use wider-compatible LLVM-OpenMP, bundle lilbgfortran dylibs for macOS wheels May 16, 2024
@agriyakhetarpal agriyakhetarpal changed the title Use wider-compatible LLVM-OpenMP, bundle lilbgfortran dylibs for macOS wheels Use wider-compatible LLVM-OpenMP, bundle libgfortran dylibs for macOS wheels May 16, 2024
@agriyakhetarpal
Copy link
Member Author

Wheel builds are finally passing (🎉): https://github.com/agriyakhetarpal/PyBaMM/actions/runs/9117439816

I can open this up for review now, cc: @kratman and @martinjrobins. The Codacy failures are irrelevant for the most part since we need to call a few subprocesses, I don't seem to remember how one can ignore those.

@agriyakhetarpal agriyakhetarpal marked this pull request as ready for review May 16, 2024 18:46
.github/workflows/publish_pypi.yml Outdated Show resolved Hide resolved
.github/workflows/publish_pypi.yml Show resolved Hide resolved
.github/workflows/publish_pypi.yml Show resolved Hide resolved
Copy link
Member

@arjxn-py arjxn-py left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@agriyakhetarpal, is there anything still left in this one? Looks pretty good to me.
Let know if you still plan to do something in this PR, happy to approve and merge if not.

@agriyakhetarpal
Copy link
Member Author

I don't think there should be anything else. I checked the --force and --install-dir posargs combinations for pybamm-requires as well locally – everything seems to install in the right place and work as desired. Until XCode's Clang adds support for the OpenMP specification someday in a later macOS version, this piece of code can stay for now. I would note that the same bundling process for libgfortran is present for the NumPy wheels as well, besides SciPy.

@arjxn-py, you can go ahead with your approval and merge this. If we receive any complaints from macOS arm64 users about the IDAKLU solver, we'll just ask them to upgrade from macOS X 1A.B to 11.1 or later, that should fix their issues (because the CasADi issue above will probably remain open for a while).

cc: @BradyPlanden – this fixes the issue that was being discussed at the PyBOP 16/05/2024 developer meeting :)

@agriyakhetarpal
Copy link
Member Author

Also, cc: @cringeyburger @prady0t this PR, while this should be okay to merge for now, is one of the reasons why we should be working towards a mechanism for testing the built wheels this year ;) currently we just check if the IDAKLU solver can be imported correctly, but it can hide issues related to the solver at runtime—see the older issue #3783 for example—running the full test suite against the wheels (or a fraction of it) would be great

Copy link
Member

@arjxn-py arjxn-py left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @agriyakhetarpal 🎉

@arjxn-py arjxn-py merged commit becb1a4 into pybamm-team:develop May 18, 2024
39 of 40 checks passed
@agriyakhetarpal agriyakhetarpal deleted the fix-macosx-deployment-target branch May 18, 2024 02:51
agriyakhetarpal added a commit to agriyakhetarpal/PyBaMM that referenced this pull request May 24, 2024
kratman pushed a commit that referenced this pull request May 24, 2024
js1tr3 pushed a commit to js1tr3/PyBaMM that referenced this pull request Aug 12, 2024
…OS wheels (pybamm-team#4092)

* Set `MACOSX_DEPLOYMENT_TARGET` for prerequisites

* Remove previously set environment variables

* Remove Homebrew libomp installations

* Ignore `pybamm-requires` OpenMP downloads

* Rework docs to add OpenMP instructions

* Download LLVM-OpenMP on macOS

* Remove unused compiler argument

* Fix rpath for libomp dylib at wheel repair

* Fix a failing link

* Revert "Fix rpath for libomp dylib at wheel repair"

This reverts commit 6fb8ef0.

* Fix RPATHs for SuiteSparse and libomp

* 11.1 for arm64 macOS and 11.0 for amd64 macOS

* Set up Fortran compiler via GHA

* Downgrade to gcc 12

* Try fix for macOS amd64 wheels first

* Build on macos-13 image instead

* Debug further, try building against 11.0

* And now build for macOS arm64

* Try building arm64 against 11.0

* Rename wheel from 11_1 to 11_0

* Cleanup and fix up commands

* Rename wheel in dest dir instead

* Cleanup plus references to SciPy license
js1tr3 pushed a commit to js1tr3/PyBaMM that referenced this pull request Aug 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Fortnightly build for wheels failed / PyBaMM import kills process on macOS M-series
2 participants