Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extend pybamm_install_odes to include support for macOS systems #3417

Merged
merged 48 commits into from
Jan 6, 2024

Conversation

arjxn-py
Copy link
Member

@arjxn-py arjxn-py commented Oct 6, 2023

Description

Fixes #3404

Type of change

Please add a line in the relevant section of CHANGELOG.md to document the change (include PR #) - note reverse order of PR #s. If necessary, also add to the list of breaking changes.

  • New feature (non-breaking change which adds functionality)
  • Optimization (back-end change that speeds up the code)
  • Bug fix (non-breaking change which fixes an issue)

Key checklist:

  • No style issues: $ pre-commit run (or $ nox -s pre-commit) (see CONTRIBUTING.md for how to set this up to run automatically when committing locally, in just two lines of code)
  • All tests pass: $ python run-tests.py --all (or $ nox -s tests)
  • The documentation builds: $ python run-tests.py --doctest (or $ nox -s doctests)

You can run integration tests, unit tests, and doctests together at once, using $ python run-tests.py --quick (or $ nox -s quick).

Further checks:

  • Code is commented, particularly in hard-to-understand areas
  • Tests added that prove fix is effective or that feature works

@arjxn-py arjxn-py marked this pull request as draft October 6, 2023 16:35
@codecov
Copy link

codecov bot commented Oct 6, 2023

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (ee64eaf) 99.59% compared to head (738cd57) 99.59%.
Report is 26 commits behind head on develop.

Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #3417      +/-   ##
===========================================
- Coverage    99.59%   99.59%   -0.01%     
===========================================
  Files          258      258              
  Lines        20797    20796       -1     
===========================================
- Hits         20712    20711       -1     
  Misses          85       85              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@agriyakhetarpal
Copy link
Member

Seems like we were testing pybamm_install_odes earlier (#2973) but that was back when we were using tox, do you remember why we removed it? I can't seem to find it anywhere. Maybe it was because we decided not to have a nox equivalent of tox -e odes or we just forgot to add one

@arjxn-py
Copy link
Member Author

arjxn-py commented Oct 13, 2023

Seems like we were testing pybamm_install_odes earlier (#2973) but that was back when we were using tox, do you remember why we removed it? I can't seem to find it anywhere. Maybe it was because we decided not to have a nox equivalent of tox -e odes or we just forgot to add one

I don't remember exactly but it might be because of we wanted to cut time in CI or just might have forgotten to have a session for odes.

Edit: I can make the same nox session again to test them.

@arjxn-py arjxn-py force-pushed the install_odes branch 2 times, most recently from 5e8d33e to 7018c19 Compare October 14, 2023 21:00
.github/workflows/test_on_push.yml Outdated Show resolved Hide resolved
.github/workflows/test_on_push.yml Outdated Show resolved Hide resolved
.github/workflows/test_on_push.yml Outdated Show resolved Hide resolved
.github/workflows/test_on_push.yml Outdated Show resolved Hide resolved
.github/workflows/test_on_push.yml Outdated Show resolved Hide resolved
arjxn-py and others added 2 commits October 15, 2023 16:07
Co-authored-by: Agriya Khetarpal <74401230+agriyakhetarpal@users.noreply.github.com>
.github/workflows/test_on_push.yml Outdated Show resolved Hide resolved
.github/workflows/test_on_push.yml Outdated Show resolved Hide resolved
.github/workflows/test_on_push.yml Outdated Show resolved Hide resolved
.github/workflows/test_on_push.yml Outdated Show resolved Hide resolved
Co-authored-by: Agriya Khetarpal <74401230+agriyakhetarpal@users.noreply.github.com>
@arjxn-py
Copy link
Member Author

Thanks @agriyakhetarpal, waiting on tests to pass the initial steps and then fail while installing odes to observe logs.

@agriyakhetarpal
Copy link
Member

agriyakhetarpal commented Oct 15, 2023

It looks like it worked without any additional configuration (just like it did for me locally). Is there an error in specific that we should reproduce to see if it is robust enough? Edit: similar to #2973, we can try installing an older version of SUNDIALS and check if it fails on macOS and GNU/Linux

We could refactor the script a bit and add some additional configuration (since this entry point is for user installations, it would be wise to add .zshrc alongside .bashrc in the update_LD_LIBRARY_PATH function because it is enabled by default on newer macOS systems). Some other stuff I identified that we can do:

  1. We ask users to run the script and that it works out of the box, but it would fail on not finding wget and cmake. We could install both of them with pip inside a subprocess or instead throw more helpful errors
  2. Use f-strings in the script since they are more efficient
  3. scikits_odes_setup.log is not git-ignored currently
  4. add a nox session that installs cmake and wget and wraps pybamm_install_odes (actually, this might not be needed because users don't get or need nox installed as a dependency unless they install the dev dependencies). Developers can already install SUNDIALS but with KLU enabled using pybamm-requires.
  5. Update the documentation about macOS support for this entry point in the optional solvers section and other areas (and maybe make the script exit early with a helpful error if on a Windows system because we haven't done so yet). We should update the CHANGELOG about this too since this is rather a user-facing change.

@agriyakhetarpal
Copy link
Member

We should also run pip cache purge in the installation step so that the runners don't end up using the cached wheels between runs and/or remove the cache configuration from the setup-python step

@agriyakhetarpal
Copy link
Member

I will test the changes locally in a moment after removing my local installation of SUNDIALS

Copy link
Member

@agriyakhetarpal agriyakhetarpal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested locally without a pip-cached scikits.odes wheel and without SUNDIALS installed via Homebrew nor present in ~/.local/ and it works, thanks, @arjxn-py! I left a few comments, feel free to address.

@Saransh-cpp, I remember you were having problems with getting scikits.odes to work on your machine before, could you test this and see if it works?

.github/workflows/run_periodic_tests.yml Outdated Show resolved Hide resolved
docs/source/user_guide/installation/GNU-linux.rst Outdated Show resolved Hide resolved
docs/source/user_guide/installation/GNU-linux.rst Outdated Show resolved Hide resolved
docs/source/user_guide/installation/GNU-linux.rst Outdated Show resolved Hide resolved
pybamm/install_odes.py Show resolved Hide resolved
@agriyakhetarpal
Copy link
Member

Feel free to add some of the un-fixable broken links to the .lycheeignore file

@arjxn-py
Copy link
Member Author

arjxn-py commented Dec 23, 2023

Feel free to add some of the un-fixable broken links to the .lycheeignore file

Most of them are due to file rename which I hope should be fixed except this one, let me know how should i go with this.

@agriyakhetarpal
Copy link
Member

agriyakhetarpal commented Dec 23, 2023

Most of them are due to file rename which I hope should be fixed except this one, let me know how should i go with this.

It can be added to .lycheeignore as well, or we can ignore files or entire folders using glob patterns in the workflow file, let me know if neither of these options work. Some of the links such as those coming from the rename to gnu-linux-mac will get fixed upon merging.

@arjxn-py
Copy link
Member Author

Added the non-fixable one in .lycheeignore, hopefully others would be fixed upon merging.

Copy link
Member

@agriyakhetarpal agriyakhetarpal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I realised I missed reading through the workflow so I have a few comments. Could you please run it on your fork once and add a link to the logs here?

I think we can leave wget as-is for this PR. We can look into removing it in #3651 together after this gets merged. Could you resolve the conflicts here?

docs/source/user_guide/installation/gnu-linux-mac.rst Outdated Show resolved Hide resolved
.github/workflows/run_periodic_tests.yml Outdated Show resolved Hide resolved
.github/workflows/run_periodic_tests.yml Show resolved Hide resolved
.github/workflows/run_periodic_tests.yml Show resolved Hide resolved
.github/workflows/run_periodic_tests.yml Outdated Show resolved Hide resolved
@arjxn-py arjxn-py force-pushed the install_odes branch 2 times, most recently from 59d4401 to 96d63de Compare December 24, 2023 15:24
arjxn-py and others added 3 commits December 24, 2023 20:55
Co-authored-by: Agriya Khetarpal <74401230+agriyakhetarpal@users.noreply.github.com>
@arjxn-py
Copy link
Member Author

Could you please run it on your fork once and add a link to the logs here?

Here's the link to the workflow run on the respective branch, waiting for tests to pass.

@arjxn-py
Copy link
Member Author

Those failing example tests on linux are unrelated, right?

@agriyakhetarpal
Copy link
Member

Yes, opening a new issue about them

Copy link
Member

@Saransh-cpp Saransh-cpp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, thanks for picking this up, @arjxn-py!

.github/workflows/run_periodic_tests.yml Outdated Show resolved Hide resolved
Co-authored-by: Saransh Chopra <saransh0701@gmail.com>
@Saransh-cpp Saransh-cpp merged commit 0b87513 into pybamm-team:develop Jan 6, 2024
39 of 41 checks passed
@arjxn-py arjxn-py deleted the install_odes branch January 7, 2024 07:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Extend pybamm_install_odes to include support for macOS systems
3 participants