Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extend pybamm_install_odes to include support for macOS systems #3417

Merged
merged 48 commits into from
Jan 6, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
148ea18
Test `pybamm_install_odes` on macOS on CI
arjxn-py Oct 6, 2023
d9743ec
Add parallel job
arjxn-py Oct 7, 2023
7c7420a
Test before unit tests in mac
arjxn-py Oct 7, 2023
906683b
Install `wget` before odes
arjxn-py Oct 7, 2023
a36ec4f
Merge branch 'develop' into install_odes
arjxn-py Oct 11, 2023
dda612a
Add parallel job to test `install_odes`
arjxn-py Oct 13, 2023
af43996
Install `pathlib` as required
arjxn-py Oct 14, 2023
7018c19
Remove condition
arjxn-py Oct 14, 2023
7f0bea9
Apply suggestions from code review
arjxn-py Oct 15, 2023
7ee2a9a
Correctly indent key
arjxn-py Oct 15, 2023
9728d17
Apply suggestions from code review
arjxn-py Oct 15, 2023
559a703
Merge branch 'develop' into install_odes
arjxn-py Oct 20, 2023
2713ce5
Add `.zshrc` for macOS
arjxn-py Oct 20, 2023
8db4f7c
Install required modules before initializing
arjxn-py Oct 20, 2023
45de356
Using f-strings instead of `format()`
arjxn-py Oct 20, 2023
4a8f13c
gitignore `scikits_odes_setup.log`
arjxn-py Oct 20, 2023
edcccf5
Update doc in solver section for `install_odes`
arjxn-py Oct 20, 2023
9655550
Exit early on windows
arjxn-py Oct 20, 2023
6150207
Merge branch 'develop' into install_odes
arjxn-py Oct 20, 2023
8b33770
Changelog
arjxn-py Oct 20, 2023
ac803c5
Remove cache before installation
arjxn-py Oct 20, 2023
0ed80bb
Applied suggestions
arjxn-py Oct 23, 2023
2698a87
Move `test_install_odes` to scheduled
arjxn-py Oct 23, 2023
c062b17
Check platform without function
arjxn-py Oct 24, 2023
e3c62b5
Import module with importlib
arjxn-py Oct 24, 2023
79539f4
Define sundials version on top
arjxn-py Oct 24, 2023
6292cbf
Detect terminal with `os.environ`
arjxn-py Oct 24, 2023
b9668f7
Merge branch 'develop' into install_odes
arjxn-py Nov 5, 2023
53c358d
Merge branch 'develop' into install_odes
arjxn-py Nov 17, 2023
7464857
Resolve merge conflicts & merge to develop
arjxn-py Dec 16, 2023
777850c
Merge branch 'develop' into install_odes
arjxn-py Dec 16, 2023
60c6e02
Prevent separate function to install dependencies
arjxn-py Dec 17, 2023
c8266ed
Check for shell files directly
arjxn-py Dec 20, 2023
4979598
Merge branch 'develop' into install_odes
arjxn-py Dec 22, 2023
7e0cc70
Add note to avoid installation failure
arjxn-py Dec 22, 2023
15e0597
Add note for path validation
arjxn-py Dec 23, 2023
6221017
Apply suggestions from code review
arjxn-py Dec 23, 2023
b8eaaab
Rename file & suggested fixes
arjxn-py Dec 23, 2023
4e1dbec
Set `CMAKE_BUILD_PARALLEL_LEVEL`
arjxn-py Dec 23, 2023
e770c92
Fix broken doctree due to rename
arjxn-py Dec 23, 2023
34d3e6b
Fix title underline
arjxn-py Dec 23, 2023
e766cca
Fix table malformation
arjxn-py Dec 23, 2023
96d63de
Add non-fixable link to `.lycheeignore`
arjxn-py Dec 23, 2023
dddea9c
Merge branch 'develop' into install_odes
arjxn-py Dec 24, 2023
430c86f
style: pre-commit fixes
pre-commit-ci[bot] Dec 24, 2023
6aa6685
Apply suggestions from code review
arjxn-py Dec 24, 2023
ed3f741
Merge branch 'pybamm-team:develop' into install_odes
agriyakhetarpal Dec 27, 2023
738cd57
Use `python -m pip` invocation instead
agriyakhetarpal Jan 5, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 47 additions & 0 deletions .github/workflows/run_periodic_tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -153,3 +153,50 @@ jobs:
eval "$(pyenv init -)"
pyenv activate pybamm-${{ matrix.python-version }}
pyenv uninstall -f $( python --version )

test_install_odes:
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ubuntu-latest, macos-latest]
python-version: ["3.8", "3.9", "3.10", "3.11"]
fail-fast: false
name: Test pybamm_install_odes on ${{ matrix.os }}

steps:
- name: Check out PyBaMM repository
uses: actions/checkout@v4

arjxn-py marked this conversation as resolved.
Show resolved Hide resolved
- name: Install Linux system dependencies
if: matrix.os == 'ubuntu-latest'
run: |
sudo apt-get update
sudo apt-get install gfortran gcc libopenblas-dev
- name: Install macOS system dependencies
arjxn-py marked this conversation as resolved.
Show resolved Hide resolved
if: matrix.os == 'macos-latest'
env:
# Homebrew environment variables
HOMEBREW_NO_INSTALL_CLEANUP: 1
HOMEBREW_NO_AUTO_UPDATE: 1
HOMEBREW_NO_COLOR: 1
# Speed up CI
NONINTERACTIVE: 1
run: |
brew analytics off
brew install openblas
brew reinstall gcc gfortran

- name: Set up Python ${{ matrix.python-version }}
id: setup-python
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}

- name: Install PyBaMM
run: python -m pip install -e .

- name: Test pybamm_install_odes on ${{ matrix.os }}
run: |
python -m pip cache purge
python -m pip install wget cmake
pybamm_install_odes
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,9 @@ KLU_module_deps
# setup
setup.log

# odes setup
scikits_odes_setup.log

# test
test.c
test.json
Expand Down
1 change: 1 addition & 0 deletions .lycheeignore
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# a list of links/files to be ignored by lychee link checker (see workflow file)
https://github.com/LLNL/sundials/releases/download/v%7BSUNDIALS_VERSION%7D/sundials-%7BSUNDIALS_VERSION%7D.tar.gz

# Errors in docs/source/user_guide/getting_started.md
file:///home/runner/work/PyBaMM/PyBaMM/docs/source/user_guide/api_docs
Expand Down
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

## Features

- The `pybamm_install_odes` command now includes support for macOS systems and can be used to set up SUNDIALS and install the `scikits.odes` solver on macOS ([#3417](https://github.com/pybamm-team/PyBaMM/pull/3417))
- Added support for Python 3.12 ([#3531](https://github.com/pybamm-team/PyBaMM/pull/3531))
- Added method to get QuickPlot axes by variable ([#3596](https://github.com/pybamm-team/PyBaMM/pull/3596))
- Added custom experiment terminations ([#3596](https://github.com/pybamm-team/PyBaMM/pull/3596))
Expand Down
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -104,7 +104,7 @@ PyBaMM makes releases every four months and we use [CalVer](https://calver.org/)

PyBaMM is available on GNU/Linux, MacOS and Windows.
We strongly recommend to install PyBaMM within a python virtual environment, in order not to alter any distribution python files.
For instructions on how to create a virtual environment for PyBaMM, see [the documentation](https://docs.pybamm.org/en/latest/source/user_guide/installation/GNU-linux.html#user-install).
For instructions on how to create a virtual environment for PyBaMM, see [the documentation](https://docs.pybamm.org/en/latest/source/user_guide/installation/gnu-linux-mac.html#user-install).

### Using pip

Expand All @@ -130,8 +130,8 @@ conda install -c conda-forge pybamm

Following GNU/Linux and macOS solvers are optionally available:

- [scikits.odes](https://scikits-odes.readthedocs.io/en/latest/)-based solver, see [the documentation](https://docs.pybamm.org/en/latest/source/user_guide/installation/GNU-linux.html#optional-scikits-odes-solver).
- [jax](https://jax.readthedocs.io/en/latest/notebooks/quickstart.html)-based solver, see [the documentation](https://docs.pybamm.org/en/latest/source/user_guide/installation/GNU-linux.html#optional-jaxsolver).
- [scikits.odes](https://scikits-odes.readthedocs.io/en/latest/)-based solver, see [the documentation](https://docs.pybamm.org/en/latest/source/user_guide/installation/gnu-linux-mac.html#optional-scikits-odes-solver).
- [jax](https://jax.readthedocs.io/en/latest/notebooks/quickstart.html)-based solver, see [the documentation](https://docs.pybamm.org/en/latest/source/user_guide/installation/gnu-linux-mac.html#optional-jaxsolver).

## 📖 Citing PyBaMM

Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
GNU-Linux & MacOS
GNU/Linux & macOS
=================

.. contents::
Expand Down Expand Up @@ -102,17 +102,15 @@ For an introduction to virtual environments, see
Optional - scikits.odes solver
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Users can install `scikits.odes <https://github.com/bmcage/odes>`__ in
order to use the wrapped SUNDIALS ODE and DAE
`solvers <https://docs.pybamm.org/en/latest/source/api/solvers/scikits_solvers.html>`__.
Users can install `scikits.odes <https://github.com/bmcage/odes>`__ to utilize its interfaced SUNDIALS ODE and DAE `solvers <https://docs.pybamm.org/en/latest/source/api/solvers/scikits_solvers.html>`__ wrapped in PyBaMM.

.. note::

Currently, only GNU/Linux and macOS are supported.

.. note::

The ``scikits.odes`` solver is not supported on Python 3.12 yet, please refer to https://github.com/bmcage/odes/issues/162.
The ``scikits.odes`` solver is not supported on Python 3.12 yet. Please refer to https://github.com/bmcage/odes/issues/162.
There is support for Python 3.8, 3.9, 3.10, and 3.11.

.. tab:: GNU/Linux
Expand All @@ -121,10 +119,9 @@ order to use the wrapped SUNDIALS ODE and DAE

.. code:: bash

apt install libopenblas-dev
pybamm_install_odes
apt install libopenblas-dev
pybamm_install_odes

The ``pybamm_install_odes`` command is installed with PyBaMM. It automatically downloads and installs the SUNDIALS library on your
system (under ``~/.local``), before installing ``scikits.odes``. (Alternatively, one can install SUNDIALS without this script and run ``pip install pybamm[odes]`` to install ``pybamm`` with ``scikits.odes``.)

.. tab:: macOS
Expand All @@ -133,9 +130,29 @@ order to use the wrapped SUNDIALS ODE and DAE

.. code:: bash

pip install scikits.odes
brew install openblas gcc gfortran
pybamm_install_odes

Assuming that SUNDIALS was installed as described :ref:`above<user-install-label>`.
The ``pybamm_install_odes`` command, installed with PyBaMM, automatically downloads and installs the SUNDIALS library on your
system (under ``~/.local``), before installing `scikits.odes <https://scikits-odes.readthedocs.io/en/stable/installation.html>`__ . (Alternatively, one can install SUNDIALS without this script and run ``pip install pybamm[odes]`` to install ``pybamm`` with `scikits.odes <https://scikits-odes.readthedocs.io/en/stable/installation.html>`__)

To avoid installation failures when using ``pip install pybamm[odes]``, make sure to set the ``SUNDIALS_INST`` environment variable. If you have installed SUNDIALS using Homebrew, set the variable to the appropriate location. For example:

.. code:: bash

export SUNDIALS_INST=$(brew --prefix sundials)

Ensure that the path matches the installation location on your system. You can verify the installation location by running:

.. code:: bash

brew info sundials

Look for the installation path, and use that path to set the ``SUNDIALS_INST`` variable.

Note: The location where Homebrew installs SUNDIALS might vary based on the system architecture (ARM or Intel). Adjust the path in the ``export SUNDIALS_INST`` command accordingly.

To avoid manual setup of path the ``pybamm_install_odes`` is recommended for a smoother installation process, as it takes care of automatically downloading and installing the SUNDIALS library on your system.

Optional - JaxSolver
~~~~~~~~~~~~~~~~~~~~
Expand Down
18 changes: 9 additions & 9 deletions docs/source/user_guide/installation/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -47,8 +47,8 @@ Optional solvers

Following GNU/Linux and macOS solvers are optionally available:

* `scikits.odes <https://scikits-odes.readthedocs.io/en/latest/>`_ -based solver, see `Optional - scikits.odes solver <https://docs.pybamm.org/en/latest/source/user_guide/installation/GNU-linux.html#optional-scikits-odes-solver>`_.
* `jax <https://jax.readthedocs.io/en/latest/notebooks/quickstart.html>`_ -based solver, see `Optional - JaxSolver <https://docs.pybamm.org/en/latest/source/user_guide/installation/GNU-linux.html#optional-jaxsolver>`_.
* `scikits.odes <https://scikits-odes.readthedocs.io/en/latest/>`_ -based solver, see `Optional - scikits.odes solver <https://docs.pybamm.org/en/latest/source/user_guide/installation/gnu-linux-mac.html#optional-scikits-odes-solver>`_.
* `jax <https://jax.readthedocs.io/en/latest/notebooks/quickstart.html>`_ -based solver, see `Optional - JaxSolver <https://docs.pybamm.org/en/latest/source/user_guide/installation/gnu-linux-mac.html#optional-jaxsolver>`_.

Dependencies
------------
Expand Down Expand Up @@ -233,15 +233,15 @@ odes dependencies

Installable with ``pip install "pybamm[odes]"``

================================================================================================================================ ================== ================== =============================
Dependency Minimum Version pip extra Notes
================================================================================================================================ ================== ================== =============================
`scikits.odes <https://docs.pybamm.org/en/latest/source/user_guide/installation/GNU-linux.html#optional-scikits-odes-solver>`__ \- odes For scikits ODE & DAE solvers
================================================================================================================================ ================== ================== =============================
======================================================================================================================================= ================== ================== =============================
Dependency Minimum Version pip extra Notes
======================================================================================================================================= ================== ================== =============================
`scikits.odes <https://docs.pybamm.org/en/latest/source/user_guide/installation/gnu-linux-mac.html#optional-scikits-odes-solver>`__ \- odes For scikits ODE & DAE solvers
======================================================================================================================================= ================== ================== =============================

.. note::

Before running ``pip install "pybamm[odes]"``, make sure to install ``scikits.odes`` build-time requirements as described `here <https://docs.pybamm.org/en/latest/source/user_guide/installation/GNU-linux.html#optional-scikits-odes-solver>`_ .
Before running ``pip install "pybamm[odes]"``, make sure to install ``scikits.odes`` build-time requirements as described `here <https://docs.pybamm.org/en/latest/source/user_guide/installation/gnu-linux-mac.html#optional-scikits-odes-solver>`_ .

Full installation guide
-----------------------
Expand All @@ -251,7 +251,7 @@ Installing a specific version? Installing from source? Check the advanced instal
.. toctree::
:maxdepth: 1

GNU-linux
gnu-linux-mac
windows
windows-wsl
install-from-source
Expand Down
2 changes: 1 addition & 1 deletion docs/source/user_guide/installation/windows-wsl.rst
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ Get PyBaMM's Source Code
5. Follow the Installation Steps
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Follow the `installation instructions for PyBaMM on Linux <GNU-linux.html>`__.
Follow the `installation instructions for PyBaMM on Linux <gnu-linux-mac.html>`__.

Using Visual Studio Code with the WSL
---------------------------------------
Expand Down
4 changes: 2 additions & 2 deletions pybamm/expression_tree/operations/evaluate_python.py
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ class JaxCooMatrix:
def __init__(self, row, col, data, shape):
if not pybamm.have_jax(): # pragma: no cover
raise ModuleNotFoundError(
"Jax or jaxlib is not installed, please see https://docs.pybamm.org/en/latest/source/user_guide/installation/GNU-linux.html#optional-jaxsolver"
"Jax or jaxlib is not installed, please see https://docs.pybamm.org/en/latest/source/user_guide/installation/gnu-linux-mac.html#optional-jaxsolver"
)

self.row = jax.numpy.array(row)
Expand Down Expand Up @@ -523,7 +523,7 @@ class EvaluatorJax:
def __init__(self, symbol):
if not pybamm.have_jax(): # pragma: no cover
raise ModuleNotFoundError(
"Jax or jaxlib is not installed, please see https://docs.pybamm.org/en/latest/source/user_guide/installation/GNU-linux.html#optional-jaxsolver"
"Jax or jaxlib is not installed, please see https://docs.pybamm.org/en/latest/source/user_guide/installation/gnu-linux-mac.html#optional-jaxsolver"
)

constants, python_str = pybamm.to_python(symbol, debug=False, output_jax=True)
Expand Down
50 changes: 35 additions & 15 deletions pybamm/install_odes.py
arjxn-py marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,18 @@
import sys
import logging
import subprocess
from multiprocessing import cpu_count

from pybamm.util import root_dir

if sys.platform == "win32":
raise Exception("pybamm_install_odes is not supported on Windows.")

SUNDIALS_VERSION = "6.5.0"

# Build in parallel wherever possible
os.environ["CMAKE_BUILD_PARALLEL_LEVEL"] = str(cpu_count())

try:
# wget module is required to download SUNDIALS or SuiteSparse.
import wget
Expand All @@ -33,19 +42,13 @@ def download_extract_library(url, directory):
def install_sundials(download_dir, install_dir):
# Download the SUNDIALS library and compile it.
logger = logging.getLogger("scikits.odes setup")
sundials_version = "6.5.0"

try:
subprocess.run(["cmake", "--version"])
except OSError:
raise RuntimeError("CMake must be installed to build SUNDIALS.")

url = (
"https://github.com/LLNL/"
+ "sundials/releases/download/v{}/sundials-{}.tar.gz".format(
sundials_version, sundials_version
)
)
url = f"https://github.com/LLNL/sundials/releases/download/v{SUNDIALS_VERSION}/sundials-{SUNDIALS_VERSION}.tar.gz"
logger.info("Downloading sundials")
download_extract_library(url, download_dir)

Expand All @@ -54,7 +57,7 @@ def install_sundials(download_dir, install_dir):
"-DSUNDIALS_INDEX_SIZE=32",
"-DBUILD_ARKODE:BOOL=OFF",
"-DEXAMPLES_ENABLE:BOOL=OFF",
"-DCMAKE_INSTALL_PREFIX=" + install_dir,
f"-DCMAKE_INSTALL_PREFIX={install_dir}",
]

# SUNDIALS are built within directory 'build_sundials' in the PyBaMM root
Expand All @@ -66,7 +69,7 @@ def install_sundials(download_dir, install_dir):

print("-" * 10, "Running CMake prepare", "-" * 40)
subprocess.run(
["cmake", f"../sundials-{sundials_version}", *cmake_args],
["cmake", f"../sundials-{SUNDIALS_VERSION}", *cmake_args],
cwd=build_directory,
check=True,
)
Expand All @@ -77,23 +80,40 @@ def install_sundials(download_dir, install_dir):


def update_LD_LIBRARY_PATH(install_dir):
# Look for current python virtual env and add export statement
# for LD_LIBRARY_PATH in activate script. If no virtual env found,
# then the current user's .bashrc file is modified instead.
# Look for the current python virtual env and add an export statement
# for LD_LIBRARY_PATH in the activate script. If no virtual env is found,
# the current user's .bashrc file is modified instead.

export_statement = f"export LD_LIBRARY_PATH={install_dir}/lib:$LD_LIBRARY_PATH"

home_dir = os.environ.get("HOME")
bashrc_path = os.path.join(home_dir, ".bashrc")
zshrc_path = os.path.join(home_dir, ".zshrc")
venv_path = os.environ.get("VIRTUAL_ENV")

if venv_path:
script_path = os.path.join(venv_path, "bin/activate")
else:
script_path = os.path.join(os.environ.get("HOME"), ".bashrc")
if os.path.exists(bashrc_path):
script_path = os.path.join(os.environ.get("HOME"), ".bashrc")
elif os.path.exists(zshrc_path):
script_path = os.path.join(os.environ.get("HOME"), ".zshrc")
elif os.path.exists(bashrc_path) and os.path.exists(zshrc_path):
print(
"Both .bashrc and .zshrc found in the home directory. Setting .bashrc as path"
)
script_path = os.path.join(os.environ.get("HOME"), ".bashrc")
else:
print("Neither .bashrc nor .zshrc found in the home directory.")

if os.getenv("LD_LIBRARY_PATH") and f"{install_dir}/lib" in os.getenv(
"LD_LIBRARY_PATH"
):
print(f"{install_dir}/lib was found in LD_LIBRARY_PATH.")
print("--> Not updating venv activate or .bashrc scripts")
if os.path.exists(bashrc_path):
print("--> Not updating venv activate or .bashrc scripts")
if os.path.exists(zshrc_path):
print("--> Not updating venv activate or .zshrc scripts")
else:
with open(script_path, "a+") as fh:
# Just check that export statement is not already there.
Expand Down Expand Up @@ -136,7 +156,7 @@ def main(arguments=None):
else os.path.join(pybamm_dir, args.install_dir)
)

# Check is sundials is already installed
# Check if sundials is already installed
SUNDIALS_LIB_DIRS = [join(os.getenv("HOME"), ".local"), "/usr/local", "/usr"]

if args.sundials_libs:
Expand Down
2 changes: 1 addition & 1 deletion pybamm/solvers/jax_bdf_solver.py
Original file line number Diff line number Diff line change
Expand Up @@ -1009,7 +1009,7 @@ def jax_bdf_integrate(func, y0, t_eval, *args, rtol=1e-6, atol=1e-6, mass=None):
"""
if not pybamm.have_jax():
raise ModuleNotFoundError(
"Jax or jaxlib is not installed, please see https://docs.pybamm.org/en/latest/source/user_guide/installation/GNU-linux.html#optional-jaxsolver"
"Jax or jaxlib is not installed, please see https://docs.pybamm.org/en/latest/source/user_guide/installation/gnu-linux-mac.html#optional-jaxsolver"
)

def _check_arg(arg):
Expand Down
2 changes: 1 addition & 1 deletion pybamm/solvers/jax_solver.py
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ def __init__(
):
if not pybamm.have_jax():
raise ModuleNotFoundError(
"Jax or jaxlib is not installed, please see https://docs.pybamm.org/en/latest/source/user_guide/installation/GNU-linux.html#optional-jaxsolver"
"Jax or jaxlib is not installed, please see https://docs.pybamm.org/en/latest/source/user_guide/installation/gnu-linux-mac.html#optional-jaxsolver"
)

# note: bdf solver itself calculates consistent initial conditions so can set
Expand Down
Loading