Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Resolving environment file conda selector syntax #1603

Merged
merged 11 commits into from
Jul 22, 2024
14 changes: 7 additions & 7 deletions .github/actions/install-parcels/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,11 @@ name: Setup Conda and install parcels
description: >
In-repo composite action to setup Conda and install parcels. Installation of parcels relies on
`setup.py` file being available in the root. For general setup of Anaconda environments, just use
the `mamba-org/setup-micromamba@v1` action (setting C variables as required), or the `conda-incubator/setup-miniconda` action.
the `conda-incubator/setup-miniconda` action (setting C variables as required).
inputs:
environment-file:
description: Conda environment file to use.
default: environment.yml
environment-name:
description: Name to use for the Conda environment
default: test
runs:
using: "composite"
steps:
Expand All @@ -18,14 +15,17 @@ runs:
uses: al-cheb/configure-pagefile-action@v1.3
with:
minimum-size: 8GB
- name: Install micromamba (${{ inputs.environment-file }})
uses: mamba-org/setup-micromamba@v1
- name: Install miniconda (${{ inputs.environment-file }})
uses: conda-incubator/setup-miniconda@v3
VeckoTheGecko marked this conversation as resolved.
Show resolved Hide resolved
with:
environment-file: ${{ inputs.environment-file }}
environment-name: ${{ inputs.environment-name }}
channels: conda-forge
cache-environment: true
cache-downloads: true
- name: MPI support
if: ${{ ! (runner.os == 'Windows') }}
run: conda install -c conda-forge mpich mpi4py
shell: bash -el {0}
- name: Install parcels
run: pip install .
shell: bash -el {0}
1 change: 0 additions & 1 deletion .github/workflows/integration-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,6 @@ jobs:
uses: ./.github/actions/install-parcels
with:
environment-file: environment.yml
environment-name: py3_parcels
- name: Integration test
run: |
coverage run -m pytest -v -s --nbval-lax -k "not documentation" --html="${{ matrix.os }}_integration_test_report.html" --self-contained-html docs/examples
Expand Down
1 change: 0 additions & 1 deletion .github/workflows/unit-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,6 @@ jobs:
uses: ./.github/actions/install-parcels
with:
environment-file: environment.yml
environment-name: py3_parcels
- name: Unit test
run: |
coverage run -m pytest -v -s --html=${{ matrix.os }}_unit_test_report.html --self-contained-html tests
Expand Down
2 changes: 1 addition & 1 deletion .readthedocs.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -11,4 +11,4 @@ sphinx:
configuration: docs/conf.py

conda:
environment: docs/environment_docs.yml
environment: environment.yml
50 changes: 0 additions & 50 deletions docs/environment_docs.yml

This file was deleted.

4 changes: 2 additions & 2 deletions docs/examples/documentation_MPI.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -13,9 +13,9 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"Parcels can be run in Parallel with MPI. To do this, you will need to also install the `mpich` and `mpi4py` packages (e.g. `conda install mpich mpi4py`). \n",
"Parcels can be run in Parallel with MPI. To do this, you will need to also install the `mpich` and `mpi4py` packages (i.e., `conda install -c conda forge mpich mpi4py`).\n",
"\n",
"Note that these packages don't work on Windows, and that they have been installed if you followed the [Installation for Developers](https://docs.oceanparcels.org/en/latest/installation.html#installation-for-developers) instructions.\n"
"Note that MPI support is only for Linux and macOS. There is no Windows support.\n"
]
},
{
Expand Down
4 changes: 3 additions & 1 deletion docs/examples/example_peninsula.py
Original file line number Diff line number Diff line change
Expand Up @@ -185,7 +185,9 @@ def test_peninsula_fieldset_AnalyticalAdvection(mode, mesh, tmpdir):
method=AdvectionAnalytical)
# Test advection accuracy by comparing streamline values
err_adv = np.array([abs(p.p_start - p.p) for p in pset])
assert (err_adv <= 1.e-1).all()

tol = {"scipy": 3.e-1, "jit": 1.e-1}.get(mode)
assert (err_adv <= tol).all()


def fieldsetfile(mesh, tmpdir):
Expand Down
14 changes: 6 additions & 8 deletions docs/installation.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@ If you want to install the latest development version of Parcels and work with f

The steps below are the installation instructions for Linux, macOS and Windows.

.. _step 1 above:

**Step 1:** Install Anaconda's Miniconda following the steps at https://conda.io/docs/user-guide/install/, making sure to select the Python-3 version. If you're on Linux /macOS, the following assumes that you installed Miniconda to your home directory.

**Step 2:** Start a terminal (Linux / macOS) or the Anaconda prompt (Windows). Activate the ``base`` environment of your Miniconda and create an environment containing Parcels, all its essential dependencies, and the nice-to-have cartopy and jupyter packages:
Expand Down Expand Up @@ -62,25 +64,21 @@ The steps below are the installation instructions for Linux, macOS and Windows.
Installation for developers
===========================

If you would prefer to have a development installation of Parcels (i.e., where the code can be actively edited), you can do so by cloning the Parcels repo, installing dependencies using the environment file, and then installing Parcels in an editable mode such that changes to the cloned code can be tested during development.

**Step 1:** Install `micromamba <https://mamba.readthedocs.io/en/latest/index.html>`_, a lightweight version of conda, following the instructions `here <https://mamba.readthedocs.io/en/latest/installation/micromamba-installation.html>`_.

.. note::
If you would prefer to have a development installation of Parcels (i.e., where the code can be actively edited), you can do so by setting up Miniconda (as detailed in step 1 above), cloning the Parcels repo, installing dependencies using the environment file, and then installing Parcels in an editable mode such that changes to the cloned code can be tested during development.

If you prefer to work with Anaconda or Miniconda instead of Micromamba, you will have to remove the ``sel(unix):`` commands in the ``environment.yml`` file (or remove these lines altogether if you're on Windows), since conda does not (yet) support these `selectors`. After that, just replace ``micromamba`` with ``conda`` in the commands below.
**Step 1:** Same as `step 1 above`_.

**Step 2:** Clone the Parcels repo and create a new environment with the dependencies:

.. code-block:: bash

git clone https://github.com/OceanParcels/parcels.git
cd parcels
micromamba env create -f environment.yml
conda env create -f environment.yml

**Step 3:** Activate the environment and install Parcels in editable mode:

.. code-block:: bash

micromamba activate parcels
conda activate parcels
pip install --no-build-isolation --no-deps -e .
2 changes: 0 additions & 2 deletions environment.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,6 @@ dependencies:
- git
- jupyter
- matplotlib-base>=2.0.2
- sel(unix): mpi4py
- sel(unix): mpich
- netcdf4>=1.1.9
- numpy>=1.9.1
- platformdirs
Expand Down
8 changes: 4 additions & 4 deletions tests/test_fieldset_sampling.py
Original file line number Diff line number Diff line change
Expand Up @@ -126,8 +126,8 @@ def test_fieldset_sample(fieldset, xdim=120, ydim=80):
lat = np.linspace(-80, 80, ydim, dtype=np.float32)
v_s = np.array([fieldset.UV[0, 0., 70., x][1] for x in lon])
u_s = np.array([fieldset.UV[0, 0., y, -45.][0] for y in lat])
assert np.allclose(v_s, lon, rtol=1e-7)
assert np.allclose(u_s, lat, rtol=1e-7)
assert np.allclose(v_s, lon, rtol=1e-5) # Tolerances were rtol=1e-7, increased due to numpy v2 float32 changes (see #1603)
assert np.allclose(u_s, lat, rtol=1e-5)


def test_fieldset_sample_eval(fieldset, xdim=60, ydim=60):
Expand All @@ -136,8 +136,8 @@ def test_fieldset_sample_eval(fieldset, xdim=60, ydim=60):
lat = np.linspace(-80, 80, ydim, dtype=np.float32)
v_s = np.array([fieldset.UV.eval(0, 0., 70., x)[1] for x in lon])
u_s = np.array([fieldset.UV.eval(0, 0., y, 0.)[0] for y in lat])
assert np.allclose(v_s, lon, rtol=1e-7)
assert np.allclose(u_s, lat, rtol=1e-7)
assert np.allclose(v_s, lon, rtol=1e-5) # Tolerances were rtol=1e-7, increased due to numpy v2 float32 changes (see #1603)
assert np.allclose(u_s, lat, rtol=1e-5)


@pytest.mark.parametrize('mode', ['scipy', 'jit'])
Expand Down
Loading