Skip to content

Local version of NSIDC passive microwaves sea ice tools

License

Notifications You must be signed in to change notification settings

andypbarrett/nsidc-seaice

Repository files navigation

This is a modified version of the NSIDC sea ice CLI and tools. It has been modified to work on a local directory structure. Routines to read binary sea ice concentration grids into xarray.DataArray and xarray.Dataset objects have been added. This will make the sea ice data more "Analysis Ready".

To make this work for a local directory structure, users will need to add a override.yaml file with key, value pairs for setting DEFAULT_FINAL_SEA_ICE_PATHS and DEFAULT_NRT_SEA_ICE_PATHS. These are both . in seaice.nasateam.constants.

DEFAULT_FINAL_SEA_ICE_PATHS: ['local_path_to_NSIDC-0051']
DEFAULT_NRT_SEA_ICE_PATHS: ['local_path_to_NSIDC-0081']

An environment variable OVERRIDE_NASATEAM_CONSTANTS containing the path to override.yaml will then need to be set. For bash this would be:

$ export OVERRIDE_NASATEAM_CONSTANTS=path_to_override.yaml

TODO

Add CLI to package binary grids into netCDF
Update README with install info for this version.

SeaIce

A conda package containing CLIs and subpackages to collect statistics and perform analysis on sea ice data for the Sea Ice Index, ASINA, and more.

The subpackages used to be independent packages; their old READMEs are available in README/.

If you have a project which uses the old packages and you want to upgrade to using the new singular package, see UPGRADING.md.

CircleCI

Anaconda-Server Badge Anaconda-Server Badge Anaconda-Server Badge Anaconda-Server Badge

Prerequisites

Development

Install dependencies:

$ conda env create -f environment.yml
$ source activate seaice

Run unit tests:

$ inv test.unit

Run integration tests -- requires access to the DATASETS share:

$ inv test.regression

Updating environment.yml

If changes to dependency versions must be made, environment.yml should also be updated. You can start from an environment created from the file and then install the new packages, and then run update the environment file:

$ conda install ...  # whatever new versions of packages you want
$ conda env export > environment.yml

If updating some dependencies and you want to start from scratch to ensure any unnecessary packages are not included in the environment, create a new environment, install all of the packages from both the build and run sections of recipe/meta.yaml, as well as the packages bumpversion and conda-build, and run the export command again:

$ conda create --name seaice
$ conda install bumpversion conda-build ...  # list all packages from recipe/meta.yaml here
$ conda env export > environment.yml

Workflow

TL;DR: Use GitHub Flow.

In more detail:

  1. Create a feature branch.
  2. Create and push commits on that branch.
  3. The feature branch will get built on CircleCI with each push.
  4. Update the CHANGELOG with description of changes.
  5. Create a Pull Request on BitBucket.
  6. When the feature PR is merged, master will get built on CircleCI.

Releasing

  1. Update the CHANGELOG to list the new version.

  2. Add files and commit

     $ git add CHANGELOG.md ...
     $ git commit -m "Release v.X.Y.Z"
    
  3. Bump the version to the desired level:

     $ bumpversion (major|minor|patch)
    
  4. Push

     $ git push origin master --tags
    

CircleCI will build the conda package and publish it to anaconda.org.

Installing

To install and use it in another project:

$ conda install seaice

Overriding configuration

In order to override constants, populate the OVERRIDE_NASATEAM_CONSTANTS environment variable with the path to a YAML file. In a typical NSIDC deployment, this overrides.yaml file exists on the app network share.

Docker

This project has had a history of issues with dependencies breaking over time with no changes to dependency configuration files (environment.yaml, meta.yaml). For this reason, we manually push images to DockerHub as a record of working dependencies and as a potential fallback in case a quick fix is needed.

TODO: Build continuous deployment to DockerHub in addition to Anaconda.org TODO: Start deploying seaice CLI to production as a Docker image

License

See LICENSE file for details.

Important Links

About

Local version of NSIDC passive microwaves sea ice tools

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages