Code

Use dask to run tasks#1714

Code

Closed

bouweandela wants to merge 31 commits intomainESMValGroup/ESMValCore:mainfrom dask-distributedESMValGroup/ESMValCore:dask-distributedCopy head branch name to clipboard

+292-70

Commits

Commits on Sep 2, 2022

• Use dask to run preprocessing tasks (and disable other tasks for now)

  bouweandela

  committed
  63b032a

Commits on Sep 13, 2022

• Improve use of dask client as suggested by @zklaus

  bouweandela

  committed
  4792193

• Restore sequential and parallel task run

  bouweandela

  committed
  11b12f1

Commits on Sep 23, 2022

• Add missing compute argument to save function for multimodel statistics preprocessor output files

  bouweandela

  committed
  6faa8f6

• Add distributed as a dependency

  bouweandela

  committed
  49e3589

• Restore previous API to fix tests

  bouweandela

  committed
  54edb73

• Add xarray as a dependency

  bouweandela

  committed
  021b3e0

• Merge branch 'main' into dask-distributed

  bouweandela

  authored
  53aa585

Commits on Oct 14, 2022

• Merge branch 'main' of github.com:ESMValGroup/ESMValCore into dask-distributed

  bouweandela

  committed
  d5cf4f6

• Support configuring dask

  bouweandela

  committed
  0f5bda8

Commits on Oct 16, 2022

• Add a suffix to output_directory if it exists instead of stopping

  bouweandela

  committed
  52af816

Commits on Oct 18, 2022

• Fix tests

  bouweandela

  committed
  823c731

Commits on Nov 7, 2022

• single call to compute

  fnattino

  committed
  6f5a6bf

Commits on Nov 10, 2022

• Only start cluster if necessary and fix filename-future mapping

  bouweandela

  committed
  b455bbb

Commits on Nov 21, 2022

• Merge branch 'main' of github.com:ESMValGroup/ESMValCore into dask-distributed

  bouweandela

  committed
  05d69f7

• Use iris (SciTools/iris#5031) for saving

  bouweandela

  committed
  be991f8

Commits on Nov 22, 2022

• Merge branch 'main' of github.com:ESMValGroup/ESMValCore into dask-distributed

  bouweandela

  committed
  93b6da1

Commits on Dec 5, 2022

• Point to iris branch

  bouweandela

  committed
  37bb757

• Merge branch 'main' into dask-distributed

  bouweandela

  authored
  2b60264

Commits on Mar 30, 2023

• Merge branch 'main' of github.com:ESMValGroup/ESMValCore into dask-distributed

  bouweandela

  committed
  66bdf2e

Commits on Apr 4, 2023

• Work in progress

  bouweandela

  committed
  065b8a4

• Update branch name

  bouweandela

  committed
  f47d6da

• Remove type hint

  bouweandela

  committed
  6e25901

Commits on Apr 21, 2023

• Get iris from main branch

  bouweandela

  committed
  e176745

• Merge branch 'main' into dask-distributed

  bouweandela

  authored
  d6787fa

Commits on Apr 28, 2023

• Add default scheduler

  bouweandela

  committed
  222c7e5

Commits on May 11, 2023

• Use release candidate

  bouweandela

  committed
  b760abb

• Try to install iris from PyPI

  bouweandela

  committed
  6b35638

Commits on May 12, 2023

• Merge branch 'main' of github.com:ESMValGroup/ESMValCore into dask-distributed

  bouweandela

  committed
  cf79d20

Commits on May 19, 2023

• Update dependencies

  bouweandela

  committed
  4348953

Commits on May 23, 2023

• Remove pip install of ESMValTool_sample_data

  bouweandela

  committed
  6bf6328