C3S_34e Meeting with Ouranos and CRIM: Feb 2021

[agstephens]

Proposed agenda - C3S_34e Meeting with Ouranos and CRIM: 18th Feb 2021

1. Minutes and actions from previous meeting

Previous minutes: #82

Previous actions

• AS: set up a meeting with CRIM & Ouranos - EO and ML. Agenda: Letter of intent; AI collaborations/opportunities; Other info. @cehbrecht can involve people from DKRZ and GERICS to the AI meeting.
• AS: push the idea to have an international 3-day sprint (running through the night across continents) for working on the Earth System Model STAC profile (Roadmap for merging with STAC NCAR/esm-collection-spec#21). @cehbrecht can involve people from DKRZ working on intake and STAC.

2. Project update: c3s_34e [Ag]

• current production service deadline: end of Feb
  • serving subset of CMIP6 to C3S service
  • including PROV records as JSON/PNG
• Changes to our stack:
  • removed git submodule - implementing fix so that tests will always: clone and/or pull (unless switched off via env var)
  • implemented average_over_dims() function
  • refactored clisops.ops. to use a base class for all operators
• Ready to agree on next release of clisops
• Travis-CI replacement:
  • moved to GitHub CI for now
• C3S keen for us to use the apply_fixes approach for CMIP6 decadal runs (to fix the representation of multiple time axes in the datasets)

3. Project updates: relevant Ouranos/CRIM projects [David/Tom]

Ouranos:

CRIM:

4. AI collaborations?

• Still pending:
• AS: set up meeting with CRIM & Ouranos - EO and ML. Agenda: Letter of intent; AI collaborations/opportunities; Other info

5. Discussion:

• Averaging:
• Regridding:

6. Agree next steps and next meeting

• Set up a meeting about collaboration and AI
• Next monthly meeting: Thursday 18th March, https://ukri.zoom.us/j/98246249580

[agstephens]

Minutes - C3S_34e Meeting with Ouranos and CRIM: 18th Feb 2021

1. Minutes and actions from previous meeting

Previous minutes: #82

Previous actions

• AS: set up a meeting with CRIM & Ouranos - EO and ML. Agenda: Letter of intent; AI collaborations/opportunities; Other info. @cehbrecht can involve people from DKRZ and GERICS to the AI meeting.

• AS: push the idea to have an international 3-day sprint (running through the night across continents) for working on the Earth System Model STAC profile (Roadmap for merging with STAC NCAR/esm-collection-spec#21). @cehbrecht can involve people from DKRZ working on intake and STAC.

New actions

• DH: send AS information about STAC extension approach.
• CEDA: Create next clisops release: with both new average implementations

2. Project update: c3s_34e [Ag]

• current production service deadline: end of Feb
  • serving subset of CMIP6 to C3S service
  • including PROV records as JSON/PNG
• Changes to our stack:
  • removed git submodule - implementing fix so that tests will always: clone and/or pull (unless switched off via env var)
  • implemented average_over_dims() function
  • refactored clisops.ops. to use a base class for all operators
• Ready to agree on next release of clisops
• Travis-CI replacement:
  • moved to GitHub CI for now
• C3S keen for us to use the apply_fixes approach for CMIP6 decadal runs (to fix the representation of multiple time axes in the datasets)

3. Project updates: relevant Ouranos/CRIM projects [David/Tom]

Ouranos:

• Regridding - some minor bug fixes. Some work on adaptive masking.
• Cataloguing:
  • short-term: Intake-ESM
  • medium/long-term: STAC
  • How:
    • scraping metadata from THREDDS NcML access points
    • Split of data types: reanalyses, time series, obs etc
    • Creating independent collections for each of the above - each with its own controlled vocabularies.
    • Should be able to provide over-arching search
    • Using NcML views of datasets; not working at the file level; aggregating to the realm-level.

CRIM:

• Francis is participating in OSGEO sprint - Weaver
  • implementing the RESTful interface to WPS - in owslib

4. AI collaborations?

• Still pending:

• AS: set up meeting with CRIM & Ouranos - EO and ML. Agenda: Letter of intent; AI collaborations/opportunities; Other info

• EO Common Architecture: CRIM is looking at this as a focal point for collaboration - using Machine Learning use cases:

  1. Model catalog
  2. Pipeline replication
  3. Data annotation
  4. Inference as a Service
  5. Model training

5. Discussion:

• Averaging:
  • NOTE: for some variables you need access to additional ancillary variables:
    • areacell
    • land/ocean-area fraction - e.g. used by fluxes
• Regridding:
  • we will start a clisops.ops.regrid module soon.

6. Agree next steps and next meeting

• Set up a meeting about collaboration and AI
• Next monthly meeting: Thursday 18th March, https://ukri.zoom.us/j/98246249580