Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow fix_file to return Cube and CubeList objects #2579

Open
wants to merge 36 commits into
base: main
Choose a base branch
from

Conversation

schlunma
Copy link
Contributor

@schlunma schlunma commented Nov 12, 2024

Description

Currently, if a file cannot be properly read with Iris, we use fix_file to create a copy of that file and modify it using netCDF4.Dataset (example). This is very inefficient and slow.

A much better way to deal with this is to read the file with ncdata or xarray and then use ncdata to convert that object to an Iris object. However, for this to work, we need to allow fix_file to return Iris objects (instead of paths) and load to read Iris objects (instead of paths). This PR does that.

It also provides a convenience function dataset_to_iris that allows to convert an xarray or ncdata object to an Iris object very similar to our load function.

Closes #2129
Related to #674

Link to documentation:


Before you get started

Checklist

It is the responsibility of the author to make sure the pull request is ready to review. The icons indicate whether the item will be subject to the 🛠 Technical or 🧪 Scientific review.


To help with the number pull requests:

@schlunma schlunma added the fix for dataset Related to dataset-specific fix files label Nov 12, 2024
@schlunma schlunma added this to the v2.12.0 milestone Nov 12, 2024
@schlunma schlunma self-assigned this Nov 12, 2024
Copy link

codecov bot commented Nov 12, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 94.96%. Comparing base (e09e396) to head (863b0f1).

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2579      +/-   ##
==========================================
+ Coverage   94.93%   94.96%   +0.02%     
==========================================
  Files         252      252              
  Lines       14620    14666      +46     
==========================================
+ Hits        13880    13927      +47     
+ Misses        740      739       -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@schlunma schlunma marked this pull request as ready for review December 11, 2024 13:44
@schlunma schlunma mentioned this pull request Dec 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
fix for dataset Related to dataset-specific fix files
Projects
No open projects
Development

Successfully merging this pull request may close these issues.

Rethinking fix_file
3 participants