Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add checks for missing data #71

Merged
merged 15 commits into from
Mar 21, 2023
Merged

Add checks for missing data #71

merged 15 commits into from
Mar 21, 2023

Conversation

BSchilperoort
Copy link
Contributor

@BSchilperoort BSchilperoort commented Mar 3, 2023 •
edited
Loading

Based off the LAI branch.

  • Refactored global data modules to be in a separate folder
  • Add checks for data availability (where applicable):
    • Are the expected files present?
    • Are the right variables in these files?
    • Is the location within the lat/lon bounds of the file
    • Are the start/end times within the time bounds of the file
    • Pick the nearest non-NaN location within some maximum distance

@review-notebook-app
Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.

Powered by ReviewNB

@BSchilperoort BSchilperoort marked this pull request as ready for review March 10, 2023 09:00
@geek-yang geek-yang self-requested a review March 16, 2023 11:42
geek-yang
geek-yang requested changes Mar 17, 2023
View reviewed changes
Copy link
Member

@geek-yang geek-yang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A big PR, thanks for adding these checks. In general the code looks quite good to me. Find some typos, which are pretty minor. But one thing I think could be improved is the input variables, which can be "shorten" to avoid duplication. See my comment in the code. Thanks for your work!

PyStemmusScope/global_data/utils.py Show resolved Hide resolved
PyStemmusScope/global_data/global_data_selection.py Show resolved Hide resolved
PyStemmusScope/global_data/cams_co2.py Outdated Show resolved Hide resolved
PyStemmusScope/global_data/copernicus_lai.py Outdated Show resolved Hide resolved
PyStemmusScope/global_data/era5.py Outdated Show resolved Hide resolved
tests/test_global_data_selection.py Outdated Show resolved Hide resolved
tests/test_global_data_selection.py Outdated Show resolved Hide resolved
@sonarqubecloud
Copy link

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 2 Code Smells

93.2% 93.2% Coverage
0.0% 0.0% Duplication

geek-yang
geek-yang approved these changes Mar 21, 2023
View reviewed changes
Copy link
Member

@geek-yang geek-yang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for addressing my comments @BSchilperoort .

@BSchilperoort
Copy link
Contributor Author

Thank you for the review, Yang!

@BSchilperoort BSchilperoort merged commit b35bd24 into main Mar 21, 2023
@BSchilperoort BSchilperoort deleted the missing_data_checks branch March 21, 2023 10:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@geek-yang geek-yang geek-yang approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@BSchilperoort @geek-yang