-
Notifications
You must be signed in to change notification settings - Fork 3
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Handle HRDPS change from west domain to continental domain #156
Commits on Feb 23, 2023
-
Chg weather config to 2.5km HRDPS continental product
ECCC stopped producing 2.5km HRDPS west product on 22-Feb-2023. It was part of the polar stereographic grid product suite that they replaced with the rotated lat-lon grid product, but that product is only available for the continental domain.
Configuration menu - View commit details
-
Copy full SHA for 440f897 - Browse repository at this point
Copy the full SHA 440f897View commit details
Commits on Feb 24, 2023
-
Add --run-date option to download_weather worker
Replaces --yesterday option. ECCC hpfx server holds ~60 days of archived model products. So, we now have the option of downloading for dates further into the past than just the previous day.
Configuration menu - View commit details
-
Copy full SHA for 5a4bde3 - Browse repository at this point
Copy the full SHA 5a4bde3View commit details -
Configuration menu - View commit details
-
Copy full SHA for a7427a1 - Browse repository at this point
Copy the full SHA a7427a1View commit details -
Configuration menu - View commit details
-
Copy full SHA for 6e10eef - Browse repository at this point
Copy the full SHA 6e10eefView commit details -
Configuration menu - View commit details
-
Copy full SHA for 1782ea4 - Browse repository at this point
Copy the full SHA 1782ea4View commit details -
Configuration menu - View commit details
-
Copy full SHA for cb1381e - Browse repository at this point
Copy the full SHA cb1381eView commit details
Commits on Feb 26, 2023
-
Configuration menu - View commit details
-
Copy full SHA for 00c2f51 - Browse repository at this point
Copy the full SHA 00c2f51View commit details -
Configuration menu - View commit details
-
Copy full SHA for 4922459 - Browse repository at this point
Copy the full SHA 4922459View commit details -
Configuration menu - View commit details
-
Copy full SHA for a1d97c0 - Browse repository at this point
Copy the full SHA a1d97c0View commit details -
Refactor _define_forecast_segments_nowcast()
Includes part of work required for issue #23
Configuration menu - View commit details
-
Copy full SHA for 4292e18 - Browse repository at this point
Copy the full SHA 4292e18View commit details -
Refactor _define_forecast_segments_forecast2()
Includes part of work required for issue #23
Configuration menu - View commit details
-
Copy full SHA for 5a36354 - Browse repository at this point
Copy the full SHA 5a36354View commit details
Commits on Mar 2, 2023
-
Add pywgrib2_xr package as dependency
Use SalishSeaCast branch in SalishSeaCast/pywgrib2_xr fork to work under Python 3.10 and xarray>=2022.11. Also requires installation of libwgrib2 from SalishSeaCast branch in SalishSeaCast/libwgrib2 fork.
Configuration menu - View commit details
-
Copy full SHA for 5621470 - Browse repository at this point
Copy the full SHA 5621470View commit details -
Configuration menu - View commit details
-
Copy full SHA for fd18ca7 - Browse repository at this point
Copy the full SHA fd18ca7View commit details
Commits on Mar 3, 2023
-
Improve GHA pytest-with-coverage workflow comments
Mostly to trigger another workflow run with tweaked setup.py in the pywgrib2_xr fork.
Configuration menu - View commit details
-
Copy full SHA for dfc9520 - Browse repository at this point
Copy the full SHA dfc9520View commit details -
Install vendored libwgrib2 in GHA pytest-with-coverage
Mostly to trigger another workflow run with tweaked setup.py in the pywgrib2_xr fork.
Configuration menu - View commit details
-
Copy full SHA for 6212d6f - Browse repository at this point
Copy the full SHA 6212d6fView commit details -
Configuration menu - View commit details
-
Copy full SHA for d54891d - Browse repository at this point
Copy the full SHA d54891dView commit details -
Configuration menu - View commit details
-
Copy full SHA for 9b8fab3 - Browse repository at this point
Copy the full SHA 9b8fab3View commit details -
Configuration menu - View commit details
-
Copy full SHA for 63b4eff - Browse repository at this point
Copy the full SHA 63b4effView commit details -
Drop GRIB2 file existence & 0 length checks
Artefact of early days of SalishSeaNowcast development before automation included assurance that
Configuration menu - View commit details
-
Copy full SHA for 0b126c1 - Browse repository at this point
Copy the full SHA 0b126c1View commit details -
Factor out _wgrib2_append() function
Handles finicky details of pywgrib2_xr.wgrib(): * all pywgrib2_xr.wgrib() args must be strings * files must be freed after use to close them
Configuration menu - View commit details
-
Copy full SHA for 2763c47 - Browse repository at this point
Copy the full SHA 2763c47View commit details -
Configuration menu - View commit details
-
Copy full SHA for 8dd5d2f - Browse repository at this point
Copy the full SHA 8dd5d2fView commit details -
Refactor _concat_hourly_gribs()
Includes part of work required for issue #23.
Configuration menu - View commit details
-
Copy full SHA for 5d6a8c0 - Browse repository at this point
Copy the full SHA 5d6a8c0View commit details -
Drop TCDC_Sfc variable from weather collection
Total cloud in percent that was used for parametrization of radiation missing from 2007-2014 GEMLAM. No longer needed.
Configuration menu - View commit details
-
Copy full SHA for 6ff7944 - Browse repository at this point
Copy the full SHA 6ff7944View commit details
Commits on Mar 4, 2023
-
Refactor _crop_to_watersheds()
Includes factoring out _wgrib2_crop() function to handles finicky details of pywgrib2_xr.wgrib(): * all pywgrib2_xr.wgrib() args must be strings * files must be freed after use to close them Also includes part of work required for issue #23.
Configuration menu - View commit details
-
Copy full SHA for e16fad3 - Browse repository at this point
Copy the full SHA e16fad3View commit details -
Temporarily remove launch of upload_forcing worker
Need this so that weather collection, etc. can run in automation before grib_to_netcdf rework is ready for use.
Configuration menu - View commit details
-
Copy full SHA for ecb2e84 - Browse repository at this point
Copy the full SHA ecb2e84View commit details
Commits on Mar 6, 2023
-
Configuration menu - View commit details
-
Copy full SHA for fcfff41 - Browse repository at this point
Copy the full SHA fcfff41View commit details
Commits on Mar 8, 2023
-
Revert addition of pywgrib2_xr pkg as dependency
Using forks of pywgrib2_xr and libwgrib2 was yucky, but ultimately, the inability of wgrib2 to export the rotated lat-lon grid to netCDF was the death knell.
Configuration menu - View commit details
-
Copy full SHA for a0f70a0 - Browse repository at this point
Copy the full SHA a0f70a0View commit details -
Configuration menu - View commit details
-
Copy full SHA for aa88e58 - Browse repository at this point
Copy the full SHA aa88e58View commit details -
Configuration menu - View commit details
-
Copy full SHA for 72c428d - Browse repository at this point
Copy the full SHA 72c428dView commit details -
Configuration menu - View commit details
-
Copy full SHA for e782d24 - Browse repository at this point
Copy the full SHA e782d24View commit details -
Configuration menu - View commit details
-
Copy full SHA for 043d3b7 - Browse repository at this point
Copy the full SHA 043d3b7View commit details -
Configuration menu - View commit details
-
Copy full SHA for 42248d2 - Browse repository at this point
Copy the full SHA 42248d2View commit details -
Configuration menu - View commit details
-
Copy full SHA for db6b23c - Browse repository at this point
Copy the full SHA db6b23cView commit details -
Configuration menu - View commit details
-
Copy full SHA for 417412e - Browse repository at this point
Copy the full SHA 417412eView commit details
Commits on Mar 9, 2023
-
Change 2.5km weather variable names to triplets
We need the MSC file variable name, the GRIB standard variable name, and the variable name to use in the NEMO forcing file for processing of the HRDPS continental GRIB files.
Configuration menu - View commit details
-
Copy full SHA for 14da1b9 - Browse repository at this point
Copy the full SHA 14da1b9View commit details -
Chg forcing filename template from ops_ to hrdps_
So that we can clearly distinguish from HRDPS continental grid in contrast to those from HRDPS west for GEMLAM experimental grids.
Configuration menu - View commit details
-
Copy full SHA for 5ca1efd - Browse repository at this point
Copy the full SHA 5ca1efdView commit details -
Configuration menu - View commit details
-
Copy full SHA for 8395af2 - Browse repository at this point
Copy the full SHA 8395af2View commit details
Commits on Mar 10, 2023
-
Set logging level for cfgrib library to WARNING
DEBUG is too noisy.
Configuration menu - View commit details
-
Copy full SHA for e90f26d - Browse repository at this point
Copy the full SHA e90f26dView commit details -
Add _apportion_accumulation_vars()
Add list of 2.5km forecast accumulation variables to weather config. Also fix bugs in grib files date and hours selection.
Configuration menu - View commit details
-
Copy full SHA for fb92d18 - Browse repository at this point
Copy the full SHA fb92d18View commit details -
Configuration menu - View commit details
-
Copy full SHA for 30b58f1 - Browse repository at this point
Copy the full SHA 30b58f1View commit details -
Add testing mock for _apportion_accumulation_vars()
Missed in fb92d1.
Configuration menu - View commit details
-
Copy full SHA for 112e132 - Browse repository at this point
Copy the full SHA 112e132View commit details -
Configuration menu - View commit details
-
Copy full SHA for 3898e16 - Browse repository at this point
Copy the full SHA 3898e16View commit details -
Configuration menu - View commit details
-
Copy full SHA for e5635f3 - Browse repository at this point
Copy the full SHA e5635f3View commit details -
Configuration menu - View commit details
-
Copy full SHA for c00117d - Browse repository at this point
Copy the full SHA c00117dView commit details -
Configuration menu - View commit details
-
Copy full SHA for a80de78 - Browse repository at this point
Copy the full SHA a80de78View commit details -
Configuration menu - View commit details
-
Copy full SHA for c1ad362 - Browse repository at this point
Copy the full SHA c1ad362View commit details
Commits on Mar 11, 2023
-
Configuration menu - View commit details
-
Copy full SHA for 3aaf5ac - Browse repository at this point
Copy the full SHA 3aaf5acView commit details -
Configuration menu - View commit details
-
Copy full SHA for db26d36 - Browse repository at this point
Copy the full SHA db26d36View commit details -
Configuration menu - View commit details
-
Copy full SHA for 6559288 - Browse repository at this point
Copy the full SHA 6559288View commit details -
Configuration menu - View commit details
-
Copy full SHA for 944f8ba - Browse repository at this point
Copy the full SHA 944f8baView commit details -
Configuration menu - View commit details
-
Copy full SHA for 5175936 - Browse repository at this point
Copy the full SHA 5175936View commit details
Commits on Mar 12, 2023
-
Drop unneeded coordinate variables
"time", "longitude", and "latitude" variables from GRIB files are replaced by time_counter, "nav_lon", and "nav_lat" in NEMO datasets, so they are unneeded.
Configuration menu - View commit details
-
Copy full SHA for ffc1fc6 - Browse repository at this point
Copy the full SHA ffc1fc6View commit details -
Configuration menu - View commit details
-
Copy full SHA for 711d49f - Browse repository at this point
Copy the full SHA 711d49fView commit details -
Move setting dataset history attr to _write_netcdf()
Necessary so that we can use xarray.combine_by_coords() to build 3-section run-date dataset.
Configuration menu - View commit details
-
Copy full SHA for 5ab4f7e - Browse repository at this point
Copy the full SHA 5ab4f7eView commit details -
Configuration menu - View commit details
-
Copy full SHA for 2efbc28 - Browse repository at this point
Copy the full SHA 2efbc28View commit details
Commits on Mar 13, 2023
-
Configuration menu - View commit details
-
Copy full SHA for b42988f - Browse repository at this point
Copy the full SHA b42988fView commit details -
Configuration menu - View commit details
-
Copy full SHA for f71fdb9 - Browse repository at this point
Copy the full SHA f71fdb9View commit details -
Configuration menu - View commit details
-
Copy full SHA for fd5f233 - Browse repository at this point
Copy the full SHA fd5f233View commit details -
Configuration menu - View commit details
-
Copy full SHA for deebae2 - Browse repository at this point
Copy the full SHA deebae2View commit details -
Configuration menu - View commit details
-
Copy full SHA for 26b02b5 - Browse repository at this point
Copy the full SHA 26b02b5View commit details -
Configuration menu - View commit details
-
Copy full SHA for b095d7b - Browse repository at this point
Copy the full SHA b095d7bView commit details -
Configuration menu - View commit details
-
Copy full SHA for e08666b - Browse repository at this point
Copy the full SHA e08666bView commit details -
Configuration menu - View commit details
-
Copy full SHA for f16a9ca - Browse repository at this point
Copy the full SHA f16a9caView commit details -
Configuration menu - View commit details
-
Copy full SHA for 5b1b0bc - Browse repository at this point
Copy the full SHA 5b1b0bcView commit details -
Configuration menu - View commit details
-
Copy full SHA for 27cfb2a - Browse repository at this point
Copy the full SHA 27cfb2aView commit details -
Configuration menu - View commit details
-
Copy full SHA for faf3024 - Browse repository at this point
Copy the full SHA faf3024View commit details -
Configuration menu - View commit details
-
Copy full SHA for 5445eba - Browse repository at this point
Copy the full SHA 5445ebaView commit details -
Configuration menu - View commit details
-
Copy full SHA for 18f7adc - Browse repository at this point
Copy the full SHA 18f7adcView commit details
Commits on Mar 14, 2023
-
Configuration menu - View commit details
-
Copy full SHA for 3b19e9d - Browse repository at this point
Copy the full SHA 3b19e9dView commit details -
Use persistent dask cluster on salish for processing
Reduces grib_to_netcdf memory footprint and processing time a little compared to using dask's default threads scheduler.
Configuration menu - View commit details
-
Copy full SHA for f43d102 - Browse repository at this point
Copy the full SHA f43d102View commit details -
Chg _write_netcdf() file created log msg to info level
Also improve test coverage of _write_netcdf().
Configuration menu - View commit details
-
Copy full SHA for b547803 - Browse repository at this point
Copy the full SHA b547803View commit details