Bugfix: Fix time interpolation of monthly climatology data between December 15 and January 15

[JohnHalleyGotway]

Describe the Problem

This problem was identified by @GwenChen-NOAA via dtcenter/METplus#2007 discussion. The bug exists when processing monthly climatology data for valid times between Dec 15 and Jan 15. MET attempts to find matching climo data with timestamps before and after the valid time. However for Dec dates, only the matching Dec climo data is found. And for Jan dates, only the matching Jan climo data is found. It doesn't handle the year wrap correctly.

I suspect the same problem may exist for daily climatology data at the end of each month. Be sure to check for and fix that problem if it also exists.

Expected Behavior

Time interpolation of monthly and daily climo datasets should work consistently for all valid times, regardless of their proximity to the end of the month or end of the year.

Environment

Describe your runtime environment:
1. Machine: (e.g. HPC name, Linux Workstation, Mac Laptop): Mac Laptop and WCOSS2
2. OS: (e.g. RedHat Linux, MacOS): Replicated on MacOS but not OS-dependent.
3. Software version number(s): MET version 11.0.0, but exists for earlier versions as well.

To Reproduce

Describe the steps to reproduce the behavior:
1. Pull Nov/Dec/Jan climo mean files:

wget https://dtcenter.ucar.edu/dfiles/code/METplus/MET/MET_unit_test/unit_test/climatology_data/NCEP_2.5deg/pgba_mean.19591115
wget https://dtcenter.ucar.edu/dfiles/code/METplus/MET/MET_unit_test/unit_test/climatology_data/NCEP_2.5deg/pgba_mean.19591215
wget https://dtcenter.ucar.edu/dfiles/code/METplus/MET/MET_unit_test/unit_test/climatology_data/NCEP_2.5deg/pgba_mean.19590115

2. Download a GFS file

wget https://dtcenter.ucar.edu/dfiles/code/METplus/MET/MET_unit_test/unit_test/model_data/grib2/gfsanl/gfsanl_4_20120410_1200_000.grb2

3. Save this GridStatConfig_climo.txt file.
GridStatConfig_climo.txt

4. Set November VALID_TIME and run Grid-Stat

export VALID_TIME="20221125_12";
grid_stat \
      gfsanl_4_20120409_1200_000.grb2 gfsanl_4_20120409_1200_000.grb2 \
      GridStatConfig_climo.txt -log run_${VALID_TIME}.log -v 3

Note this log message that indicates time interpolation works as expected:

DEBUG 3: For forecast valid at 20221125_120000, found 2 climatology field(s) with valid time(s): 20221115_120000, 20221215_120000
DEBUG 3: Interpolating climatology fields at 20221115_120000 and 20221215_120000 to 20221125_120000 using the DW_MEAN interpolation method.

5. Rerun using a December VALID_TIME:

export VALID_TIME="20221225_12";
grid_stat \
      gfsanl_4_20120409_1200_000.grb2 gfsanl_4_20120409_1200_000.grb2 \
      GridStatConfig_climo.txt -log run_${VALID_TIME}.log -v 3

Note this log message that indicates only the December data was found, not January data:

DEBUG 3: For forecast valid at 20221225_120000, found 1 climatology field(s) with valid time(s): 20221215_120000

And that's the problem, MET should find the monthly Dec and Jan data and interpolate them to the 20221215_12 valid time being evaluated.

Relevant Deadlines

List relevant project deadlines here or state NONE.

Funding Source

Define the source of funding and account keys here or state NONE.

Define the Metadata

Assignee

• Select engineer(s) or no engineer required
• Select scientist(s) or no scientist required

Labels

• Select component(s)
• Select priority
• Select requestor(s)

Projects and Milestone

• Select Organization level Project for support of the current coordinated release
• Select Repository level Project for development toward the next official release or add alert: NEED PROJECT ASSIGNMENT label
• Select Milestone as the next bugfix version

Define Related Issue(s)

Consider the impact to the other METplus components.

• METplus, MET, METdataio, METviewer, METexpress, METcalcpy, METplotpy
  No impacts.

Bugfix Checklist

See the METplus Workflow for details.

• Complete the issue definition above, including the Time Estimate and Funding Source.
• Fork this repository or create a branch of main_<Version>.
  Branch name: bugfix_<Issue Number>_main_<Version>_<Description>
• Fix the bug and test your changes.
• Add/update log messages for easier debugging.
• Add/update unit tests.
• Add/update documentation.
• Push local changes to GitHub.
• Submit a pull request to merge into main_<Version>.
  Pull request: bugfix <Issue Number> main_<Version> <Description>
• Define the pull request metadata, as permissions allow.
  Select: Reviewer(s) and Development issue
  Select: Organization level software support Project for the current coordinated release
  Select: Milestone as the next bugfix version
• Iterate until the reviewer(s) accept and merge your changes.
• Delete your fork or branch.
• Complete the steps above to fix the bug on the develop branch.
  Branch name: bugfix_<Issue Number>_develop_<Description>
  Pull request: bugfix <Issue Number> develop <Description>
  Select: Reviewer(s) and Development issue
  Select: Repository level development cycle Project for the next official release
  Select: Milestone as the next official version
• Close this issue.

[JohnHalleyGotway]

@j-opatz, FYI, the faulty logic lives in the read_climo_file() function. I'm stepping through the details now search for a nice solution, but it gets confusing quickly!

[j-opatz]

During a one-on-one discussion today between myself and John HG, the method for calculating the DW_MEAN of climatological interpolation for leap years, specifically that period of time residing between February 15th and March 15th, was considered. Should the MET continue its method of keeping the number of dates (and weights) consistent, thus requiring a shift in the date sides (e.g., Feb 15th to March 14th), or should it change to a new method of keeping the date sides consistent (e.g. 15th to the 15th) and have the weights of each day be slightly adjusted?

After iteration with CPC, the consensus on MET's handling of valid dates residing between February 15th and March 15th of a leap year using the DW_MEAN method of interpolation is that it's doing so correctly. The desire to maintain a consistent weight across a once-every-four-year temporal space is higher than the alternative method of keeping the bookend dates (15th to 15th of the month) consistent.

[AliciaBentley-NOAA]

I agree that we would like to maintain a consistent weight across a once-every-four-year temporal space. Thanks!