Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bugfix #2412 main_v11.0 climo #2420

Merged
merged 10 commits into from
Jan 25, 2023
Merged

Conversation

JohnHalleyGotway
Copy link
Collaborator

@JohnHalleyGotway JohnHalleyGotway commented Jan 22, 2023

Expected Differences

These code changes are more substantial than I was hoping. However, I think the resulting logic and log messages are both more clear and intuitive. For each climo record being considered, the code now computes 2 time differences: the number of DAYS that differ and the number of HMS seconds that differ. These are both stored as a +/- time offset relative to the forecast valid time in seconds.

The day difference is checked against the day_interval config entry. And the hms difference is checked against the hour_interval config entry.

  • Do these changes introduce new tools, command line arguments, or configuration file options? [No]

    If yes, please describe:

  • Do these changes modify the structure of existing or add new output data types (e.g. statistic line types or NetCDF variables)? [No]

    If yes, please describe:

Pull Request Testing

  • Describe testing already performed for these changes:

    Tested a variety of different climo dates in seneca:/d1/projects/MET/MET_pull_requests/met-11.0.1/issue_2242/test_climo as the met_test user.

Test after Dec 15th:

./test_climo.sh 20201225_06
DEBUG 3: For forecast valid at 20201225_060000, found 2 climatology field(s) with valid time(s): 20201215_060000, 20210115_060000
DEBUG 3: Interpolating climatology fields at 20201215_060000 and 20210115_060000 to 20201225_060000 using the DW_MEAN interpolation method.

Test before Jan 15:

./test_climo.sh 20200105_03
DEBUG 3: For forecast valid at 20200105_030000, found 4 climatology field(s) with valid time(s): 20191215_000000, 20191215_060000, 20200115_000000, 20200115_060000
DEBUG 3: Interpolating climatology fields at 20191215_000000 and 20200115_000000 to 20200105_000000 using the DW_MEAN interpolation method.
DEBUG 3: Interpolating climatology fields at 20191215_060000 and 20200115_060000 to 20200105_060000 using the DW_MEAN interpolation method.
DEBUG 3: Interpolating climatology fields at 20200105_000000 and 20200105_060000 to 20200105_030000 using the DW_MEAN interpolation method.
  • Recommend testing for the reviewer(s) to perform, including the location of input datasets, and any additional instructions:

  • Review the logic of the code changes.

  • Could test with more dates on seneca. Could also test with daily climo files rather than just these monthly ones.

  • Do these changes include sufficient documentation updates, ensuring that no errors or warnings exist in the build of the documentation? [No]
    I made no doc changes.

  • Do these changes include sufficient testing updates? [No]
    I did not add tests for the main_v11.0 branch. However, I do plan to add them for these changes in the develop branch with a separate PR.

  • Will this PR result in changes to the test suite? [No]

    If yes, describe the new output and/or changes to the existing output:

  • Please complete this pull request review by [Tues 1/24/23].

Pull Request Checklist

See the METplus Workflow for details.

  • Review the source issue metadata (required labels, projects, and milestone).
  • Complete the PR definition above.
  • Ensure the PR title matches the feature or bugfix branch name.
  • Define the PR metadata, as permissions allow.
    Select: Reviewer(s)
    Select: Organization level software support Project or Repository level development cycle Project
    Select: Milestone as the version that will include these changes
  • After submitting the PR, select Development issue with the original issue number.
  • After the PR is approved, merge your changes. If permissions do not allow this, request that the reviewer do the merge.
  • Close the linked issue and delete your feature or bugfix branch from GitHub.

JohnHalleyGotway and others added 9 commits January 18, 2023 15:52
…ds of day difference and seconds of year difference.
…other bug. With day_interval = 31, it find climo data for Jan 15, Feb 15, and Mar 15. The climo_hms_interp() function is smart enough to correctly interpolate between Jan 15 and Feb 15 and ignore Mar 15. However, there was a bug in the computation of the target valid time.
…nputs have the same valid time, just return the first field rather than erorring out.
…make the log messages a bit more intuitive.
@JohnHalleyGotway JohnHalleyGotway added this to the MET 11.0.1 (bugfix) milestone Jan 22, 2023
@JohnHalleyGotway JohnHalleyGotway changed the title Bugfix #2142 main_v11.0 climo Bugfix #2412 main_v11.0 climo Jan 23, 2023
…ke commands in build_met_docker.sh. This should improve the speed of compilation job in GitHub actions. Using 'make -j install' improves the compilation time from 8m40s to 2m58s. So definitely worth trying!
Copy link
Contributor

@j-opatz j-opatz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tested the feature branch against monthly and daily climatology files, both of which successfully temporally interpolated the expected files, regardless of the requested valid time and date. While it's harder to comment on the exact method of how this was accomplished with the code changes, it seems that providing the day > hour > interpolate logic method has the intended effect. The additional log comments are also greatly appreciated and will no doubt provide users with a better understanding of the back-end logic.

@JohnHalleyGotway JohnHalleyGotway mentioned this pull request Jan 25, 2023
15 tasks
@JohnHalleyGotway JohnHalleyGotway merged commit 5d36e95 into main_v11.0 Jan 25, 2023
JohnHalleyGotway added a commit that referenced this pull request Feb 16, 2023
Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu>
Co-authored-by: jprestop <jpresto@ucar.edu>
Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu>
Co-authored-by: hsoh-u <hsoh@ucar.edu>
Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com>
fix #2389 main_v11.0 flowchart (#2391)
fix definitions of G172 and G220 based on comments in NOAA-EMC/NCEPLIBS-w3emc#157. (#2405)
fix #2380 main_v11.0 override (#2381)
fix #2408 main_v11.0 empty config (#2409)
fix #2390 main_v11.0 fix compiling hdf5 with zlib and handle NetCDF-C zip (#2403)
fix #2415 main_v11.0 modulefiles (#2416)
fix #2412 main_v11.0 climo (#2420)
fix #2426 main_v11.0 buoy (#2432)
fix #2437 main_v11.0 convert (#2438)
fix for main_v11.0, for #2437, forgot one reference to the search_parent for a dictionary lookup.
JohnHalleyGotway added a commit that referenced this pull request Feb 22, 2023
Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu>
Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu>
Co-authored-by: jprestop <jpresto@ucar.edu>
Co-authored-by: hsoh-u <hsoh@ucar.edu>
Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com>
fix #2389 main_v11.0 flowchart (#2391)
fix definitions of G172 and G220 based on comments in NOAA-EMC/NCEPLIBS-w3emc#157. (#2405)
fix #2380 main_v11.0 override (#2381)
fix #2408 main_v11.0 empty config (#2409)
fix #2390 main_v11.0 fix compiling hdf5 with zlib and handle NetCDF-C zip (#2403)
fix #2415 main_v11.0 modulefiles (#2416)
fix #2412 main_v11.0 climo (#2420)
fix #2426 main_v11.0 buoy (#2432)
fix #2437 main_v11.0 convert (#2438)
fix for main_v11.0, for #2437, forgot one reference to the search_parent for a dictionary lookup.
fix 2428 python from env main v11.0 (#2443)
fix 2428 python csv input (#2450)
fix #2452 main_v11.0 airnow (#2453)
fix #2402 main_v11.0 sonarqube (First PR) (#2447)
@JohnHalleyGotway JohnHalleyGotway deleted the bugfix_2412_main_v11.0_climo branch February 28, 2023 19:41
JohnHalleyGotway added a commit that referenced this pull request Mar 31, 2023
Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu>
Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu>
Co-authored-by: John Halley Gotway <johnhg@ucar.edu>
Co-authored-by: jprestop <jpresto@ucar.edu>
Co-authored-by: hsoh-u <hsoh@ucar.edu>
Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com>
fix #2389 main_v11.0 flowchart (#2391)
fix definitions of G172 and G220 based on comments in NOAA-EMC/NCEPLIBS-w3emc#157. (#2405)
fix #2380 main_v11.0 override (#2381)
fix #2408 main_v11.0 empty config (#2409)
fix #2390 main_v11.0 fix compiling hdf5 with zlib and handle NetCDF-C zip (#2403)
fix #2415 main_v11.0 modulefiles (#2416)
fix #2412 main_v11.0 climo (#2420)
fix #2426 main_v11.0 buoy (#2432)
fix #2437 main_v11.0 convert (#2438)
fix for main_v11.0, for #2437, forgot one reference to the search_parent for a dictionary lookup.
fix 2428 python from env main v11.0 (#2443)
fix 2428 python csv input (#2450)
fix #2452 main_v11.0 airnow (#2453)
fix #2402 main_v11.0 sonarqube (First PR) (#2447)
fix #2449 main_v11.0 pdf (#2465)
fix 2428 python csv input (#2467)
JohnHalleyGotway added a commit that referenced this pull request Apr 3, 2023
Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu>
Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu>
Co-authored-by: John Halley Gotway <johnhg@ucar.edu>
Co-authored-by: jprestop <jpresto@ucar.edu>
Co-authored-by: hsoh-u <hsoh@ucar.edu>
Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com>
fix #2389 main_v11.0 flowchart (#2391)
fix definitions of G172 and G220 based on comments in NOAA-EMC/NCEPLIBS-w3emc#157. (#2405)
fix #2380 main_v11.0 override (#2381)
fix #2408 main_v11.0 empty config (#2409)
fix #2390 main_v11.0 fix compiling hdf5 with zlib and handle NetCDF-C zip (#2403)
fix #2415 main_v11.0 modulefiles (#2416)
fix #2412 main_v11.0 climo (#2420)
fix #2426 main_v11.0 buoy (#2432)
fix #2437 main_v11.0 convert (#2438)
fix for main_v11.0, for #2437, forgot one reference to the search_parent for a dictionary lookup.
fix 2428 python from env main v11.0 (#2443)
fix 2428 python csv input (#2450)
fix #2452 main_v11.0 airnow (#2453)
fix #2402 main_v11.0 sonarqube (First PR) (#2447)
fix #2449 main_v11.0 pdf (#2465)
fix 2428 python csv input (#2467)
JohnHalleyGotway pushed a commit that referenced this pull request Sep 11, 2023
Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu>
Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu>
Co-authored-by: John Halley Gotway <johnhg@ucar.edu>
Co-authored-by: jprestop <jpresto@ucar.edu>
Co-authored-by: hsoh-u <hsoh@ucar.edu>
Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com>
Co-authored-by: lisagoodrich <33230218+lisagoodrich@users.noreply.github.com>
Co-authored-by: Stephen Herbener <32968781+srherbener@users.noreply.github.com>
fix #2389 main_v11.0 flowchart (#2391)
fix definitions of G172 and G220 based on comments in NOAA-EMC/NCEPLIBS-w3emc#157. (#2405)
fix #2380 main_v11.0 override (#2381)
fix #2408 main_v11.0 empty config (#2409)
fix #2390 main_v11.0 fix compiling hdf5 with zlib and handle NetCDF-C zip (#2403)
fix #2415 main_v11.0 modulefiles (#2416)
fix #2412 main_v11.0 climo (#2420)
fix #2426 main_v11.0 buoy (#2432)
fix #2437 main_v11.0 convert (#2438)
fix for main_v11.0, for #2437, forgot one reference to the search_parent for a dictionary lookup.
fix 2428 python from env main v11.0 (#2443)
fix 2428 python csv input (#2450)
fix #2452 main_v11.0 airnow (#2453)
fix #2402 main_v11.0 sonarqube (First PR) (#2447)
fix #2449 main_v11.0 pdf (#2465)
fix 2428 python csv input (#2467)
fix #2514 main_v11.0 clang (#2556)
fix #2575 main_v11.0 python_convert (#2577)
fix 2596 main v11.0 rpath compilation (#2600)
fix 2596 main v11.0 rpath compilation (#2613)
fix #2644 main_v11.0 percentile (#2645)
JohnHalleyGotway added a commit that referenced this pull request Dec 18, 2023
…2770)

Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu>
Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu>
Co-authored-by: John Halley Gotway <johnhg@ucar.edu>
Co-authored-by: jprestop <jpresto@ucar.edu>
Co-authored-by: hsoh-u <hsoh@ucar.edu>
Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com>
Co-authored-by: lisagoodrich <33230218+lisagoodrich@users.noreply.github.com>
Co-authored-by: Stephen Herbener <32968781+srherbener@users.noreply.github.com>
fix #2389 main_v11.0 flowchart (#2391)
fix definitions of G172 and G220 based on comments in NOAA-EMC/NCEPLIBS-w3emc#157. (#2405)
fix #2380 main_v11.0 override (#2381)
fix #2408 main_v11.0 empty config (#2409)
fix #2390 main_v11.0 fix compiling hdf5 with zlib and handle NetCDF-C zip (#2403)
fix #2415 main_v11.0 modulefiles (#2416)
fix #2412 main_v11.0 climo (#2420)
fix #2426 main_v11.0 buoy (#2432)
fix #2437 main_v11.0 convert (#2438)
fix for main_v11.0, for #2437, forgot one reference to the search_parent for a dictionary lookup.
fix 2428 python from env main v11.0 (#2443)
fix 2428 python csv input (#2450)
fix #2452 main_v11.0 airnow (#2453)
fix #2402 main_v11.0 sonarqube (First PR) (#2447)
fix #2449 main_v11.0 pdf (#2465)
fix 2428 python csv input (#2467)
fix #2514 main_v11.0 clang (#2556)
fix #2575 main_v11.0 python_convert (#2577)
fix 2596 main v11.0 rpath compilation (#2600)
fix 2596 main v11.0 rpath compilation (#2613)
fix #2644 main_v11.0 percentile (#2645)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Development

Successfully merging this pull request may close these issues.

Bugfix: Fix time interpolation of monthly climatology data between December 15 and January 15
2 participants