Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update develop-ref after #1977, #2001, #2005, #2006, #2007, and #2012. #2016

Merged
merged 54 commits into from
Jan 16, 2022

Conversation

JohnHalleyGotway
Copy link
Collaborator

Pull Request Testing

This updates develop-ref to get past changes flagged in NB20220115 as well as the met-10.1.0-beta5 release on 1/14/2022.

  • Describe testing already performed for these changes:

These differences are caused by several PR's.

1.PR's #2007 and #2012 for issue #1695:

Add 12 new Ensemble-Stat output files for NetCDF and GRIB single file inputs.

    ensemble_stat/ensemble_stat_SINGLE_FILE_GRIB_NO_CTRL_20160608_060000V_ens.nc 
    ensemble_stat/ensemble_stat_SINGLE_FILE_GRIB_NO_CTRL_20160608_060000V_orank.nc 
    ensemble_stat/ensemble_stat_SINGLE_FILE_GRIB_NO_CTRL_20160608_060000V.stat 
    ensemble_stat/ensemble_stat_SINGLE_FILE_GRIB_WITH_CTRL_20160608_060000V_ens.nc 
    ensemble_stat/ensemble_stat_SINGLE_FILE_GRIB_WITH_CTRL_20160608_060000V_orank.nc 
    ensemble_stat/ensemble_stat_SINGLE_FILE_GRIB_WITH_CTRL_20160608_060000V.stat 
    ensemble_stat/ensemble_stat_SINGLE_FILE_NC_NO_CTRL_19820701_000000V_ens.nc 
    ensemble_stat/ensemble_stat_SINGLE_FILE_NC_NO_CTRL_19820701_000000V_orank.nc 
    ensemble_stat/ensemble_stat_SINGLE_FILE_NC_NO_CTRL_19820701_000000V.stat 
    ensemble_stat/ensemble_stat_SINGLE_FILE_NC_WITH_CTRL_19820701_000000V_ens.nc 
    ensemble_stat/ensemble_stat_SINGLE_FILE_NC_WITH_CTRL_19820701_000000V_orank.nc 
    ensemble_stat/ensemble_stat_SINGLE_FILE_NC_WITH_CTRL_19820701_000000V.stat 

Part of the implementation moves the ensemble control for the first ensemble member value to the last. That changes the ORANK and RELP outputs , as expected.

file1: MET-develop-ref/test_output/ensemble_stat/ensemble_stat_MASK_SID_CTRL_20120410_120000V_orank.txt
file2: MET-develop/test_output/ensemble_stat/ensemble_stat_MASK_SID_CTRL_20120410_120000V_orank.txt
ERROR: found    41 differences in row type ORANK#49     column COL_37        - max abs: 13.25 
ERROR: found    39 differences in row type ORANK#49     column COL_38        - max abs: 10.47 
ERROR: found    44 differences in row type ORANK#49     column COL_39        - max abs: 2.07 
ERROR: found    65 differences in row type ORANK#49     column COL_40        - max abs: 2.15 
ERROR: found    45 differences in row type ORANK#49     column COL_41        - max abs: 5.35 
ERROR: failed tests for ORANK#49: num
file1: MET-develop-ref/test_output/ensemble_stat/ensemble_stat_MASK_SID_CTRL_20120410_120000V.stat
file2: MET-develop/test_output/ensemble_stat/ensemble_stat_MASK_SID_CTRL_20120410_120000V.stat
ERROR: found    41 differences in row type ORANK#49     column COL_37        - max abs: 13.25 
ERROR: found    39 differences in row type ORANK#49     column COL_38        - max abs: 10.47 
ERROR: found    44 differences in row type ORANK#49     column COL_39        - max abs: 2.07 
ERROR: found    65 differences in row type ORANK#49     column COL_40        - max abs: 2.15 
ERROR: found    45 differences in row type ORANK#49     column COL_41        - max abs: 5.35 
ERROR: found     1 differences in row type RELP#31      column COL_27        - max abs: 22.5 
ERROR: found     1 differences in row type RELP#31      column COL_28        - max abs: 5 
ERROR: found     2 differences in row type RELP#31      column COL_29        - max abs: 7 
ERROR: found     2 differences in row type RELP#31      column COL_30        - max abs: 0.6666 
ERROR: found     1 differences in row type RELP#31      column COL_31        - max abs: 23.8334 
ERROR: failed tests for ORANK#49: num
ERROR: failed tests for RELP#31: num
  1. PR Feature 1695 ensemble single file (gen_ens_prod) #2001 for issue Enhance Ensemble-Stat and Gen-Ens-Prod to read all ensemble members from a single input file #1695 adds 4 new Gen-Ens-Prod output files.
    gen_ens_prod/gen_ens_prod_SINGLE_FILE_GRIB_NO_CTRL.nc 
    gen_ens_prod/gen_ens_prod_SINGLE_FILE_GRIB_WITH_CTRL.nc 
    gen_ens_prod/gen_ens_prod_SINGLE_FILE_NC_NO_CTRL.nc 
    gen_ens_prod/gen_ens_prod_SINGLE_FILE_NC_WITH_CTRL.nc 
  1. PR Feature 1926 gridstat openmp #1977 for issue Enhance Grid-Stat to use OpenMP for efficient computation of neighborhood statistics by setting $OMP_NUM_THREADS #1926 adds 12 new Grid-Stat output files.
    grid_stat/grid_stat_GRIB2_NAM_RTMA_NP2_120000L_20120409_120000V_cnt.txt 
    grid_stat/grid_stat_GRIB2_NAM_RTMA_NP2_120000L_20120409_120000V_ctc.txt 
    grid_stat/grid_stat_GRIB2_NAM_RTMA_NP2_120000L_20120409_120000V_cts.txt 
    grid_stat/grid_stat_GRIB2_NAM_RTMA_NP2_120000L_20120409_120000V_eclv.txt 
    grid_stat/grid_stat_GRIB2_NAM_RTMA_NP2_120000L_20120409_120000V_fho.txt 
    grid_stat/grid_stat_GRIB2_NAM_RTMA_NP2_120000L_20120409_120000V_nbrcnt.txt 
    grid_stat/grid_stat_GRIB2_NAM_RTMA_NP2_120000L_20120409_120000V_nbrctc.txt 
    grid_stat/grid_stat_GRIB2_NAM_RTMA_NP2_120000L_20120409_120000V_nbrcts.txt 
    grid_stat/grid_stat_GRIB2_NAM_RTMA_NP2_120000L_20120409_120000V_pairs.nc 
    grid_stat/grid_stat_GRIB2_NAM_RTMA_NP2_120000L_20120409_120000V_sl1l2.txt 
    grid_stat/grid_stat_GRIB2_NAM_RTMA_NP2_120000L_20120409_120000V.stat 
    grid_stat/grid_stat_GRIB2_NAM_RTMA_NP2_120000L_20120409_120000V_vcnt.txt 
    grid_stat/grid_stat_GRIB2_NAM_RTMA_NP2_120000L_20120409_120000V_vl1l2.txt 

This huge set of changes is caused by a one line change to a Grid-Stat config file. Replace "method = min; width = 1;" with "method = min; width = 3;".

file1: MET-develop-ref/test_output/grid_stat/grid_stat_GRIB2_NAM_RTMA_120000L_20120409_120000V_cnt.txt
file2: MET-develop/test_output/grid_stat/grid_stat_GRIB2_NAM_RTMA_120000L_20120409_120000V_cnt.txt
ERROR: header information mismatch in column INTERP_MTHD
ERROR: failed tests for CNT: hdr
file1: MET-develop-ref/test_output/grid_stat/grid_stat_GRIB2_NAM_RTMA_120000L_20120409_120000V_ctc.txt
file2: MET-develop/test_output/grid_stat/grid_stat_GRIB2_NAM_RTMA_120000L_20120409_120000V_ctc.txt
ERROR: header information mismatch in column INTERP_MTHD
ERROR: failed tests for CTC: hdr
file1: MET-develop-ref/test_output/grid_stat/grid_stat_GRIB2_NAM_RTMA_120000L_20120409_120000V_cts.txt
file2: MET-develop/test_output/grid_stat/grid_stat_GRIB2_NAM_RTMA_120000L_20120409_120000V_cts.txt
ERROR: header information mismatch in column INTERP_MTHD
ERROR: failed tests for CTS: hdr
file1: MET-develop-ref/test_output/grid_stat/grid_stat_GRIB2_NAM_RTMA_120000L_20120409_120000V_eclv.txt
file2: MET-develop/test_output/grid_stat/grid_stat_GRIB2_NAM_RTMA_120000L_20120409_120000V_eclv.txt
ERROR: header information mismatch in column INTERP_MTHD
ERROR: failed tests for ECLV#66: hdr
file1: MET-develop-ref/test_output/grid_stat/grid_stat_GRIB2_NAM_RTMA_120000L_20120409_120000V_fho.txt
file2: MET-develop/test_output/grid_stat/grid_stat_GRIB2_NAM_RTMA_120000L_20120409_120000V_fho.txt
ERROR: header information mismatch in column INTERP_MTHD
ERROR: failed tests for FHO: hdr
--
file1: MET-develop-ref/test_output/grid_stat/grid_stat_GRIB2_NAM_RTMA_120000L_20120409_120000V_pairs.nc
file2: MET-develop/test_output/grid_stat/grid_stat_GRIB2_NAM_RTMA_120000L_20120409_120000V_pairs.nc
ERROR: NetCDF headers differ:
file1: MET-develop-ref/test_output/grid_stat/grid_stat_GRIB2_NAM_RTMA_120000L_20120409_120000V_sl1l2.txt
file2: MET-develop/test_output/grid_stat/grid_stat_GRIB2_NAM_RTMA_120000L_20120409_120000V_sl1l2.txt
ERROR: header information mismatch in column INTERP_MTHD
ERROR: failed tests for SL1L2: hdr
file1: MET-develop-ref/test_output/grid_stat/grid_stat_GRIB2_NAM_RTMA_120000L_20120409_120000V.stat
file2: MET-develop/test_output/grid_stat/grid_stat_GRIB2_NAM_RTMA_120000L_20120409_120000V.stat
ERROR: header information mismatch in column INTERP_MTHD
ERROR: header information mismatch in column INTERP_MTHD
ERROR: header information mismatch in column INTERP_MTHD
ERROR: header information mismatch in column INTERP_MTHD
ERROR: header information mismatch in column INTERP_MTHD
ERROR: header information mismatch in column INTERP_MTHD
ERROR: header information mismatch in column INTERP_MTHD
ERROR: header information mismatch in column INTERP_MTHD
ERROR: failed tests for CNT: hdr
ERROR: failed tests for CTC: hdr
ERROR: failed tests for CTS: hdr
ERROR: failed tests for ECLV#66: hdr
ERROR: failed tests for FHO: hdr
ERROR: failed tests for SL1L2: hdr
ERROR: failed tests for VCNT: hdr
ERROR: failed tests for VL1L2: hdr
file1: MET-develop-ref/test_output/grid_stat/grid_stat_GRIB2_NAM_RTMA_120000L_20120409_120000V_vcnt.txt
file2: MET-develop/test_output/grid_stat/grid_stat_GRIB2_NAM_RTMA_120000L_20120409_120000V_vcnt.txt
ERROR: header information mismatch in column INTERP_MTHD
ERROR: failed tests for VCNT: hdr
file1: MET-develop-ref/test_output/grid_stat/grid_stat_GRIB2_NAM_RTMA_120000L_20120409_120000V_vl1l2.txt
file2: MET-develop/test_output/grid_stat/grid_stat_GRIB2_NAM_RTMA_120000L_20120409_120000V_vl1l2.txt
ERROR: header information mismatch in column INTERP_MTHD
ERROR: failed tests for VL1L2: hdr
  1. PR Feature 1965 ioda2nc same input #2006 for issue Fix IODA2NC to handle the same input file being provided multiple times #1965 adds 1 new ioda2nc output file.
    ioda2nc/ioda.NC001007.2020031012.same_input.nc 
  1. PR Feature 1810 shapes #2005 for issue Enhance TC-Gen to verify NHC tropical weather outlook shapefiles. Refine logic to prevent rounding shapefile points to the nearest grid point. #1810:

Adds 2 new tc_gen output files for #2005.

    tc_gen/tc_gen_2021_shape_pct.txt 
    tc_gen/tc_gen_2021_shape.stat 

Fix bug in TC-Gen in not correctly writing PJC and PRC output lines

file1: MET-develop-ref/test_output/tc_gen/tc_gen_prob_pjc.txt
file2: MET-develop/test_output/tc_gen/tc_gen_prob_pjc.txt
ERROR: line type PCT#39 not found in stat2
ERROR: line type PJC#55 not found in stat1
ERROR: failed tests for PCT#39: lty
ERROR: failed tests for PJC#55: lty
file1: MET-develop-ref/test_output/tc_gen/tc_gen_prob_prc.txt
file2: MET-develop/test_output/tc_gen/tc_gen_prob_prc.txt
ERROR: line type PCT#39 not found in stat2
ERROR: line type PRC#39 not found in stat1
ERROR: failed tests for PCT#39: lty
ERROR: failed tests for PRC#39: lty
--
file1: MET-develop-ref/test_output/tc_gen/tc_gen_prob.stat
file2: MET-develop/test_output/tc_gen/tc_gen_prob.stat
ERROR: line type PJC#55 not found in stat1
ERROR: line type PRC#39 not found in stat1
ERROR: differing number of rows 9 vs. 3 for row type PCT#39 between versions 10_1 vs. 10_1 
ERROR: failed tests for PJC#55: lty
ERROR: failed tests for PRC#39: lty
ERROR: failed tests for PCT#39: nrow

Part of the implementation is no longer rounding shapefile (lat, lon) points to the nearest grid point. That changes the shapefile output from gen_vx_mask as well.

file1: MET-develop-ref/test_output/gen_vx_mask/SHAPE_mask.nc
file2: MET-develop/test_output/gen_vx_mask/SHAPE_mask.nc
ERROR: found    21 differences in var shape_mask                             - max abs: 1 

These changes are all expected.

  • Recommend testing for the reviewer(s) to perform, including the location of input datasets, and any additional instructions:

    None

  • Do these changes include sufficient documentation updates, ensuring that no errors or warnings exist in the build of the documentation? [Yes]

  • Do these changes include sufficient testing updates? [Yes]

  • Will this PR result in changes to the test suite? [Yes]

    If yes, describe the new output and/or changes to the existing output:

    Update reference version, as expected.

  • Please complete this pull request review by [Fill in date].

Pull Request Checklist

See the METplus Workflow for details.

  • Complete the PR definition above.
  • Ensure the PR title matches the feature or bugfix branch name.
  • Define the PR metadata, as permissions allow.
    Select: Reviewer(s)
    Select: Organization level software support Project or Repository level development cycle Project
    Select: Milestone as the version that will include these changes
  • After submitting the PR, select Linked issues with the original issue number.
  • After the PR is approved, merge your changes. If permissions do not allow this, request that the reviewer do the merge.
  • Close the linked issue and delete your feature or bugfix branch from GitHub.

jprestop and others added 30 commits October 26, 2021 13:08
…tead of only the filename so that the function can tell whether or not the file exists. (#1952)

Co-authored-by: Julie Prestopnik <jpresto@seneca.rap.ucar.edu>
* Per issue #1761 in set_perc() adding code to get FBIAS numeric value, like 1.0 or 0.9, etc. SL

* Per issue #1761: in set_perc(), modified actual percentile calculation at end to use the extracted FBIAS numeric value (float). SL

* Per issue #1761: modified the check on the perc_thresh_freq_bias, just has to be > 0 now. SL

* Per issue #1761: cleaned up code in set_perc(). SL

* Per #1761, updates to Simple_Node::set_perc() to handle variable frequency bias amounts.
Changes include:
- Reverting the formatting of this back to how it originally was in the develop branch. In general, just match the formatting of the existing file, so as the minimize the number of difference lines.
- Add logic to adjust the percentile to be found based on the requested FBIAS value. Multiplying or dividing the percentile by the FBIAS value depends on the inequality type and whether we're bias adjusting the forecast or observation data.
- Adjust the log messages slightly.

Please be aware that I'm not totally confident in these changes. They warrant much more testing. This logic is very, very confusing.

* Per #1761, call compute_percentile() when double-checking the percentile values.

* Per #1761, remove unused variable.

* Per #1761, add warning for percentiles > 100.

* Per #1761. In set_perc(), after testing cleaned up code. SL

* Per issue #1761: adding new config file for testing dynamic FBIAS values. SL

* Per issue #1761: added new unit test for dynamic FBIAS values when running grid_stat. SL

* Per issue #1761, modified FBIAS section to indicated that the user can use dynamic values that are not 1.0. SL

* Update met/docs/Users_Guide/config_options.rst

Co-authored-by: johnhg <johnhg@ucar.edu>

* Update met/docs/Users_Guide/config_options.rst

Co-authored-by: johnhg <johnhg@ucar.edu>

* Update met/docs/Users_Guide/config_options.rst

Co-authored-by: johnhg <johnhg@ucar.edu>

* Update test/config/GridStatConfig_fbias_perc_thresh

Co-authored-by: johnhg <johnhg@ucar.edu>

* Update test/config/GridStatConfig_fbias_perc_thresh

Co-authored-by: johnhg <johnhg@ucar.edu>

* Update test/config/GridStatConfig_fbias_perc_thresh

Co-authored-by: johnhg <johnhg@ucar.edu>

* Update test/config/GridStatConfig_fbias_perc_thresh

Co-authored-by: johnhg <johnhg@ucar.edu>

* Per issue #1761, set nc_pairs_flag = FALSE. SL

Co-authored-by: Seth Linden <linden@kiowa.rap.ucar.edu>
Co-authored-by: John Halley Gotway <johnhg@ucar.edu>
Co-authored-by: j-opatz <59586397+j-opatz@users.noreply.github.com>
…ing the logic for computing the ensemble range back to what it was previously. The new version produced very slight differences in the 6-th or 7-th decimal place when compared to previous results. There's not good reason for these changes which were caused by the order of operations in casting from doubles to floats. Reverting back to the old logic prevents diffs for anyone else downstream and is the prudent choice.
* #1949 Added CF compliant NetCDF into data IO

* #1949 Added commas

* #1948 Some corrections for typo and added the links for CF attributes

* #1948 Added Performance with NetCDF input data

* #1949 Corrected tyoe and applied Juloie's suggestions

Co-authored-by: Howard Soh <hsoh@kiowa.rap.ucar.edu>
…OSS. The intel compiler does not allow adjacent >> characters when defining maps.
Howard Soh and others added 24 commits January 11, 2022 12:26
…in and /usr/local/netcdf/bin to the path. Also define MET_TEST_RSCRIPT to point to a new enough version of RSCRIPT that has the ncdf4 package 1.17 or later.
…on seneca. The problem here is that the ncdiff -x -v command line options that worked in version 4.7.0 NO LONGER work in 4.9.2. Added a check to skip over the time_bounds variable when processing the ncdiff output.
Co-authored-by: Howard Soh <hsoh@kiowa.rap.ucar.edu>
Co-authored-by: John Halley Gotway <johnhg@ucar.edu>
Co-authored-by: Howard Soh <hsoh@kiowa.rap.ucar.edu>
Co-authored-by: John Halley Gotway <johnhg@ucar.edu>
Co-authored-by: johnhg <johnhg@ucar.edu>
Co-authored-by: Howard Soh <hsoh@kiowa.rap.ucar.edu>
…and 4.9.2 of ncdiff. The earlier version reported 0 diffs for the time_bounds, lat, lon, latitude, and longitude variables while the newer version reports their raw values. Simply ignore these variables for now. This isn't a great solution but when we reimplement this testing logic in python, we can address this issue then.
Co-authored-by: johnhg <johnhg@ucar.edu>
Co-authored-by: John Halley Gotway <johnhg@ucar.edu>
Co-authored-by: Julie Prestopnik <jpresto@seneca.rap.ucar.edu>
Co-authored-by: johnhg <johnhg@ucar.edu>
…ces of MET_BUILD_BASE and a couple instances of MET_BASE from the examples listed in Appendix A. The inconsistent use of these variables will only cause confusion. Removing them will help avoid that confusion. Big picture... MET_BASE defines the installed 'share/met' directory. It can be used to reference the location of map or polyline files. MET_BUILD_BASE is only used in the context of the test scripts. There, it's defined as the top-level source code directory in which the code was compiled. Its used to reference the location of sample data files or Rscript included in the tarball. I left a couple references to these variables in Appendix A where thier use is not confusing. But I removed all instances which are used to the define the location of the 'bin' directory. Instead, let's just assume the MET binaries are already in their path... or the user knows where to find them.
Co-authored-by: John Halley Gotway <johnhg@ucar.edu>
Co-authored-by: John Halley Gotway <johnhg@kiowa.rap.ucar.edu>
@JohnHalleyGotway JohnHalleyGotway added this to the MET 10.1.0 milestone Jan 16, 2022
@JohnHalleyGotway JohnHalleyGotway changed the title Update develop-ref after ? Update develop-ref after #1977, #2001, #2005, #2006, #2007, and #2012. Jan 16, 2022
@JohnHalleyGotway JohnHalleyGotway merged commit 90bea06 into develop-ref Jan 16, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants