From c0e3240af3f54370d958a1e6f9911005ceb20a4c Mon Sep 17 00:00:00 2001 From: johnhg Date: Wed, 2 Nov 2022 22:24:23 -0600 Subject: [PATCH] Updated develop-ref after #2333 (#2334) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Initialze obs_data at constructor to avoid warninjg on building * Initialze obs_data at constructor to avoid a warning by compiler * #1824 ci-run-test Removed the duplicated code * Feature 1810 expand asciitable (#2086) * Feature 1583 skip_mean (#2090) * Feature 1275 MODE Object Count (#2091) * for #1275, fix issue with more than 1000 object for mode, ci-run-unit * Per #1275, just deleting stale, commented-out code. Co-authored-by: Randy Bullock * Feature 1184 dryline (#2088) Co-authored-by: davidfillmore Co-authored-by: rgbullock Co-authored-by: John Halley Gotway * Feature 1184 replace_env (#2093) * Feature 2092 v10.1.0-rc1 (#2094) * Hotfix to the develop branch to fix the MODE Makefile for the MET-10.1.0-rc1 release. * Second hotfix for the MODE Makefile in the develop branch. * Feature 1844 docs (#2096) * Feature 2097 v10.1.0 (#2099) * Feature 2098 gen ens prod doc (#2100) * update job control rules for GHA testing workflow to use version specific input data if running tests on main_v branch * Migrating a small tweak added to the v10.1.0 release into the develop branch. * Adding SonarQube location to development.seneca file. * Bugfix #2102 develop initialize modified_hdr_typ (#2104) Co-authored-by: Howard Soh Co-authored-by: johnhg * added logic to manually trigger a workflow via the GitHub… (develop) (#2109) Co-authored-by: John Halley Gotway * Feature update installation info (#2114) * Updating configuration and modulefiles * Removed jet files * create DockerHub tag that ends with -lite to use when triggering the METplus testing workflow * Bugfix #2115 develop Rotated LatLon (#2117) * Feature 1950 sort station list (#2121) * Update the top-level README file to list ioda2nc, but also to trigger GHA to run for this new main_v10.1-ref branch. * Added bool Sorted variable, set to true or false in relevant functions. Updated sort() function. SL * In parse_sid_mask(), added the line: mask_sid.sort(). SL * Per issue #1950, put in some cout statements for testing. SL * Per issue #1950, working in has() function, checking for Sort and put in initial cout's for testing. SL * Per issue #1950: Cleaned up some cout (print) statements. Left one in there that is commented out for future testing. SL * Per issue #1950: modified has() function. Added lower_bound search for sorted arrays. Left some print (cout) statements in there for further testing. SL * Per issue #1950: In has(), I commented out the print (cout) statements for time testing. SL * added logic to manually trigger a workflow via the GitHub… (#2107) * Per issue #1950, put back in some print (cout) statements for testing. SL * Per issue #1950: in has() function, experimenting using the upper_bound() search instead of lower_bound() for sorted vectors. In progress. SL * Per issue #1950: added a new has function that uses binary_search. Re-working the other has() functions related to this. SL * Per issue #1950: re-worked original has(), basically reverted back to what it was...but it's now only used for unsorted vectors or for a case insensitive search. Cleaned up. SL * Per issue #1950: in section that checks Obs Station Id in SID masking list, changed cout to a mlog (debug) statement. SL * Per issue #1950: created new mask station id list (CONUS METARs) and modified the PointStatConfig_MASK_SID config file to use this list along with the existing lists. SL * Per issue #1950: reverted back to orginal sid mask site-lists. SL * Per issue #1950: checking in this site-list that was used for testing point_stat. SL * Per issue #1950, modified and added better debugging to figure out current issue with ascii2nc. SL * Per #1950, small tweak in ascii2nc to make it slightly more efficient. After adding the new obs varable name to the end of the list, we can compute var_index direclty without needing to call StringArray::has() a second time. * Per #1950: removed mlog and cout print statements from has() functions. Cleaned up. SL * Update met/src/basic/vx_log/string_array.cc Co-authored-by: johnhg * Update met/src/basic/vx_log/string_array.cc Co-authored-by: johnhg * Update met/src/basic/vx_log/string_array.cc Co-authored-by: johnhg * Update met/src/basic/vx_log/string_array.cc Co-authored-by: johnhg * Update met/src/basic/vx_log/string_array.cc Co-authored-by: johnhg * Update met/src/basic/vx_log/string_array.h Co-authored-by: johnhg * Delete SID_CONUS.txt * Delete SID_CONUS_times11.txt Co-authored-by: John Halley Gotway Co-authored-by: Seth Linden Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> * Bugfix #2118 develop grib1_rotll (#2130) Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> * Bugfix 2123 nccf valid time (#2127) Co-authored-by: Howard Soh Co-authored-by: johnhg * Feature #2132 version 11.0.0 (#2133) * Bugfix #2106 develop gcc (#2135) Co-authored-by: Julie Prestopnik Co-authored-by: johnhg Co-authored-by: Seth Linden Co-authored-by: John Halley Gotway Co-authored-by: j-opatz <59586397+j-opatz@users.noreply.github.com> Co-authored-by: Howard Soh Co-authored-by: John Halley Gotway Co-authored-by: jprestop Co-authored-by: Howard Soh Co-authored-by: Randy Bullock Co-authored-by: davidfillmore Co-authored-by: rgbullock Co-authored-by: Seth Linden Co-authored-by: hsoh-u Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> Co-authored-by: John Halley Gotway Co-authored-by: MET Tools Test Account Co-authored-by: mo-mglover <78152252+mo-mglover@users.noreply.github.com> Co-authored-by: davidalbo Co-authored-by: lisagoodrich <33230218+lisagoodrich@users.noreply.github.com> Co-authored-by: Dan Adriaansen * Doc-only update to expliciltly list the maximum character lengths for variable-length strings passed to ascii2nc. * Added pcp_combine -subtract example (#2153) * Add testing and docs status badges. * Feature #2152 poly_xy (#2154) * #14 Using literal instead of numbers and make sure no overflow * #14 Using literal instead of numbers * #14 Added tmp_buf_size * #14 Define n_kw_infos first * #14 Using literal instead of numbers * Bugfix #2148 develop misses (#2150) * dtcenter/METplus-Internal#14 formatting code * METplus-Internal #16 Added python scanning * METplus-Internal #16 changed name * METplus-Internal #16 initial release for python scanning * METplus-Internalv#19 added maic.cc & main.h * METplus-Internalv#19 renamed main to met_main and added get_tool_name * METplus-Internalv#19 Initial release * changing 'Linked issue' to 'Development issue' * change Linked issue/Development issue ci-skip-all * change Linked issue/Development issue ci-skip-all * change Linked issue/Development issue ci-skip-all * Feature 1920 repo (#2161) * List line types that can be aggregated per dtcenter/METplus#1625 * Hotfix for the develop branch to correct a bad MET_TEST_BASE path that causes the nightly build to fail. * Feature 2072 tc rmw winds (#2165) * bug fix for NAN and for using missing data as if its not missing * added parameters to control converting winds to tangential/radial components * bug fixes * Refactored to put wind conversion into it's own class TCRMW_WindConverter * changed default for converting winds to TRUE, and put parameters for wind conversion into the unit test config files * Per #2072, updating tc_rmw Makefile.in to compile new files. * Per #2072, update to the output from the lex/yacc version on seneca. * Changed Error to Warning in mlog messages, for consistency with MET conventions * Removing commented out code. * Per #2072, combine warnings spread across multiple lines into a single call to the logger. Replace references to ugrid/vgrid with u/v wind. Fix a typo and clean up some whitespace. Co-authored-by: Dave Albo Co-authored-by: John Halley Gotway * change Linked issue/Development issue ci-skip-all * Bugfix #2173 develop shift_data_plane (#2175) * Bugfix #2170 develop agg_ecnt (#2177) * Feature 2022 anom corr to vcnt (#2171) * Per #2022: Added ANOM_CORR, ANOM_CORR_BCL, ANOM_CORR_BCU to vcnt_columns. SL * Per #2022: Added ANOM_CORR, ANOM_CORR_BCL, ANOM_CORR_BCU to write_vcnt_cols. SL * Per #2022: Added ANOM_CORR to VL1L2Info. SL * Per #2022: updated the VL1L2Info functions to initialize and compute ANOM_CORR. SL * Per #2022: in calc_ncep_stats modified calculation for ANOM_CORR to check for bad data in the denominator first. SL * Per #2022: for the VL1L2Info, changed all derived VCNT variables to be CIInfo objects (instead of doubles). SL * Per #2022: making changes to VL1L2Info related to making all VCTN variables CIInfo objects. In progress. SL * Per #2022: Added ANOM_CORR_UNCNTR. SL * Per issue 2022: Modified functions in the VL1L2Info class to set the NCEP stat variables correctly since they are now CIInfo class memembers. SL * Per issue 2022: Added ANOM_CORR_UNCNTR (_BCL, _BCU). SL * Per issue 2022: modified write_vcnt_cols(), added ANOM_CORR_UNCNTR (_BCL, _BCU). SL * Per issue #2022: updated write_vcnt_cols. VCNT variables are now CIInfo objects to have to .v for the value. SL * Per issue #2022: In calc_ncep_stats, put in place-holder for ANOM_CORR.v value SL * Per issue #2022: in calc_ncep_stats(), changed placeholder for ANOM_CORR. SL * Per issue #2022: moved alpha initialization to the init_from_scratch() function. SL * Per issue #2022: in calc_ncep_stats(), set ANOM_CORR to bad_data_double as a placeholder for now. SL * Per issue #2022: in vcnt_cols(), added ANOM_CORR_NCL/NCU. SL * Per issue #2022: in write_vcnt_cols(), added ANOM_CORR_NCL/NCU variables. SL * Per #2022, adding new VCNT columns to the header table files, User's Guide documentation, and lining some code up more consistently. * Per #2022, added FA_SPEED_BAR and OA_SPEEED_BAR to the VAL1L2 line type. Added code to compute the ANOM_CORR confidence interval. However, I still need to update the writing of the VCNT line type to set the alpha value and actually write the computed CI's. * Per #2022, update the writing of the VCNT output line to write the alpha value and anomaly correlation CI's. * Per #2022, update point_stat and grid_stat to allocate alpha values for VL1L2Info objects to store CI's. * Per #2022, call VL1L2Info::zero_out() instead of clear() so that we don't throw away allocated alpha values. * Per #2022, update the Point-Stat and Grid-Stat logic to include the number of alpha values when deciding how many VCNT lines to write. * Per #2022, update the VL1L2Info::operator+= logic to copy over the allocated alpha values. * Per #2022, enhance stat_analysis to support an aggregate_stat job type for VAL1L2->VCNT. This writes ANOM_CORR and ANOM_CORR_UNCNTR output columns. However, there are still things to address. The other stats should be written as NA instead of 0. Recommend changing the VL1L2Info::zero_out() function to intialize stat values to NA instead of 0. Need to update the docs to reflect this support. Also need to actually compute the parametric CI's for ANOM_CORR. Big picture, we should store the VCNT stats in a new VCNTInfo class... just like the SL1L2Info/CNTInfo setup. Storing the sums AND stats in the same VL1L2Info class is unnecessarily confusing. * Per #2022, update Stat-Analysis docs to list VAL1L2 -> VCNT as a supported conversion type. * Per #2022, when computing VL1L2Info CI's, n should be set to vacount rather than the number of input pairs. * Per #2022, add a new call to Stat-Analysis to exercise the VAL1L2 to VCNT conversion. * Per #2022, ci-run-unit fix an aggregation bug in VL1L2Info::operator+=(). * Per issue #2022, added _vcnt.txt output files to check to relevant tests based on if vcnt output file is being generated via the config file. SL Co-authored-by: Seth Linden Co-authored-by: John Halley Gotway * #1815 Renamed time_as_offset to Is_offset * #1815 Added vx_data2d_dim_by_value * #1815 Added convert_value_to_offset and renamed _time_dim_offset to _cur_time_index * #1815 Added convert_value_to_offset and renamed _time_dim_offset to _cur_time_index. Support value instead of offset for time and z level * #1815 Support @ for the value insteaed of offset for slicing * #1815 Support value insteaed of offset for slicing * #1815 Deleted unused local variable * #1815 Deleted unused local variable * #1815 Added level by value with @ * #1815 Added plot_data_plane_NCCF_latlon_by_value * #1815 Added level by value with @ * Feature #2180 develop kft (#2181) * dtcenter/METplus-Internal#16 Moved from scripts/sonarqube to internal/scripts/sonarqube * #1815 Updated level * Feature 2022 stat_analysis (#2185) Proceeding with the merge without an external review based on the details listed above. * Feature #2147 HSS (#2169) * #1815 call is_eq to comapre two double values * Per #2147, this is a HOTFIX to the develop branch after PR #2169 added new CTC and CTS columns. Prevoiusly the CTC columns = NBRCTC columns and CTS = NBRCTS. But that is no longer the case. This updates write_nbrctc_cols() and write_nbrcts_cols() to actually write them rather than just calling the functions to write CTC and CTS cols, respectively. * #2187 Added add_offset_att_name & scale_factor_att_name * #2187 Apply add_offse or scale_factor if defined * ##2187 Removed handling add_offset and scale factor. They are applied at the base API * #2187 Changed a log message * Per #2193, add release notes for met-11.0.0-beta1. (#2195) Co-authored-by: MET Tools Test Account * Hotfix to develop since I forgot to update the version number in conf.py. * #1815 Added more attribute names * #1815 Added get_index_for_dim * #1815 Formatting * #1815 Support a differnt dimension name from thw first data variable * #1815 Use constants for attr names * Hotfix to the develop branch to fix paths in Dockefiles after restructuing the MET repo. * Another hotfix to develop to get the Docker build step working again. The failure from the configure script is caused by the version of the "alocal" executable. The contents of develop are based on seneca, which has: aclocal (GNU automake) 1.16.1 By comparison, my laptop does NOT require that bootstrap be run for: aclocal (GNU automake) 1.16.5 But the Docker image has an older version which triggers the error: aclocal (GNU automake) 1.13.4 Any machine with aclocal 1.16.x works fine. Likely any machine with a version that differs from 1.16.x does not. Running bootstrap fixes that... but also changes the contents of all the Makefile.in files making them appear as a large set of diffs in the repo. This is not an issue inside the docker image since we generally remove the source code after compiling anyway. * Updating the png version of the older flowcharts. * #1815 cleanup * #1815 cleanup * #1851 cleanup * #1851 cleanup * #1815 Added get_att_unixtime * #1851 cleanup * #1815 cleanup * #1815 cleanup * #1815 Support @ for pinterp * #1815 Do not call to_string * #1815 cleanup * Per issue #2081, updated thecontents to match MRMS Version 12.2. Updated existing entries, removed old entries, added new entries. SL (#2199) Co-authored-by: Seth Linden * #1815 Changed @time to time at plot_data_plane_NCCF_time * Revert "Feature 1815 level value" (#2201) * #1815 Use NcVarInfo members instead API calls * #1815 Added find_var_by_dim_name * #1815 Restored changes for this branch. The changes were reset after revert and merging with latest develop branch * #METplus-Internal_19 cleanup * #METplus-Internal_19 Added more functiofnds and corrected typo * #METplus-Internal_19 Added main.h and main.cc * #METplus-Internal_19 adjusted link order to set oom * #METplus-Internal_19 Renamed main to met_main. Moved oom handler to base API * #METplus-Internal commented out segv_handler which failed to compile at Docker * Fix typo - maintianing * Feature #2204 sum (#2205) * #2202 Added APIs to check the point data (missing data and data types) * #2202 Renamed pointdata_from_xarray to pointdata_from_python_list * #2202 Support python list * #2202 Avoid seg fault by checking point data from python embedding and provides the better log messages * #2202 Support python list * #2202 Added APIs to check the point data (missing data and data types) * #2202 Added APIs to check the point data (missing data and data types) * #2202 Added APIs to check the point data (missing data and data types) * #2202 Added MaskedArray to allowed array data type * Per #2211, enhance warning/error messages when parsing threshold types in dictionary.cc. * #2208 Check existence of ensemble member data before checking bad_data * 2215 Added station_ob to message_type & station_id to station_id for metadata_map * 2215 Add log message for missing message_type and station_id metadata variables. Set NA if a message type is empty string * 2215 Filter out XXX@RecMetaData variables from @MetaData variables * Per #2215, update the user's guide to make it consistent with the default ioda2nc config file. * #METplus-Internal_19 Renamed main to met_main and added get_tool_name * Feature sonarqube v11 beta2 (#2222) Co-authored-by: Howard Soh * Per #2220, update the version number and release notes. * Per #2220, fix typo. * Per #2220, format the METplus-Internal release notes for MET in the same way they were handled for the METplus beta1 release. * Added *hpp * Separated template functions * Separated template functions * Changed argument for get_var_fill_value * Feature 1971 mtd error to warning (#2221) Co-authored-by: Dave Albo Co-authored-by: John Halley Gotway * #2225 @level string is not applied as an index and index string without @ is not applied as value * Feature #2220 v11.0.0-beta2_try2 (#2228) * Updating Hera and Jet files for 11.0.0 release * Adding 11.0.0 modulefiles for Hera and Jet * Updating cheyenne installation files * Reformatted slashes in paths for hera config files * Updating installation files for Orion * Removed beta2 references * Adding configuration and modulefiles for gaea installation * Changing METdatadb references to METdataio * Documentation-only commit directly to develop to correct a typo. The default dotsize if 1.0 rather than 10. * Feature 2055 nbm grib2 update (#2234) * Per issue #2055, in read_grib2_record_list() added new PDS numbers 6, 10, 15 (for NBM data). SL * Per issue #2055: in read_grib2_record_list(), in section that validates PDS number changed, for undefined PDS numbers, changed Error to Warning and continue (instead of exit). SL * Per issue #2055, created new entries for NBM grib2 data fields. SL * Per issue #2055. Added more NBM fields based on a 12z, f003 file. SL * Per issue #2055, after interogating several NBM lead-time files, added TMAX field. SL * Per issue #2055, added grib2_nbm.txt to the list. SL * Per #2055, update the grib2_nbm.txt file entries in the table_files Makefiles. * Per issue #2055, fixed syntax entry for TSTM field. SL * Per issue #2055, fixed a few more syntax errors for some of the fields. SL * Per issue #2055, in read_grib2_record_list(), added section to set level values for template number 6 (NBM data). Also added some temporary print statements. SL * Per issue #2055, in read_grib2_record_list(), added code to set levels for table number 10 (like 6). SL * Per #2055, adding support for new GRIB2_perc_val configuration option to filter records based on the forecast percentile value used by GRIB2 product definition templates 6 and 10. Seth, note that I also included the probability filtering logic tweak we discussed, since it was already present in that same file. * Per issue #2055: in find_record_matches(), in section that resolves Prob fields, changed 'break' to 'continue' for non Probs. SL * Per issue #2055, in read_grib2_record_list() cleaned up some commented out code. SL * Per issue #2055, in read_grib2_record_list() cleaned up all print statements. SL * Per issue #2055, added 5 new unit tests for NBM data. SL * Update data2d_grib2.cc * Update data2d_grib2.cc Co-authored-by: Seth Linden Co-authored-by: John Halley Gotway * Feature 2207 map (#2241) * Feature #2237 tmp_dir (#2242) * Per #2196, updating files for changes to met base image (#2246) * Hotfix to the develop branch to fix typos in the names of 3 tools (grid_stat, gribtab_dat_to_flat, and gsidens2orank). Also removed commented out code from main.cc. * Feature 2250 accum (#2251) * Feature 2198 zenodo (#2252) * Per #2198, adding Seth Linden and Dave Albo to the Author List * Per #2198, updated Dave Albo to David Albo * Per #2198, added three new authors based on submitted AMS abstract * Per #2198, adding .zenodo.json file * Per #2198, adding orcids for all based on Tatiana's suggestion * Update .zenodo.json Co-authored-by: johnhg * Update .zenodo.json Co-authored-by: johnhg * Update .zenodo.json Co-authored-by: johnhg Co-authored-by: johnhg * Per #1744, adding names for rst files in index.rst * Reword to Grid-Stat opening paragraph * Feature 2196 update dockerfile (#2249) * Per #2196, update for upgrade to Python 3.8.6 * Updated based on new structure of METbaseimage * Changed MET_BASE_IMAGE references to MET_BASE_TAG; Added information to the README.md file * Changed MET_BASE_IMAGE references to MET_BASE_TAG; Updated the version from v1.0 to v1.1 * Update internal/scripts/docker/README.md Based on suggestion from John HG. Co-authored-by: johnhg * Update internal/scripts/docker/README.md Based on suggestion from John HG. Co-authored-by: johnhg * Update internal/scripts/docker/README.md Based on suggestion from John HG. Co-authored-by: johnhg * Update internal/scripts/docker/README.md Based on suggestion from John HG. Co-authored-by: johnhg * Per #2196, updating README.md based on suggestions from John HG Co-authored-by: johnhg * Feature 1774 contrib outline (#2253) * creating chapter files * putting a space in to work through the git add and commit process. * Changed formatting of one header Co-authored-by: Lisa Goodrich Co-authored-by: jprestop * Fine tune to grid-stat opening paragraph * Feature 2206 fair crps to ecnt (#2247) * Per issue #2206, for the ECNT line type, added CRPS_EMP_FAIR. SL * Per issue #2206, for ECNT line type, added CRPS_EMP_FAIR. SL * Per issue #2206, for ecnt_columns, added CRPS_EMP_FAIR. SL * Per issue #2206, in write_ecnt_cols(), added crps_emp_fair. SL * Per issue #2206, in ECNTInfo, added crps_emp_fair. SL * Per issue #2206, started stubbing code pieced to calculate crps_emp_fair. SL * Per issue #2206, added new function mean_abs_diff(). SL * Per issue #2206, in compute_pair_vals() updated the code to calculate crps_emp and crps_emp_fair. SL * Per issue #2206, added crps_emp_fair (CRPS_EMP_FAIR) to relevant code pieces. SL * Per issue #2206, added wording for CRPS_EMP_FAIR, also added it to last column of ECNT line type. SL * Per issue #2206, added info for CRPS_EMP_FAIR and also added the math equation for this. SL * Per issue #2206, update the math equation for CRPS_EMP_FAIR. SL * Per issue #2206, added new function weighted_mean_absolute_diff(). SL * Per issue #2206, updated the crps_emp_fair calculation to subtract the weighted_mean_abs_diff. SL * Per issue #2206, in aggr_orank_lines(), updated calculation for crps_emp_fair to use weighted_mean_abs_diff. SL * Per issue #2206, renamed weighted_mean_abs_diff() to wmean_abs_diff(). SL * Per issue #2206, for the crps_emp_fair calculation, changed weighted_mean_abs_diff() to wmean_abs_diff(). SL * Per issue #2206, in aggr_orank_lines(), for the crps_emp_fair calculation, changed weighted_mean_abs_diff() to wmean_abs_diff(). SL * Per issue #2206, updated the math equation for the CRPS_EMP_FAIR calculation. SL * Per issue #2206, fixed a bug in the wmean_abs_diff() function. SL * Per issue #2206, added code for crps_emp_fair. SL * Per issue #2206, in set() function, added checks for crps_emp and crps_emp_fair, for values of 0.0. SL Co-authored-by: Seth Linden * automated tests: added ability to trigger build of develop-lite docker image that it used in the METplus automated tests * to fix issues with python embedding when the path changes, update development.docker file (apparently unused or rarely previously) to include changes for Python 3.8 including setting the LDFLAG to include -rpath to prevent needing LD_LIBRARY_PATH set to run the tools. Source the env file in the build MET docker script instead of setting the env vars in the configure command * Hotfix to the develop branch to revert the python version from 3.8 back to 3.6. This is done by rolling back the base image in the MET Dockerfiles to dtcenter/met-base:v1.0. Using 3.8 caused the downstream METplus python embedding examples to fail via GHA in the METplus repo. Will write up an issue to investigate more during the MET-11.0.0-beta4 development cycle. * Tweak the build_docker_image.sh script by removing references to MET_BASE_IMAGE and MET_BASE_TAG since those variables are hard-coded in the Dockerfiles themselves and this script includes no logic to handle them. * Revert "Tweak the build_docker_image.sh script by removing references to MET_BASE_IMAGE and MET_BASE_TAG since those variables are hard-coded in the Dockerfiles themselves and this script includes no logic to handle them." This reverts commit 5efc65edbae942d978f28a141f79b648cbc1e07f. * Hotfix, found 2 more spots where I needed to change v1.1 to v1.0. We should really revisit having this setting spread across so many locations. * Bugfix 2238 link error (#2255) Co-authored-by: Howard Soh * Feature dtcenter/METplus-Internal#15 add checksum to release (#2254) Co-authored-by: jprestop Co-authored-by: John Halley Gotway * feature 2216 ioda2nc station_id (#2259) Co-authored-by: Howard Soh Co-authored-by: John Halley Gotway * Feature 1942 seeps (#2257) Co-authored-by: Howard Soh Co-authored-by: John Halley Gotway Co-authored-by: MET Tools Test Account * Feature 2142 ascii2nc airnow (#2256) Co-authored-by: Dave Albo Co-authored-by: John Halley Gotway * Feature 1954 semilatlon (#2262) Co-authored-by: j-opatz <59586397+j-opatz@users.noreply.github.com> * Removed future imports since they were used to transition from python 2 to 3. Rearranged logic to be more readible by checking error conditions and exiting if they are not met instead of putting all of the logic to run inside the if statement * Feature 1908 remove ens (#2265) * Feature 2266 v11.0.0-beta3 (#2268) * Minor hotfix for the develop branch. Running bootstrap for develop on seneca changes the order of vx_grid semilatlon slightly. We probably manually editted that file instead of running bootstrap to regenerate it. * Bugfix #2271 develop nbrctc (#2272) Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> * Feature 2282 docs (#2283) * #2270 Removed version number for SonrQube scanner pathdiff * Feature #2286 grib_lookup (#2288) * Feature 2227 namespace (#2290) * Per #2227 changes in this commit resolve all errors from removing 'using namespace std' from all header files in the src/basic directory * Attempt to resolve errors in GitHub actions from removing 'using namepsace std' from all src/basic/ libraries, plus removal from src/libcode/vx_gnomon/gnomon.h and src/libcode/vx_nc_util/nc_utils.h * Attempt to fix GHA errors with compilation after removal of 'using namespace std'; locally compiles fine * Per #2227, attempting to fix GHA compilation; Compiltion on seneca works fine * Per #2227, attempt to resolve GHA compilation issues; seneca compilation works fine * Per #2227, attempt to resolve GHA compilation issues; seneca compilation works fine * Per #2227, removing namespace from vx_series_data library * Per #2227, removing namespace std from vx_shapedata library * Per #2227, removing namespace std from vx_data2d_grib2 and vx_data2d_grib libraries * Per #2227, removing namespace std from vx_statistics library * Per #2227, removing namespace std from src/tools/tc_utils * Per #2227, removed namespace std from /src/tools/core header files * Per #2227, clean up and continued work * Per #2227, changes to remove namespace std and netCDF from header files * Per #2227, resolve errors in GitHub Actions build * Per #2227, resolve new errors in GitHub Actions build * Per #2227, resolve new errors in GitHub Actions build * Per #2227, resolve new errors in GitHub Actions build * Per #2227, realign variables * Per #2227, updating comment * Per #2227, updating comment * Per #2227, realign variables * Per #2227, realign variables * Per #2227, realign variables * Per #2227, updating comment * Update src/basic/vx_config/config.tab.cc Co-authored-by: johnhg Co-authored-by: Julie Prestopnik Co-authored-by: johnhg * Feature #2237 test_empty_config (#2291) * Feature #2297 gha (#2299) * GHA hotfix to develop Changes for #2297 included a small typo that I'm fixing directly in the develop branch. * Feature #2297 hotfix to develop A final hotfix to the develop branch to get the build_docker_trigger_metplus.yml workflow working again. I tested on feature_2297_gha branch to confirm that no double-quotes are actually needed. * Update GHA versions * Feature 2281 linker (#2296) Co-authored-by: Julie Prestopnik * Feature 2036 enhance tc pairs (#2301) Co-authored-by: Seth Linden Co-authored-by: John Halley Gotway * Feature 2276 enhance ascii2nc to read ndbc buoy data (#2294) Co-authored-by: Dave Albo Co-authored-by: MET Tools Test Account * Feature 2155 time units months and years (#2300) Co-authored-by: Howard Soh * removing release-notes out of overview and creating a seperate chapter for this information. (#2304) * Bugfix #2309 develop tcmpr (#2310) * Bugfix #2306 ascii2nc airnow hourly (#2314) Co-authored-by: Dave Albo * Feature #2058 bias_ratio (#2317) * Feature 2232 refine grid diag (#2316) Co-authored-by: Seth Linden Co-authored-by: John Halley Gotway Co-authored-by: MET Tools Test Account * Feature #392 tcdiag_line_type (#2315) * Feature #2068 ioda2nc v2.0 (#2307) Co-authored-by: Howard Soh Co-authored-by: John Halley Gotway Co-authored-by: MET Tools Test Account * Feature #2322 v11.0.0-beta4 (#2323) * Feature #2325 ECNT_MAE (#2326) * Hotfix to the develop branch to correct the column numbers in the CNT line type table for the RMSOA, ANOM_CORR_UNCNTR, and SI statistics. * Feature #2332 spread_md (#2333) * Per #2232, add SPREAD_MD to the ECNT line type from Ensemble-Stat and computed for HiRA. * Per #2332, fix compilation error. * Per #2332, update date and release notes for met-11.0.0-beta4. * Per #2332, fix compilation error. Co-authored-by: Howard Soh Co-authored-by: hsoh-u Co-authored-by: Randy Bullock Co-authored-by: davidfillmore Co-authored-by: rgbullock Co-authored-by: John Halley Gotway Co-authored-by: Dan Adriaansen Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> Co-authored-by: MET Tools Test Account Co-authored-by: jprestop Co-authored-by: Seth Linden Co-authored-by: Seth Linden Co-authored-by: Julie Prestopnik Co-authored-by: Seth Linden Co-authored-by: j-opatz <59586397+j-opatz@users.noreply.github.com> Co-authored-by: Howard Soh Co-authored-by: John Halley Gotway Co-authored-by: mo-mglover <78152252+mo-mglover@users.noreply.github.com> Co-authored-by: davidalbo Co-authored-by: lisagoodrich <33230218+lisagoodrich@users.noreply.github.com> Co-authored-by: Molly Smith Co-authored-by: Dave Albo Co-authored-by: Jonathan Vigh Co-authored-by: bikegeek <3753118+bikegeek@users.noreply.github.com> Co-authored-by: Lisa Goodrich --- data/table_files/met_header_columns_V11.0.txt | 2 +- docs/Users_Guide/appendixC.rst | 29 ++++++++++++++++--- docs/Users_Guide/ensemble-stat.rst | 19 +++++++----- docs/Users_Guide/point-stat.rst | 6 ++-- docs/Users_Guide/release-notes.rst | 3 +- docs/conf.py | 2 +- internal/test_unit/hdr/met_11_0.hdr | 2 +- src/basic/vx_util/stat_column_defs.h | 7 +++-- src/libcode/vx_stat_out/stat_columns.cc | 24 ++++++++------- src/libcode/vx_statistics/ens_stats.cc | 18 +++++++----- src/libcode/vx_statistics/ens_stats.h | 2 +- .../vx_statistics/pair_data_ensemble.cc | 24 +++++++++------ .../vx_statistics/pair_data_ensemble.h | 3 +- .../core/stat_analysis/aggr_stat_line.cc | 5 +++- .../core/stat_analysis/parse_stat_line.cc | 9 +++--- .../core/stat_analysis/parse_stat_line.h | 2 +- 16 files changed, 101 insertions(+), 56 deletions(-) diff --git a/data/table_files/met_header_columns_V11.0.txt b/data/table_files/met_header_columns_V11.0.txt index e7a896a335..2c9cf33896 100644 --- a/data/table_files/met_header_columns_V11.0.txt +++ b/data/table_files/met_header_columns_V11.0.txt @@ -19,7 +19,7 @@ V11.0 : STAT : PJC : VERSION MODEL DESC FCST_LEAD FCST_VALID_BEG FCST_VALID V11.0 : STAT : PRC : VERSION MODEL DESC FCST_LEAD FCST_VALID_BEG FCST_VALID_END OBS_LEAD OBS_VALID_BEG OBS_VALID_END FCST_VAR FCST_UNITS FCST_LEV OBS_VAR OBS_UNITS OBS_LEV OBTYPE VX_MASK INTERP_MTHD INTERP_PNTS FCST_THRESH OBS_THRESH COV_THRESH ALPHA LINE_TYPE TOTAL (N_THRESH) THRESH_[0-9]* PODY_[0-9]* POFD_[0-9]* V11.0 : STAT : PSTD : VERSION MODEL DESC FCST_LEAD FCST_VALID_BEG FCST_VALID_END OBS_LEAD OBS_VALID_BEG OBS_VALID_END FCST_VAR FCST_UNITS FCST_LEV OBS_VAR OBS_UNITS OBS_LEV OBTYPE VX_MASK INTERP_MTHD INTERP_PNTS FCST_THRESH OBS_THRESH COV_THRESH ALPHA LINE_TYPE TOTAL (N_THRESH) BASER BASER_NCL BASER_NCU RELIABILITY RESOLUTION UNCERTAINTY ROC_AUC BRIER BRIER_NCL BRIER_NCU BRIERCL BRIERCL_NCL BRIERCL_NCU BSS BSS_SMPL THRESH_[0-9]* V11.0 : STAT : ECLV : VERSION MODEL DESC FCST_LEAD FCST_VALID_BEG FCST_VALID_END OBS_LEAD OBS_VALID_BEG OBS_VALID_END FCST_VAR FCST_UNITS FCST_LEV OBS_VAR OBS_UNITS OBS_LEV OBTYPE VX_MASK INTERP_MTHD INTERP_PNTS FCST_THRESH OBS_THRESH COV_THRESH ALPHA LINE_TYPE TOTAL BASER VALUE_BASER (N_PTS) CL_[0-9]* VALUE_[0-9]* -V11.0 : STAT : ECNT : VERSION MODEL DESC FCST_LEAD FCST_VALID_BEG FCST_VALID_END OBS_LEAD OBS_VALID_BEG OBS_VALID_END FCST_VAR FCST_UNITS FCST_LEV OBS_VAR OBS_UNITS OBS_LEV OBTYPE VX_MASK INTERP_MTHD INTERP_PNTS FCST_THRESH OBS_THRESH COV_THRESH ALPHA LINE_TYPE TOTAL N_ENS CRPS CRPSS IGN ME RMSE SPREAD ME_OERR RMSE_OERR SPREAD_OERR SPREAD_PLUS_OERR CRPSCL CRPS_EMP CRPSCL_EMP CRPSS_EMP CRPS_EMP_FAIR MAE MAE_OERR BIAS_RATIO N_GE_OBS ME_GE_OBS N_LT_OBS ME_LT_OBS +V11.0 : STAT : ECNT : VERSION MODEL DESC FCST_LEAD FCST_VALID_BEG FCST_VALID_END OBS_LEAD OBS_VALID_BEG OBS_VALID_END FCST_VAR FCST_UNITS FCST_LEV OBS_VAR OBS_UNITS OBS_LEV OBTYPE VX_MASK INTERP_MTHD INTERP_PNTS FCST_THRESH OBS_THRESH COV_THRESH ALPHA LINE_TYPE TOTAL N_ENS CRPS CRPSS IGN ME RMSE SPREAD ME_OERR RMSE_OERR SPREAD_OERR SPREAD_PLUS_OERR CRPSCL CRPS_EMP CRPSCL_EMP CRPSS_EMP CRPS_EMP_FAIR SPREAD_MD MAE MAE_OERR BIAS_RATIO N_GE_OBS ME_GE_OBS N_LT_OBS ME_LT_OBS V11.0 : STAT : RPS : VERSION MODEL DESC FCST_LEAD FCST_VALID_BEG FCST_VALID_END OBS_LEAD OBS_VALID_BEG OBS_VALID_END FCST_VAR FCST_UNITS FCST_LEV OBS_VAR OBS_UNITS OBS_LEV OBTYPE VX_MASK INTERP_MTHD INTERP_PNTS FCST_THRESH OBS_THRESH COV_THRESH ALPHA LINE_TYPE TOTAL N_PROB RPS_REL RPS_RES RPS_UNC RPS RPSS RPSS_SMPL RPS_COMP V11.0 : STAT : RHIST : VERSION MODEL DESC FCST_LEAD FCST_VALID_BEG FCST_VALID_END OBS_LEAD OBS_VALID_BEG OBS_VALID_END FCST_VAR FCST_UNITS FCST_LEV OBS_VAR OBS_UNITS OBS_LEV OBTYPE VX_MASK INTERP_MTHD INTERP_PNTS FCST_THRESH OBS_THRESH COV_THRESH ALPHA LINE_TYPE TOTAL (N_RANK) RANK_[0-9]* V11.0 : STAT : PHIST : VERSION MODEL DESC FCST_LEAD FCST_VALID_BEG FCST_VALID_END OBS_LEAD OBS_VALID_BEG OBS_VALID_END FCST_VAR FCST_UNITS FCST_LEV OBS_VAR OBS_UNITS OBS_LEV OBTYPE VX_MASK INTERP_MTHD INTERP_PNTS FCST_THRESH OBS_THRESH COV_THRESH ALPHA LINE_TYPE TOTAL BIN_SIZE (N_BIN) BIN_[0-9]* diff --git a/docs/Users_Guide/appendixC.rst b/docs/Users_Guide/appendixC.rst index f40e779ebe..0ac8b09cbf 100644 --- a/docs/Users_Guide/appendixC.rst +++ b/docs/Users_Guide/appendixC.rst @@ -977,7 +977,7 @@ CRPS Called "CRPS", "CRPSCL", "CRPS_EMP", "CRPS_EMP_FAIR" and "CRPSCL_EMP" in ECNT output :numref:`table_ES_header_info_es_out_ECNT` -The continuous ranked probability score (CRPS) is the integral, over all possible thresholds, of the Brier scores (:ref:`Gneiting et al., 2004 `). In MET, the CRPS is calculated two ways: using a normal distribution fit to the ensemble forecasts (CRPS and CRPSCL), and using the empirical ensemble distribution (CRPS_EMP and CRPSCL_EMP). The empirical ensemble CRPS is also adjusted by subtracting 1/2(m) times the mean absolute difference of the ensemble members (where m is the ensemble size), this is saved as CRPS_EMP_FAIR. In some cases, use of other distributions would be better. +The continuous ranked probability score (CRPS) is the integral, over all possible thresholds, of the Brier scores (:ref:`Gneiting et al., 2004 `). In MET, the CRPS is calculated two ways: using a normal distribution fit to the ensemble forecasts (CRPS and CRPSCL), and using the empirical ensemble distribution (CRPS_EMP and CRPSCL_EMP). The empirical ensemble CRPS can be adjusted (bias corrected) by subtracting 1/(2*m) times the mean absolute difference of the ensemble members, where m is the ensemble size. This is reported as a separate statistic called CRPS_EMP_FAIR. In some cases, use of other distributions would be better. WARNING: The normal distribution is probably a good fit for temperature and pressure, and possibly a not horrible fit for winds. However, the normal approximation will not work on most precipitation forecasts and may fail for many other atmospheric variables. @@ -987,13 +987,34 @@ Closed form expressions for the CRPS are difficult to define when using data rat In this equation, the y represents the event threshold. The estimated mean and standard deviation of the ensemble forecasts ( :math:`\mu \text{ and } \sigma`) are used as the parameters of the normal distribution. The values of the normal distribution are represented by the probability density function (PDF) denoted by :math:`\Phi` and the cumulative distribution function (CDF), denoted in the above equation by :math:`\phi`. -The overall CRPS is calculated as the average of the individual measures. In equation form: :math:`\text{CRPS} = \text{average(crps) } = \frac{1}{N} \sum_i^N \text{crps}_i`. +The overall CRPS is calculated as the average of the individual measures. In equation form: + +.. math:: \text{CRPS} = \text{average(crps) } = \frac{1}{N} \sum_{i=1}^N \text{crps}_i The score can be interpreted as a continuous version of the mean absolute error (MAE). Thus, the score is negatively oriented, so smaller is better. Further, similar to MAE, bias will inflate the CRPS. Thus, bias should also be calculated and considered when judging forecast quality using CRPS. -To calculate CRPS_EMP_FAIR (bias adjusted, empirical ensemble CRPS) +To calculate crps_emp_fair (bias adjusted, empirical ensemble CRPS) for each individual observation with m ensemble members: + +.. math:: \text{crps_emp_fair}_i = \text{crps_emp}_i - \frac{1}{2*m} * \frac{1}{m*(m-1)} \sum_{i \ne j}|f_{i} - f_{j}| + +The overall CRPS_EMP_FAIR is calculated as the average of the individual measures. In equation form: + +.. math:: \text{CRPS_EMP_FAIR} = \text{average(crps_emp_fair) } = \frac{1}{N} \sum_{i=1}^N \text{crps_emp_fair}_i + +Ensemble Mean Absolute Difference +--------------------------------- + +Called "SPREAD_MD" in ECNT output :numref:`table_ES_header_info_es_out_ECNT` + +The ensemble mean absolute difference is an alternative measure of ensemble spread. It is computed for each individual observation (denoted by a lowercase spread_md) with m ensemble members: + +.. math:: \text{spread_md}_i = \frac{1}{m*(m-1)} \sum_{i \ne j}|f_{i} - f_{j}| + +The overall SPREAD_MD is calculated as the average of the individual measures. In equation form: + +.. math:: \text{SPREAD_MD} = \text{average(spread_md) } = \frac{1}{N} \sum_{i=1}^N \text{spread_md}_i -.. math:: \text{CRPS_EMP_FAIR} = \text{CRPS_EMP} - \frac{1}{2*n} * \frac{1}{n*(n-1)} \sum|f_{i} - f_{j}| +A perfect forecast would have ensemble mean absolute difference = 0. CRPS Skill Score ---------------- diff --git a/docs/Users_Guide/ensemble-stat.rst b/docs/Users_Guide/ensemble-stat.rst index de7b167edc..f8e6bfdcd8 100644 --- a/docs/Users_Guide/ensemble-stat.rst +++ b/docs/Users_Guide/ensemble-stat.rst @@ -48,7 +48,7 @@ Often, the goal of ensemble forecasting is to reproduce the distribution of obse The relative position (RELP) is a count of the number of times each ensemble member is closest to the observation. For stochastic or randomly derived ensembles, this statistic is meaningless. For specified ensemble members, however, it can assist users in determining if any ensemble member is performing consistently better or worse than the others. -The ranked probability score (RPS) is included in the Ranked Probability Score (RPS) line type. It is the mean of the Brier scores computed from ensemble probabilities derived for each probability category threshold (prob_cat_thresh) specified in the configuration file. The continuous ranked probability score (CRPS) is the average the distance between the forecast (ensemble) cumulative distribution function and the observation cumulative distribution function. It is an analog of the Brier score, but for continuous forecast and observation fields. The CRPS statistic is computed using two methods: assuming a normal distribution defined by the ensemble mean and spread (:ref:`Gneiting et al., 2004 `) and using the empirical ensemble distribution (:ref:`Hersbach, 2000 `). The CRPS statistic using the empirical ensemble distribution can be adjusted (bias corrected) by subtracting 1/2(m) times the mean absolute difference of the ensemble members (where m is the ensemble size), this is saved as a separate stastics called CRPS_EMP_FAIR. The CRPS (and CRPS FAIR) statistic is included in the Ensemble Continuous Statistics (ECNT) line type, along with other statistics quantifying the ensemble spread and ensemble mean skill. +The ranked probability score (RPS) is included in the Ranked Probability Score (RPS) line type. It is the mean of the Brier scores computed from ensemble probabilities derived for each probability category threshold (prob_cat_thresh) specified in the configuration file. The continuous ranked probability score (CRPS) is the average the distance between the forecast (ensemble) cumulative distribution function and the observation cumulative distribution function. It is an analog of the Brier score, but for continuous forecast and observation fields. The CRPS statistic is computed using two methods: assuming a normal distribution defined by the ensemble mean and spread (:ref:`Gneiting et al., 2004 `) and using the empirical ensemble distribution (:ref:`Hersbach, 2000 `). The CRPS statistic using the empirical ensemble distribution can be adjusted (bias corrected) by subtracting 1/(2*m) times the mean absolute difference of the ensemble members, where m is the ensemble size. This is reported as a separate statistic called CRPS_EMP_FAIR. The empirical CRPS and its fair version are included in the Ensemble Continuous Statistics (ECNT) line type, along with other statistics quantifying the ensemble spread and ensemble mean skill. The Ensemble-Stat tool can derive ensemble relative frequencies and verify them as probability forecasts all in the same run. Note however that these simple ensemble relative frequencies are not actually calibrated probability forecasts. If probabilistic line types are requested (output_flag), this logic is applied to each pair of fields listed in the forecast (fcst) and observation (obs) dictionaries of the configuration file. Each probability category threshold (prob_cat_thresh) listed for the forecast field is applied to the input ensemble members to derive a relative frequency forecast. The probability category threshold (prob_cat_thresh) parsed from the corresponding observation entry is applied to the (gridded or point) observations to determine whether or not the event actually occurred. The paired ensemble relative freqencies and observation events are used to populate an Nx2 probabilistic contingency table. The dimension of that table is determined by the probability PCT threshold (prob_pct_thresh) configuration file option parsed from the forecast dictionary. All probabilistic output types requested are derived from the this Nx2 table and written to the ascii output files. Note that the FCST_VAR name header column is automatically reset as "PROB({FCST_VAR}{THRESH})" where {FCST_VAR} is the current field being evaluated and {THRESH} is the threshold that was applied. @@ -622,26 +622,29 @@ The format of the STAT and ASCII output of the Ensemble-Stat tool are described - The Continuous Ranked Probability Skill Score (empirical distribution) * - 41 - CRPS_EMP_FAIR - - The Continuous Ranked Probability Skill Score (empirical distribution) adjusted by subtracting 1/2(m) times the mean absolute difference of the ensemble members (m is the ensemble size) + - The Continuous Ranked Probability Score (empirical distribution) adjusted by the mean absolute difference of the ensemble members * - 42 + - SPREAD_MD + - The pairwise Mean Absolute Difference of the unperturbed ensemble members + * - 43 - MAE - The Mean Absolute Error of the ensemble mean (unperturbed or supplied) - * - 43 + * - 44 - MAE_OERR - The Mean Absolute Error of the PERTURBED ensemble mean (e.g. with Observation Error) - * - 44 + * - 45 - BIAS_RATIO - The Bias Ratio - * - 45 + * - 46 - N_GE_OBS - The number of ensemble values greater than or equal to their observations - * - 46 + * - 47 - ME_GE_OBS - The Mean Error of the ensemble values greater than or equal to their observations - * - 47 + * - 48 - N_LT_OBS - The number of ensemble values less than their observations - * - 48 + * - 49 - ME_LT_OBS - The Mean Error of the ensemble values less than or equal to their observations diff --git a/docs/Users_Guide/point-stat.rst b/docs/Users_Guide/point-stat.rst index 9b9b9806cb..a26544ed19 100644 --- a/docs/Users_Guide/point-stat.rst +++ b/docs/Users_Guide/point-stat.rst @@ -866,13 +866,13 @@ The first set of header columns are common to all of the output files generated * - 113-115 - RMSFA, :raw-html:`
` RMSFA_BCL, :raw-html:`
` RMSFA_BCU - Root mean squared forecast anomaly (f-c) including bootstrap upper and lower confidence limits - * - 117-119 + * - 116-118 - RMSOA, :raw-html:`
` RMSOA_BCL, :raw-html:`
` RMSOA_BCU - Root mean squared observation anomaly (o-c) including bootstrap upper and lower confidence limits - * - 120-122 + * - 119-121 - ANOM_CORR_UNCNTR, :raw-html:`
` ANOM_CORR_UNCNTR_BCL, :raw-html:`
` ANOM_CORR_UNCNTR_BCU - The uncentered Anomaly Correlation excluding mean error including bootstrap upper and lower confidence limits - * - 123-125 + * - 122-124 - SI, :raw-html:`
` SI_BCL, :raw-html:`
` SI_BCU - Scatter Index including bootstrap upper and lower confidence limits diff --git a/docs/Users_Guide/release-notes.rst b/docs/Users_Guide/release-notes.rst index b3d654196c..91d48848f3 100644 --- a/docs/Users_Guide/release-notes.rst +++ b/docs/Users_Guide/release-notes.rst @@ -11,7 +11,7 @@ Important issues are listed **in bold** for emphasis. .. warning:: **Ensemble post-processing was added to Gen-Ens-Prod in version 10.1.0 and will be removed from Ensemble-Stat in version 11.0.0!** -MET Version 11.0.0-beta4 release notes (20221031) +MET Version 11.0.0-beta4 release notes (20221103) ------------------------------------------------- * Repository, build, and testing: @@ -44,6 +44,7 @@ MET Version 11.0.0-beta4 release notes (20221031) * **Enhance ASCII2NC to read NDBC buoy data** (`#2276 `_). * **Enhance IODA2NC to support IODA v2.0 format** (`#2068 `_). + * **Add the Mean Absolute Difference (SPREAD_MD) to the ECNT line type** (`#2332 `_). * **Add MAE to the ECNT line type from Ensemble-Stat and for HiRA** (`#2325 `_). * **Add new bias ratio statistic to the ECNT line type from Ensemble-Stat and for HiRA** (`#2058 `_). * **Enhance TC-Pairs consensus logic to compute the spread of the location, wind speed, and pressure** (`#2036 `_). diff --git a/docs/conf.py b/docs/conf.py index 3635edc6d6..039520f8fd 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -24,7 +24,7 @@ verinfo = version release = f'{version}' release_year = '2022' -release_date = f'{release_year}-10-31' +release_date = f'{release_year}-11-03' copyright = f'{release_year}, {author}' # -- General configuration --------------------------------------------------- diff --git a/internal/test_unit/hdr/met_11_0.hdr b/internal/test_unit/hdr/met_11_0.hdr index 2d08a8fbd5..c70e192768 100644 --- a/internal/test_unit/hdr/met_11_0.hdr +++ b/internal/test_unit/hdr/met_11_0.hdr @@ -17,7 +17,7 @@ PJC : VERSION MODEL DESC FCST_LEAD FCST_VALID_BEG FCST_VALID_END OBS_L PRC : VERSION MODEL DESC FCST_LEAD FCST_VALID_BEG FCST_VALID_END OBS_LEAD OBS_VALID_BEG OBS_VALID_END FCST_VAR FCST_UNITS FCST_LEV OBS_VAR OBS_UNITS OBS_LEV OBTYPE VX_MASK INTERP_MTHD INTERP_PNTS FCST_THRESH OBS_THRESH COV_THRESH ALPHA LINE_TYPE TOTAL N_THRESH _VAR_ PSTD : VERSION MODEL DESC FCST_LEAD FCST_VALID_BEG FCST_VALID_END OBS_LEAD OBS_VALID_BEG OBS_VALID_END FCST_VAR FCST_UNITS FCST_LEV OBS_VAR OBS_UNITS OBS_LEV OBTYPE VX_MASK INTERP_MTHD INTERP_PNTS FCST_THRESH OBS_THRESH COV_THRESH ALPHA LINE_TYPE TOTAL N_THRESH BASER BASER_NCL BASER_NCU RELIABILITY RESOLUTION UNCERTAINTY ROC_AUC BRIER BRIER_NCL BRIER_NCU BRIERCL BRIERCL_NCL BRIERCL_NCU BSS BSS_SMPL _VAR_ ECLV : VERSION MODEL DESC FCST_LEAD FCST_VALID_BEG FCST_VALID_END OBS_LEAD OBS_VALID_BEG OBS_VALID_END FCST_VAR FCST_UNITS FCST_LEV OBS_VAR OBS_UNITS OBS_LEV OBTYPE VX_MASK INTERP_MTHD INTERP_PNTS FCST_THRESH OBS_THRESH COV_THRESH ALPHA LINE_TYPE TOTAL BASE N_PTS _VAR_ -ECNT : VERSION MODEL DESC FCST_LEAD FCST_VALID_BEG FCST_VALID_END OBS_LEAD OBS_VALID_BEG OBS_VALID_END FCST_VAR FCST_UNITS FCST_LEV OBS_VAR OBS_UNITS OBS_LEV OBTYPE VX_MASK INTERP_MTHD INTERP_PNTS FCST_THRESH OBS_THRESH COV_THRESH ALPHA LINE_TYPE TOTAL N_ENS CRPS CRPSS IGN ME RMSE SPREAD ME_OERR RMSE_OERR SPREAD_OERR SPREAD_PLUS_OERR CRPSCL CRPS_EMP CRPSCL_EMP CRPSS_EMP CRPS_EMP_FAIR MAE MAE_OERR BIAS_RATIO N_GE_OBS ME_GE_OBS N_LT_OBS ME_LT_OBS +ECNT : VERSION MODEL DESC FCST_LEAD FCST_VALID_BEG FCST_VALID_END OBS_LEAD OBS_VALID_BEG OBS_VALID_END FCST_VAR FCST_UNITS FCST_LEV OBS_VAR OBS_UNITS OBS_LEV OBTYPE VX_MASK INTERP_MTHD INTERP_PNTS FCST_THRESH OBS_THRESH COV_THRESH ALPHA LINE_TYPE TOTAL N_ENS CRPS CRPSS IGN ME RMSE SPREAD ME_OERR RMSE_OERR SPREAD_OERR SPREAD_PLUS_OERR CRPSCL CRPS_EMP CRPSCL_EMP CRPSS_EMP CRPS_EMP_FAIR SPREAD_MD MAE MAE_OERR BIAS_RATIO N_GE_OBS ME_GE_OBS N_LT_OBS ME_LT_OBS RPS : VERSION MODEL DESC FCST_LEAD FCST_VALID_BEG FCST_VALID_END OBS_LEAD OBS_VALID_BEG OBS_VALID_END FCST_VAR FCST_UNITS FCST_LEV OBS_VAR OBS_UNITS OBS_LEV OBTYPE VX_MASK INTERP_MTHD INTERP_PNTS FCST_THRESH OBS_THRESH COV_THRESH ALPHA LINE_TYPE TOTAL N_PROB RPS_REL RPS_RES RPS_UNC RPS RPSS RPSS_SMPL RPS_COMP RHIST : VERSION MODEL DESC FCST_LEAD FCST_VALID_BEG FCST_VALID_END OBS_LEAD OBS_VALID_BEG OBS_VALID_END FCST_VAR FCST_UNITS FCST_LEV OBS_VAR OBS_UNITS OBS_LEV OBTYPE VX_MASK INTERP_MTHD INTERP_PNTS FCST_THRESH OBS_THRESH COV_THRESH ALPHA LINE_TYPE TOTAL CRPS IGN N_RANK CRPSS SPREAD _VAR_ PHIST : VERSION MODEL DESC FCST_LEAD FCST_VALID_BEG FCST_VALID_END OBS_LEAD OBS_VALID_BEG OBS_VALID_END FCST_VAR FCST_UNITS FCST_LEV OBS_VAR OBS_UNITS OBS_LEV OBTYPE VX_MASK INTERP_MTHD INTERP_PNTS FCST_THRESH OBS_THRESH COV_THRESH ALPHA LINE_TYPE TOTAL BIN_SIZE N_BIN _VAR_ diff --git a/src/basic/vx_util/stat_column_defs.h b/src/basic/vx_util/stat_column_defs.h index a96fa2d26f..009ec547bb 100644 --- a/src/basic/vx_util/stat_column_defs.h +++ b/src/basic/vx_util/stat_column_defs.h @@ -267,9 +267,10 @@ static const char * ecnt_columns [] = { "RMSE", "SPREAD", "ME_OERR", "RMSE_OERR", "SPREAD_OERR", "SPREAD_PLUS_OERR", "CRPSCL", "CRPS_EMP", "CRPSCL_EMP", - "CRPSS_EMP", "CRPS_EMP_FAIR", "MAE", - "MAE_OERR", "BIAS_RATIO", "N_GE_OBS", - "ME_GE_OBS", "N_LT_OBS", "ME_LT_OBS" + "CRPSS_EMP", "CRPS_EMP_FAIR", "SPREAD_MD", + "MAE", "MAE_OERR", "BIAS_RATIO", + "N_GE_OBS", "ME_GE_OBS", "N_LT_OBS", + "ME_LT_OBS" }; static const char * rps_columns [] = { diff --git a/src/libcode/vx_stat_out/stat_columns.cc b/src/libcode/vx_stat_out/stat_columns.cc index 51ad6faee6..7844205758 100644 --- a/src/libcode/vx_stat_out/stat_columns.cc +++ b/src/libcode/vx_stat_out/stat_columns.cc @@ -4225,9 +4225,10 @@ void write_ecnt_cols(const ECNTInfo &ecnt_info, // RMSE, SPREAD, ME_OERR, // RMSE_OERR, SPREAD_OERR, SPREAD_PLUS_OERR, // CRPSCL, CRPS_EMP, CRPSCL_EMP, - // CRPSS_EMP CRPS_EMP_FAIR, MAE, - // MAE_OERR, BIAS RATIO, N_GE_OBS, - // ME_GE_OBS, N_LT_OBS, ME_LT_OBS + // CRPSS_EMP CRPS_EMP_FAIR, SPREAD_MD, + // MAE, MAE_OERR, BIAS_RATIO, + // N_GE_OBS, ME_GE_OBS, N_LT_OBS, + // ME_LT_OBS // at.set_entry(r, c+0, // Total Number of Pairs ecnt_info.n_pair); @@ -4280,25 +4281,28 @@ void write_ecnt_cols(const ECNTInfo &ecnt_info, at.set_entry(r, c+16, // Empirical ensemble CRPS FAIR ecnt_info.crps_emp_fair); - at.set_entry(r, c+17, // MAE for unperturbed ensemble mean + at.set_entry(r, c+17, // Mean absolute difference measure of spread + ecnt_info.spread_md); + + at.set_entry(r, c+18, // MAE for unperturbed ensemble mean ecnt_info.mae); - at.set_entry(r, c+18, // MAE for mean of perturbed members + at.set_entry(r, c+19, // MAE for mean of perturbed members ecnt_info.mae_oerr); - at.set_entry(r, c+19, // Bias Ratio + at.set_entry(r, c+20, // Bias Ratio ecnt_info.bias_ratio); - at.set_entry(r, c+20, // Number of ensemble values >= observations + at.set_entry(r, c+21, // Number of ensemble values >= observations ecnt_info.n_ge_obs); - at.set_entry(r, c+21, // ME of ensemble values >= observations + at.set_entry(r, c+22, // ME of ensemble values >= observations ecnt_info.me_ge_obs); - at.set_entry(r, c+22, // Number of ensemble values < observations + at.set_entry(r, c+23, // Number of ensemble values < observations ecnt_info.n_lt_obs); - at.set_entry(r, c+23, // ME of ensemble values < observations + at.set_entry(r, c+24, // ME of ensemble values < observations ecnt_info.me_lt_obs); return; diff --git a/src/libcode/vx_statistics/ens_stats.cc b/src/libcode/vx_statistics/ens_stats.cc index b64341a3dc..f0d28ac9c9 100644 --- a/src/libcode/vx_statistics/ens_stats.cc +++ b/src/libcode/vx_statistics/ens_stats.cc @@ -176,13 +176,14 @@ void ECNTInfo::init_from_scratch() { void ECNTInfo::clear() { othresh.clear(); - n_ens = n_pair = 0; - crps_emp = crpscl_emp = crpss_emp = crps_emp_fair = bad_data_double; - crps_gaus = crpscl_gaus = crpss_gaus = bad_data_double; - ign = bad_data_double; - me = mae = rmse = spread = bad_data_double; - me_oerr = mae_oerr = rmse_oerr = spread_oerr = bad_data_double; - spread_plus_oerr = bad_data_double; + n_ens = n_pair = 0; + crps_emp = crpscl_emp = crpss_emp = bad_data_double; + crps_emp_fair = spread_md = bad_data_double; + crps_gaus = crpscl_gaus = crpss_gaus = bad_data_double; + ign = bad_data_double; + me = mae = rmse = spread = bad_data_double; + me_oerr = mae_oerr = rmse_oerr = spread_oerr = bad_data_double; + spread_plus_oerr = bad_data_double; n_ge_obs = n_lt_obs = 0; me_ge_obs = me_lt_obs = bias_ratio = bad_data_double; @@ -203,6 +204,7 @@ void ECNTInfo::assign(const ECNTInfo &c) { crpscl_emp = c.crpscl_emp; crpss_emp = c.crpss_emp; crps_emp_fair = c.crps_emp_fair; + spread_md = c.spread_md; crps_gaus = c.crps_gaus; crpscl_gaus = c.crpscl_gaus; @@ -246,6 +248,8 @@ void ECNTInfo::set(const PairDataEnsemble &pd) { crps_emp_fair = pd.crps_emp_fair_na.wmean(pd.wgt_na); if(is_eq(crps_emp_fair, 0.0)) crps_emp_fair = 0.0; + + spread_md = pd.spread_md_na.wmean(pd.wgt_na); crpscl_emp = pd.crpscl_emp_na.wmean(pd.wgt_na); crpss_emp = (is_bad_data(crps_emp) || diff --git a/src/libcode/vx_statistics/ens_stats.h b/src/libcode/vx_statistics/ens_stats.h index 08b2a19baa..399df564ed 100644 --- a/src/libcode/vx_statistics/ens_stats.h +++ b/src/libcode/vx_statistics/ens_stats.h @@ -76,7 +76,7 @@ class ECNTInfo { // Number of ensemble members and pairs int n_ens, n_pair; - double crps_emp, crpscl_emp, crpss_emp, crps_emp_fair; + double crps_emp, crpscl_emp, crpss_emp, crps_emp_fair, spread_md; double crps_gaus, crpscl_gaus, crpss_gaus; double ign, me, mae, rmse, spread; double me_oerr, mae_oerr, rmse_oerr, spread_oerr; diff --git a/src/libcode/vx_statistics/pair_data_ensemble.cc b/src/libcode/vx_statistics/pair_data_ensemble.cc index da1345528a..f9aa0ecf08 100644 --- a/src/libcode/vx_statistics/pair_data_ensemble.cc +++ b/src/libcode/vx_statistics/pair_data_ensemble.cc @@ -100,6 +100,7 @@ void PairDataEnsemble::clear() { crps_emp_na.clear(); crps_emp_fair_na.clear(); + spread_md_na.clear(); crpscl_emp_na.clear(); crps_gaus_na.clear(); crpscl_gaus_na.clear(); @@ -172,6 +173,7 @@ void PairDataEnsemble::extend(int n) { r_na.extend (n); crps_emp_na.extend (n); crps_emp_fair_na.extend (n); + spread_md_na.extend (n); crpscl_emp_na.extend (n); crps_gaus_na.extend (n); crpscl_gaus_na.extend (n); @@ -231,15 +233,16 @@ void PairDataEnsemble::assign(const PairDataEnsemble &pd) { cdf_na = pd.cdf_na; // PairDataEnsemble - v_na = pd.v_na; - r_na = pd.r_na; - crps_emp_na = pd.crps_emp_na; + v_na = pd.v_na; + r_na = pd.r_na; + crps_emp_na = pd.crps_emp_na; crps_emp_fair_na = pd.crps_emp_fair_na; - crpscl_emp_na = pd.crpscl_emp_na; - crps_gaus_na = pd.crps_gaus_na; - crpscl_gaus_na = pd.crpscl_gaus_na; - ign_na = pd.ign_na; - pit_na = pd.pit_na; + spread_md_na = pd.spread_md_na; + crpscl_emp_na = pd.crpscl_emp_na; + crps_gaus_na = pd.crps_gaus_na; + crpscl_gaus_na = pd.crpscl_gaus_na; + ign_na = pd.ign_na; + pit_na = pd.pit_na; n_ge_obs_na = pd.n_ge_obs_na; me_ge_obs_na = pd.me_ge_obs_na; @@ -440,6 +443,7 @@ void PairDataEnsemble::compute_pair_vals(const gsl_rng *rng_ptr) { r_na.add(bad_data_int); crps_emp_na.add(bad_data_double); crps_emp_fair_na.add(bad_data_double); + spread_md_na.add(bad_data_double); crpscl_emp_na.add(bad_data_double); crps_gaus_na.add(bad_data_double); crpscl_gaus_na.add(bad_data_double); @@ -507,6 +511,7 @@ void PairDataEnsemble::compute_pair_vals(const gsl_rng *rng_ptr) { double crps_emp = compute_crps_emp(o_na[i], cur_ens); crps_emp_na.add(crps_emp); crps_emp_fair_na.add(crps_emp - cur_ens.wmean_abs_diff()); + spread_md_na.add(cur_ens.mean_abs_diff()); crpscl_emp_na.add(compute_crps_emp(o_na[i], cur_clm)); // Ensemble mean and standard deviation @@ -859,7 +864,7 @@ PairDataEnsemble PairDataEnsemble::subset_pairs_obs_thresh(const SingleThresh &o // // Include in subset: // wgt_na, o_na, cmn_na, csd_na, v_na, r_na, - // crps_emp_na, crps_emp_fair_na, crpscl_emp_na, crps_gaus_na, crpscl_gaus_na, + // crps_emp_na, crps_emp_fair_na, spread_md_na, crpscl_emp_na, crps_gaus_na, crpscl_gaus_na, // ign_na, pit_na, n_gt_obs_na, me_gt_obs_na, n_lt_obs_na, me_lt_obs_na, // var_na, var_oerr_na, var_plus_oerr_na, // mn_na, mn_oerr_na, e_na @@ -877,6 +882,7 @@ PairDataEnsemble PairDataEnsemble::subset_pairs_obs_thresh(const SingleThresh &o pd.r_na.add(r_na[i]); pd.crps_emp_na.add(crps_emp_na[i]); pd.crps_emp_fair_na.add(crps_emp_fair_na[i]); + pd.spread_md_na.add(spread_md_na[i]); pd.crpscl_emp_na.add(crpscl_emp_na[i]); pd.crps_gaus_na.add(crps_gaus_na[i]); pd.crpscl_gaus_na.add(crpscl_gaus_na[i]); diff --git a/src/libcode/vx_statistics/pair_data_ensemble.h b/src/libcode/vx_statistics/pair_data_ensemble.h index 3c68caee7e..5c7d4bb741 100644 --- a/src/libcode/vx_statistics/pair_data_ensemble.h +++ b/src/libcode/vx_statistics/pair_data_ensemble.h @@ -79,7 +79,8 @@ class PairDataEnsemble : public PairBase { NumArray r_na; // Observation ranks [n_obs] NumArray crps_emp_na; // Empirical Continuous Ranked Probability Score [n_obs] - NumArray crps_emp_fair_na; // Empirical Continuous Ranked Probability Score [n_obs], adjusted with the mean absolute difference of the ensmble members + NumArray crps_emp_fair_na; // Fair Empirical Continuous Ranked Probability Score [n_obs] + NumArray spread_md_na; // Mean absolute difference of ensemble members [n_obs] NumArray crpscl_emp_na; // Empirical climatological CRPS [n_obs] NumArray crps_gaus_na; // Gaussian CRPS [n_obs] diff --git a/src/tools/core/stat_analysis/aggr_stat_line.cc b/src/tools/core/stat_analysis/aggr_stat_line.cc index f81edb481f..fe4a55d2ec 100644 --- a/src/tools/core/stat_analysis/aggr_stat_line.cc +++ b/src/tools/core/stat_analysis/aggr_stat_line.cc @@ -2570,7 +2570,7 @@ void aggr_ecnt_lines(LineDataFile &f, STATAnalysisJob &job, AggrENSInfo aggr; ECNTData cur; ConcatString key; - double crps_emp, crps_emp_fair, crpscl_emp, crps_gaus, crpscl_gaus, v; + double crps_emp, crps_emp_fair, spread_md, crpscl_emp, crps_gaus, crpscl_gaus, v; map::iterator it; // @@ -2630,6 +2630,7 @@ void aggr_ecnt_lines(LineDataFile &f, STATAnalysisJob &job, // m[key].ens_pd.crps_emp_na.add(cur.crps_emp); m[key].ens_pd.crps_emp_fair_na.add(cur.crps_emp_fair); + m[key].ens_pd.spread_md_na.add(cur.spread_md); m[key].ens_pd.crpscl_emp_na.add(cur.crpscl_emp); m[key].ens_pd.crps_gaus_na.add(cur.crps_gaus); m[key].ens_pd.crpscl_gaus_na.add(cur.crpscl_gaus); @@ -2687,6 +2688,7 @@ void aggr_ecnt_lines(LineDataFile &f, STATAnalysisJob &job, crps_emp = it->second.ens_pd.crps_emp_na.wmean(it->second.ens_pd.wgt_na); crps_emp_fair = it->second.ens_pd.crps_emp_fair_na.wmean(it->second.ens_pd.wgt_na); + spread_md = it->second.ens_pd.spread_md_na.wmean(it->second.ens_pd.wgt_na); crpscl_emp = it->second.ens_pd.crpscl_emp_na.wmean(it->second.ens_pd.wgt_na); crps_gaus = it->second.ens_pd.crps_gaus_na.wmean(it->second.ens_pd.wgt_na); crpscl_gaus = it->second.ens_pd.crpscl_gaus_na.wmean(it->second.ens_pd.wgt_na); @@ -3197,6 +3199,7 @@ void aggr_orank_lines(LineDataFile &f, STATAnalysisJob &job, double crps_emp = compute_crps_emp(cur.obs, cur.ens_na); m[key].ens_pd.crps_emp_na.add(crps_emp); m[key].ens_pd.crps_emp_fair_na.add(crps_emp - cur.ens_na.wmean_abs_diff()); + m[key].ens_pd.spread_md_na.add(cur.ens_na.mean_abs_diff()); m[key].ens_pd.crpscl_emp_na.add(compute_crps_emp(cur.obs, climo_vals)); // Store Gaussian CRPS stats diff --git a/src/tools/core/stat_analysis/parse_stat_line.cc b/src/tools/core/stat_analysis/parse_stat_line.cc index 921297c2a2..a5410e2a53 100644 --- a/src/tools/core/stat_analysis/parse_stat_line.cc +++ b/src/tools/core/stat_analysis/parse_stat_line.cc @@ -357,10 +357,11 @@ void parse_ecnt_line(STATLine &l, ECNTData &e_data) { e_data.total = atoi(l.get_item("TOTAL")); e_data.n_ens = atof(l.get_item("N_ENS")); - e_data.crps_emp = atof(l.get_item("CRPS_EMP")); - e_data.crps_emp_fair = atof(l.get_item("CRPS_EMP_FAIR")); - e_data.crpscl_emp = atof(l.get_item("CRPSCL_EMP")); - e_data.crpss_emp = atof(l.get_item("CRPSS_EMP")); + e_data.crps_emp = atof(l.get_item("CRPS_EMP")); + e_data.crps_emp_fair = atof(l.get_item("CRPS_EMP_FAIR")); + e_data.spread_md = atof(l.get_item("SPREAD_MD")); + e_data.crpscl_emp = atof(l.get_item("CRPSCL_EMP")); + e_data.crpss_emp = atof(l.get_item("CRPSS_EMP")); e_data.crps_gaus = atof(l.get_item("CRPS")); e_data.crpscl_gaus = atof(l.get_item("CRPSCL")); diff --git a/src/tools/core/stat_analysis/parse_stat_line.h b/src/tools/core/stat_analysis/parse_stat_line.h index 3885538b08..62718981b2 100644 --- a/src/tools/core/stat_analysis/parse_stat_line.h +++ b/src/tools/core/stat_analysis/parse_stat_line.h @@ -61,7 +61,7 @@ struct MPRData { // Ensemble continuous statistics (ECNT) data structure struct ECNTData { int total, n_ens; - double crps_emp, crps_emp_fair, crpscl_emp, crpss_emp; + double crps_emp, crps_emp_fair, spread_md, crpscl_emp, crpss_emp; double crps_gaus, crpscl_gaus, crpss_gaus; double ign, me, mae, rmse, spread; double me_oerr, mae_oerr, rmse_oerr, spread_oerr;