Feature 1746 wavelet stat #1851
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
added 3 commits
July 8, 2021 11:05
… (or NA) for forecast and observation thresholds in order to skip applying the threhsolds, but it will still compute stats with the raw fields. SL
… empty lists for the forecast and observation thresholds. SL
…orecast and observation cat thresholds to an empty list in order to skip the binary masking (and consider all grid-points for stats). SL
21 tasks
…r the forecast and observation cat thresholds. SL
…the postscript output to be [-n,n] where n is the maximum value of the maximum absolute difference and 1.0, and also fix a bug. When the NA threshold comes AFTER a real threshold, the resulting data and difference values were not being updated.
…_plot dictionary by setting plot_min = plot_max = 0.0. That enables the default logic of the tool to take effect. Choose the plotting range of the wavelet plots as [-n,n], where n is the maximum of 1.0 and the maximum absolute difference.
|
When thresholds are applied, the difference fields in the PostScript output are plotted using a [-1, 1] range. When thresholds are NOT applied, the difference fields are plotted using [-n, n] where n is the maximum absolute value of the raw difference field. This is a good default value since it'll always work. However, it's a bad default since most of the differences are washed out and down closer to 0. Difference field plotted using the old default [-1, 1]: Difference fields plotted using the new [-104, 104]: So what's the best way to handle the default plotting range for difference fields? |
added 2 commits
July 12, 2021 15:09
… skip applying the categorical threhsolds by putting an empty list or NA in the configuration file. SL
…to NA but the observation threshold is not NA (a numeric threshold) and vice versa. SL
TaraJensen
approved these changes
Jul 15, 2021
JohnHalleyGotway
approved these changes
Jul 15, 2021
…consistent use of the NA threshold type.
… n() to make the code more concise.
…h to true inside the loop since the NA threshold can appear anywhere in the list of thresholds.
12 tasks
JohnHalleyGotway
added a commit
that referenced
this pull request
Jul 15, 2021
* Update install_met_env.kiowa Commented out items and added text to description * #1817 Added to_north argument at two_to_one() * #1817 Calls two_to_one() with false for the third argument, to_north, if latitude moves north to south * Feature 1819 automation doc warnings (#1836) * Removing "/lib" that should not be there. * #1508 checking stat() methoid returns no error before looking at FILE flag * #1838 Added log message for lat/lon values * #1838 Give warning if the first and lat lat/lon is same * #1838 Support the double data type variable on reading float type values * Feature 1833 develop discussions (#1849) * Per #1833, changed references to met_help to Discussions. * Per #1833, fixed typo and removed an unnecessary word * Feature 1746 wavelet stat (#1851) * For issue #1746 modified code to allow users to pass in an empty list (or NA) for forecast and observation thresholds in order to skip applying the threhsolds, but it will still compute stats with the raw fields. SL * For issue #1746, added new unit test that uses a config file that has empty lists for the forecast and observation thresholds. SL * For issue #1746 Added some content related to allowing users to set forecast and observation cat thresholds to an empty list in order to skip the binary masking (and consider all grid-points for stats). SL * Per #1746, cleaning up for consistent indentation. * Per #1746, cleaning up for consistent indentation. * Per #1746, add a revision history note, update the plotting range in the postscript output to be [-n,n] where n is the maximum value of the maximum absolute difference and 1.0, and also fix a bug. When the NA threshold comes AFTER a real threshold, the resulting data and difference values were not being updated. * Per #1746, change the Wavelet-Stat config file values in the the wvlt_plot dictionary by setting plot_min = plot_max = 0.0. That enables the default logic of the tool to take effect. Choose the plotting range of the wavelet plots as [-n,n], where n is the maximum of 1.0 and the maximum absolute difference. * Per #1746, used apply_fcst_thresh where it should have been apply_obs_thresh. * Per issue #1746, modified some content related to users being able to skip applying the categorical threhsolds by putting an empty list or NA in the configuration file. SL * Per issue #1746 Added some warnings if the forecast threshold is set to NA but the observation threshold is not NA (a numeric threshold) and vice versa. SL * Per #1746, fix a couple of typos and tweak wording in the wavelet-stat chapter. * Per #1746, loop over each pair of fcst/obs thresholds to check for inconsistent use of the NA threshold type. * Per #1746, a bit of code cleanup replacing calls to n_elements() with n() to make the code more concise. * Per #1746, need to reinitialize apply_fcst_thresh and apply_obs_thresh to true inside the loop since the NA threshold can appear anywhere in the list of thresholds. Co-authored-by: Seth Linden <linden@kiowa.rap.ucar.edu> Co-authored-by: John Halley Gotway <johnhg@ucar.edu> Co-authored-by: jprestop <jpresto@ucar.edu> Co-authored-by: Howard Soh <hsoh@kiowa.rap.ucar.edu> Co-authored-by: hsoh-u <hsoh@ucar.edu> Co-authored-by: Seth Linden <linden@ucar.edu> Co-authored-by: Seth Linden <linden@kiowa.rap.ucar.edu>
JohnHalleyGotway
added a commit
that referenced
this pull request
Aug 2, 2021
* Update install_met_env.kiowa Commented out items and added text to description * #1817 Added to_north argument at two_to_one() * #1817 Calls two_to_one() with false for the third argument, to_north, if latitude moves north to south * Feature 1819 automation doc warnings (#1836) * Removing "/lib" that should not be there. * #1508 checking stat() methoid returns no error before looking at FILE flag * #1838 Added log message for lat/lon values * #1838 Give warning if the first and lat lat/lon is same * #1838 Support the double data type variable on reading float type values * Feature 1833 develop discussions (#1849) * Per #1833, changed references to met_help to Discussions. * Per #1833, fixed typo and removed an unnecessary word * #1852 Removed unused nc_buf_size. Make cur and dim to the same size (called API handles 2D or 3D) * #1852 Check the start and count before calling NetCDF API * Feature 1746 wavelet stat (#1851) * For issue #1746 modified code to allow users to pass in an empty list (or NA) for forecast and observation thresholds in order to skip applying the threhsolds, but it will still compute stats with the raw fields. SL * For issue #1746, added new unit test that uses a config file that has empty lists for the forecast and observation thresholds. SL * For issue #1746 Added some content related to allowing users to set forecast and observation cat thresholds to an empty list in order to skip the binary masking (and consider all grid-points for stats). SL * Per #1746, cleaning up for consistent indentation. * Per #1746, cleaning up for consistent indentation. * Per #1746, add a revision history note, update the plotting range in the postscript output to be [-n,n] where n is the maximum value of the maximum absolute difference and 1.0, and also fix a bug. When the NA threshold comes AFTER a real threshold, the resulting data and difference values were not being updated. * Per #1746, change the Wavelet-Stat config file values in the the wvlt_plot dictionary by setting plot_min = plot_max = 0.0. That enables the default logic of the tool to take effect. Choose the plotting range of the wavelet plots as [-n,n], where n is the maximum of 1.0 and the maximum absolute difference. * Per #1746, used apply_fcst_thresh where it should have been apply_obs_thresh. * Per issue #1746, modified some content related to users being able to skip applying the categorical threhsolds by putting an empty list or NA in the configuration file. SL * Per issue #1746 Added some warnings if the forecast threshold is set to NA but the observation threshold is not NA (a numeric threshold) and vice versa. SL * Per #1746, fix a couple of typos and tweak wording in the wavelet-stat chapter. * Per #1746, loop over each pair of fcst/obs thresholds to check for inconsistent use of the NA threshold type. * Per #1746, a bit of code cleanup replacing calls to n_elements() with n() to make the code more concise. * Per #1746, need to reinitialize apply_fcst_thresh and apply_obs_thresh to true inside the loop since the NA threshold can appear anywhere in the list of thresholds. Co-authored-by: Seth Linden <linden@kiowa.rap.ucar.edu> Co-authored-by: John Halley Gotway <johnhg@ucar.edu> * Update README.md * #1852 get_string_val: check if 2D variable. Also checking if the variable does not have dimension (#1860) Co-authored-by: Howard Soh <hsoh@kiowa.rap.ucar.edu> * Feature 1453 create pdf user guide (#1859) * Enable PDF creation * Modify inputenc, fontenc latex_elements * Fix typo in conf.py * Replace unicode with ascii * Replace unicode exponent with math mode * Change eqnarray to align * Replace gif images with png for PDF * Fix png file names * Escape underscore in variable name in math mode * Escape underscore in variable name in math mode * Remove underscores from variables in math mode * Replace bar with overline in some equations * Test derefencing underscores in tables * Test math changes in grid stat * Test math formatting * Test math formatting * Add sphinx.ext.pngmath to conf.py * Fix typo in conf.py * Add sphinx.ext.mathjax to conf.py * Add sphinx.ext.imgmath to conf.py * Fix typo in conf.py * Change math formatting in Appendix C * Temporarily disable equations with errors in Appendix C * Remove sphinx.ext.imgmath in conf.py * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Add sphinx.ext.imgmath to conf.py * Modify math formatting in Appendix C * Temporarily disable equations with errors in Appendix C * Remove sphinx.ext.imgmath in conf.py * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Temporarily disable equations with errors * Modify math formatting in Appendix C * Add usepackage{amssymb} to conf.py * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Fix warnings in wavelet-stat.rst * Omit Indices and tables in PDF version of User Guide * Modify math formatting in Appendix G * Modify math formatting in Appendix E * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Change release date in conf.py Co-authored-by: John Halley Gotway <johnhg@ucar.edu> * Feature 1853 rps doc (#1861) * Per #1853, adding RPS docs from Eric. * Per #1853, work in progress getting the formatting correct. * Per #1853, work in progress getting the formatting correct. * Per #1853, work in progress getting the formatting correct. * Per #1853, work in progress getting the formatting correct. * Per #1853, adding a manual page break after the ROC image to get it to stay in the right spot. * Per #1853, backing out last change which didn't make the ROC image stay in the right spot. * Add definition for the compilation environment on seneca. * Feature 1843 scatter index (#1863) * Added scatter index variables: SI, SI_BCL, SI_BCU to cnt columns. SL * Per issue #1843 Added Scatter Index variables (SI, SI_BCL, SI_BCU) to the CNT stat type. SL * Added Scatter Index (SI, SI_BCL, SI_BCU) to cnt stat type header. SL * Per issue #1843, added Scatter Index variables to write_cnt_cols(). SL * Per issue #1843 added Scatter Index (si) to CNTinfo class. SL * Per issue #1843 added Scatter Index (SI, si) to CNTInfo::get_stat(). SL * Per issue #1843 added Scatter Index (SI, si) calculation to both versions of compute_cntinfo(). SL * Per issue #1843, forgot semi-colon after SI calculation, fixed. SL * Added Scatter Index (si) to some of the clear() and allocate() functions. SL * Per issue #1843, added Scatter Index (si) to compute_cnt_mean(). SL * Per issue #1843, added Scatter Index (si) to bootstrap interval calculation and to the CNT write function. SL * Per issue #1843 added Scatter Index (SI, SI_BCL, SI_BCU) to store_stat_cnt(). SL * Per issue #1843, in get_stat() moved Scatterd Index (SI) line to be below RMSE for consitency. SL * Added Scatter Index (SI) to the CNT output format table. SL * Added Scatter Index (SI) to CNT statistics definitions. SL * Update appendixC.rst * Update stat_columns.cc * Update compute_ci.cc * Update met_stats.cc * Per #1843, update write_cntinfo() function. Since we're READING the SI value immediately after the RMSE value when computing CI's, we also need to write SI immediately after RMSE. Otherwise, we'll be computing CI's using the wrong statistics replicates. * Per #1843, check for divide by zero when computing SI... this could happen if comparing the same input file to itself with Grid-Stat. * Per #1843, correct definition of SI from RMSE/ME to RMSE/OBAR. * Per #1843, updated the SI definition to divide by OBAR instead of ME, but forgot to update the divide-by-zero check. Co-authored-by: Seth Linden <linden@kiowa.rap.ucar.edu> Co-authored-by: johnhg <johnhg@ucar.edu> Co-authored-by: jprestop <jpresto@ucar.edu> Co-authored-by: Howard Soh <hsoh@kiowa.rap.ucar.edu> Co-authored-by: hsoh-u <hsoh@ucar.edu> Co-authored-by: Seth Linden <linden@ucar.edu> Co-authored-by: Seth Linden <linden@kiowa.rap.ucar.edu> Co-authored-by: Keith Searight <keith.searight@noaa.gov> Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu>
JohnHalleyGotway
added a commit
that referenced
this pull request
Aug 18, 2021
* Update install_met_env.kiowa Commented out items and added text to description * #1817 Added to_north argument at two_to_one() * #1817 Calls two_to_one() with false for the third argument, to_north, if latitude moves north to south * Feature 1819 automation doc warnings (#1836) * Removing "/lib" that should not be there. * #1508 checking stat() methoid returns no error before looking at FILE flag * Enable PDF creation * Modify inputenc, fontenc latex_elements * Fix typo in conf.py * Replace unicode with ascii * Replace unicode exponent with math mode * Change eqnarray to align * Replace gif images with png for PDF * Fix png file names * Escape underscore in variable name in math mode * Escape underscore in variable name in math mode * Remove underscores from variables in math mode * Replace bar with overline in some equations * Test derefencing underscores in tables * Test math changes in grid stat * Test math formatting * Test math formatting * #1838 Added log message for lat/lon values * #1838 Give warning if the first and lat lat/lon is same * #1838 Support the double data type variable on reading float type values * Feature 1833 develop discussions (#1849) * Per #1833, changed references to met_help to Discussions. * Per #1833, fixed typo and removed an unnecessary word * #1852 Removed unused nc_buf_size. Make cur and dim to the same size (called API handles 2D or 3D) * #1852 Check the start and count before calling NetCDF API * Feature 1746 wavelet stat (#1851) * For issue #1746 modified code to allow users to pass in an empty list (or NA) for forecast and observation thresholds in order to skip applying the threhsolds, but it will still compute stats with the raw fields. SL * For issue #1746, added new unit test that uses a config file that has empty lists for the forecast and observation thresholds. SL * For issue #1746 Added some content related to allowing users to set forecast and observation cat thresholds to an empty list in order to skip the binary masking (and consider all grid-points for stats). SL * Per #1746, cleaning up for consistent indentation. * Per #1746, cleaning up for consistent indentation. * Per #1746, add a revision history note, update the plotting range in the postscript output to be [-n,n] where n is the maximum value of the maximum absolute difference and 1.0, and also fix a bug. When the NA threshold comes AFTER a real threshold, the resulting data and difference values were not being updated. * Per #1746, change the Wavelet-Stat config file values in the the wvlt_plot dictionary by setting plot_min = plot_max = 0.0. That enables the default logic of the tool to take effect. Choose the plotting range of the wavelet plots as [-n,n], where n is the maximum of 1.0 and the maximum absolute difference. * Per #1746, used apply_fcst_thresh where it should have been apply_obs_thresh. * Per issue #1746, modified some content related to users being able to skip applying the categorical threhsolds by putting an empty list or NA in the configuration file. SL * Per issue #1746 Added some warnings if the forecast threshold is set to NA but the observation threshold is not NA (a numeric threshold) and vice versa. SL * Per #1746, fix a couple of typos and tweak wording in the wavelet-stat chapter. * Per #1746, loop over each pair of fcst/obs thresholds to check for inconsistent use of the NA threshold type. * Per #1746, a bit of code cleanup replacing calls to n_elements() with n() to make the code more concise. * Per #1746, need to reinitialize apply_fcst_thresh and apply_obs_thresh to true inside the loop since the NA threshold can appear anywhere in the list of thresholds. Co-authored-by: Seth Linden <linden@kiowa.rap.ucar.edu> Co-authored-by: John Halley Gotway <johnhg@ucar.edu> * Add sphinx.ext.pngmath to conf.py * Fix typo in conf.py * Add sphinx.ext.mathjax to conf.py * Add sphinx.ext.imgmath to conf.py * Fix typo in conf.py * Change math formatting in Appendix C * Temporarily disable equations with errors in Appendix C * Remove sphinx.ext.imgmath in conf.py * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Add sphinx.ext.imgmath to conf.py * Modify math formatting in Appendix C * Temporarily disable equations with errors in Appendix C * Remove sphinx.ext.imgmath in conf.py * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Temporarily disable equations with errors * Update README.md * Modify math formatting in Appendix C * Add usepackage{amssymb} to conf.py * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Fix warnings in wavelet-stat.rst * Omit Indices and tables in PDF version of User Guide * Modify math formatting in Appendix G * Modify math formatting in Appendix E * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Change release date in conf.py * #1852 get_string_val: check if 2D variable. Also checking if the variable does not have dimension (#1860) Co-authored-by: Howard Soh <hsoh@kiowa.rap.ucar.edu> * Feature 1453 create pdf user guide (#1859) * Enable PDF creation * Modify inputenc, fontenc latex_elements * Fix typo in conf.py * Replace unicode with ascii * Replace unicode exponent with math mode * Change eqnarray to align * Replace gif images with png for PDF * Fix png file names * Escape underscore in variable name in math mode * Escape underscore in variable name in math mode * Remove underscores from variables in math mode * Replace bar with overline in some equations * Test derefencing underscores in tables * Test math changes in grid stat * Test math formatting * Test math formatting * Add sphinx.ext.pngmath to conf.py * Fix typo in conf.py * Add sphinx.ext.mathjax to conf.py * Add sphinx.ext.imgmath to conf.py * Fix typo in conf.py * Change math formatting in Appendix C * Temporarily disable equations with errors in Appendix C * Remove sphinx.ext.imgmath in conf.py * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Add sphinx.ext.imgmath to conf.py * Modify math formatting in Appendix C * Temporarily disable equations with errors in Appendix C * Remove sphinx.ext.imgmath in conf.py * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Temporarily disable equations with errors * Modify math formatting in Appendix C * Add usepackage{amssymb} to conf.py * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Fix warnings in wavelet-stat.rst * Omit Indices and tables in PDF version of User Guide * Modify math formatting in Appendix G * Modify math formatting in Appendix E * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Change release date in conf.py Co-authored-by: John Halley Gotway <johnhg@ucar.edu> * Feature 1853 rps doc (#1861) * Per #1853, adding RPS docs from Eric. * Per #1853, work in progress getting the formatting correct. * Per #1853, work in progress getting the formatting correct. * Per #1853, work in progress getting the formatting correct. * Per #1853, work in progress getting the formatting correct. * Per #1853, adding a manual page break after the ROC image to get it to stay in the right spot. * Per #1853, backing out last change which didn't make the ROC image stay in the right spot. * #1855 Removed break after return (SobarQube) * #1855 Removed break after exit * 1855 Added return at MetNcPointObsOut::write_to_netcdf * #1855 Avoid releasing memory twice * #1855 Removed unused code * #1855 Corrected syntax error * #1855 Changed the second call of close() to release _ncFile * #1855 Deleted break after exit * #1855 Chamnged while to if * #1855 Added constructor TCLineCounts and initialized the members * #1855 Added break statement back for case '-' * #1855 Check the new size before extending to avoid thereference of the null pointer * #1855 Changed count to processed_count for log messages. Initialized variables at get_filtered_nc_data\*() * #1855 Corected the number of memory copy * #1855 Exit with an error message if both fgi & bgi are null * #1855 Clear Lead_Times before extening. Formatting for error messages * Add definition for the compilation environment on seneca. * #1855 Extend Lead_Times for new times * #1855 Removed break after exit * #1855 define "c" dynamically * #1855 Move the log message within if statement to avoid dereferencing a null pointer * Feature 1843 scatter index (#1863) * Added scatter index variables: SI, SI_BCL, SI_BCU to cnt columns. SL * Per issue #1843 Added Scatter Index variables (SI, SI_BCL, SI_BCU) to the CNT stat type. SL * Added Scatter Index (SI, SI_BCL, SI_BCU) to cnt stat type header. SL * Per issue #1843, added Scatter Index variables to write_cnt_cols(). SL * Per issue #1843 added Scatter Index (si) to CNTinfo class. SL * Per issue #1843 added Scatter Index (SI, si) to CNTInfo::get_stat(). SL * Per issue #1843 added Scatter Index (SI, si) calculation to both versions of compute_cntinfo(). SL * Per issue #1843, forgot semi-colon after SI calculation, fixed. SL * Added Scatter Index (si) to some of the clear() and allocate() functions. SL * Per issue #1843, added Scatter Index (si) to compute_cnt_mean(). SL * Per issue #1843, added Scatter Index (si) to bootstrap interval calculation and to the CNT write function. SL * Per issue #1843 added Scatter Index (SI, SI_BCL, SI_BCU) to store_stat_cnt(). SL * Per issue #1843, in get_stat() moved Scatterd Index (SI) line to be below RMSE for consitency. SL * Added Scatter Index (SI) to the CNT output format table. SL * Added Scatter Index (SI) to CNT statistics definitions. SL * Update appendixC.rst * Update stat_columns.cc * Update compute_ci.cc * Update met_stats.cc * Per #1843, update write_cntinfo() function. Since we're READING the SI value immediately after the RMSE value when computing CI's, we also need to write SI immediately after RMSE. Otherwise, we'll be computing CI's using the wrong statistics replicates. * Per #1843, check for divide by zero when computing SI... this could happen if comparing the same input file to itself with Grid-Stat. * Per #1843, correct definition of SI from RMSE/ME to RMSE/OBAR. * Per #1843, updated the SI definition to divide by OBAR instead of ME, but forgot to update the divide-by-zero check. Co-authored-by: Seth Linden <linden@kiowa.rap.ucar.edu> Co-authored-by: johnhg <johnhg@ucar.edu> * #1855 Corected typo - compare the numnber of U and V * #1855 Check if var_info is hull * #1855 Avoid un-initialized variables: dt at open(), v at lat() & lon(), and local variables at data() * #1855 Check if ldf is null pointer * #1855 Removed break after exit * #1855 Removed break after exit * Update pcp_combine.cc * Feature 1864 config urls (#1868) * Per #1864, remove stale GitHub pages URL for the MET User's Guide. * Per #1864, found stale GitHub IO links in the issue templates. Should make this same fix in the develop and default branches for all the METplus repos. * Per #1864, fix stale GitHub IO link in the PR template. * Move ROC curve in Appendix C * Crop ROC image for better PDF formatting * Shrink ROC image for better PDF formatting * Modify figure alignment in conf.py * Change text placement for ROC curve figure * Feature 1834 faq (#1867) * testing possible new drop-down menus for FAQ * adding sphinx-panels to extensions to be able to use drop down menus in appendixA * Per #1834, adding sphinx-panels to requirements.txt to see if that fixes the error * Per #1834, added sphinx.ext to beginning of sphinx_panels to see if that fixes the error * Per #1834, remove sphinx.ext and add trailing comma to end of extensions to see if that fixed the error * Per #1834, added sphinx-panels to pip install command to see if that fixes the error * Per #1834, added extra space in comment to get docs to build to see if change to documentation.yml will fix an error * Per #1834, added return character to get rid of warning * testing dropdown menus #1 * testing dropdown menus #2 removing lines and dotted lines * testing #3 * testing 4 * fixing numbering, removing 2nd pulldown menu, trying to fix box width * adding code block in dropdown * trying to fix width * trying to fix width with spacing * trying to fix width with spacing 2nd dropdown * removing panels * solved problem, just use dropdown no panels * adding second dropdown menu * removing the dropdown menus and fixing the spacing. * fixing typos * adding in matched pairs * adding in formating files for NetCDF and making FILE_IO a header * fixing spacing * time slice * fixing spacing * UNIX time conversion * fixing spacing * fixed-width * scientific notation * removing which per Julie P * adding Gen_Vx_Mask section * fixing formatting * changing to numbers * complex masking region * neighborhood methods boundaries * Neighborhood Methods to Compute Fraction * adding Grid_Stat - How do I use Config File Setup to Read a NetCDF file * using backticks with asterics * one more backtick with asterics * Grid_Stat - What would be an example of Verifying Probabilities * fixing typos * What is an example of using Grid-Stat with Regridding and Masking Turned On * fixing typos * Grid_Stat - What would be an example of Verifying Probabilities Example2 * trying to fix typos * trying to fix code-block spacing * Per #1834, attempting to fix warning. * How do I use different masks in MET tools using MODE as an example * trying to fix bullet list * trying to fix bullet list 2 * trying to fix bullet list 2 * trying to fix bullet list 3 * Per #1834, attempting formatting changes * Per #1834, attempting formatting changes * Per #1834, attempting formatting changes * Per #1834, attempting formatting changes * Per #1834, attempting formatting changes * Per #1834, modified formatting, including adding backslashes and removing extra returns in command line calls, and adding indentation for config file notation, and adding a return after each 'A.' * Per #1834, changed code-block from ini to none to fix formatting * *Pcp-Combine - What are some examples using -add * Pcp-Combine - How do I add and subtract with Pcp-Combine * Pcp-Combine - How do I Combine 12-hour Accumulated Precipitation from Two Different Initialization Times * Pcp-Combine - How Do I Correct a Precipitation Time Range * Pcp-Combine - What Data Formats does MET Read * Pcp-Combine - How does pass through work * Pcp_Combine - How do I use -pcprx to run a project faster * Pcp-Combine - How Do I Enter the Time Format Correctly * Pcp-Combine - How do I use -subtract * fixing formatting * Pcp_Combine - How Do I Use -sum, -add, and -subtract to Achieve Same Accumulation Interval * Pcp-Combine - What is the difference between -sum vs. -add * Pcp-Combine - How Do I Select a Specific GRIB Record to Use * fixing spacing for original text * Plot-Data-Plane - How Do I Inspect Gen_Vx_Mask Output * fixing typo * Plot-Data-Plane - How Do I Specify GRIB Version * Plot-Data-Plane - last 2 entries * 2 misc. entries * fixing typo * Stat_Analysis - How does -aggregate_stat work * Stat_Analysis - remaining entries * fixing typo * tc-pairs * tc-stat * troubleshooting area * another stat-analysis * another stat-analysis * utilities * fixing dashes * Per #1834, trying out syntax * Per #1834, removing added syntax * fixing returns with bolding * Per #1834, modified formatting * adding italics for directories #1834 * replacing code-block ini with none to remove red lettering * fixing questions to lower case #1834 * linking to section using numref #1834 * linking to section using numref #1834 attempt 2 * linking to section using numref #1834 attempt 3 * linking to section using numref #1834 attempt 4 * linking to section using numref #1834 attempt 5 * linking to section using numref #1834 attempt 6 * linking to section using numref #1834 attempt 7 * linking to section using numref #1834 attempt 8 * linking to anothre section #1834 * linking to another section #1834 * removing old info #1834 * grammar updates #1834 * grammar updates #1834 * fixing typos #1834 * email changes with Julie #1834 * trying to fix warning #1834 * changes #1834 * Per #1834, updating question titles and answers * Per #1834, made updates to the content. * Per #1834, made updates to the content * Per #1834, made updates to the content * Per #1834, made some major changes to the first 3 FAQ's. I don't have the time/bandwidth to review these completely at this time, but wanted to include at least these updates. * testing some of the updates John O. suggested. * updates John O. suggested take 2 #1834. * updates John O. suggested take 3 #1834. * updates John O. suggested take 4 #1834. * updates John O. suggested take 5 #1834. * updates John O. suggested. Some didn't show up even though they've been removed #1834. * trying to fix numbered list indenting #1834 * fixing numbered list indenting #1834 * cleaning up formatting #1834 * fixing broken links #1834 Co-authored-by: Julie.Prestopnik <jpresto@ucar.edu> Co-authored-by: John Halley Gotway <johnhg@ucar.edu> * bugfix 1872 plot_tcmpr.R -lookin directory (develop) (#1874) * Bugfix 1875 develop memory (#1877) * Per #1875, fix set_cdf_ta() to initialize cdf_ta prior to populating it. * Per #1875, add debug(4) log messages whenever defining a new aggregation case in stat-analysis. * Per #1875, switch log messages about new cases from Debug(4) to Debug(3). * Per #1879, porting the same changes over to the develop branch. (#1881) * Feature 1673 gbeta (#1857) * Per #1673, add a distance_map.beta_value entry to all the Grid-Stat config files. * Per #1673, enhance Grid-Stat to parse the new beta_value option and set its value in the library code. * Per #1673, set the user-defined beta_value when processing the distance map scores. * Per #1673, add 4 new DMAP columns write them. Just writing bad data values right now, still need to actually compute them! Also need to update Stat-Analysis to update the parsing/aggregation of these new values. * Per #1673, enable beta_value to be set to a default bad data value. * Per #1673 and @ericgilleland instruction (#1673 (comment)), strip asymmetric G-Beta (AGBETA) from the output. * Per #1673, fix typo that was causing the code not to compile. * Per #1673, update MET User's Guide about G/GBETA. * Per #1673, report BETA_VALUE = TOTAL / 2 in the output instead of NA, as directed by Eric. * Per #1673, remove descriptions of Grid-Stat-specific config options for nbrhd, fourier, gradient, and distance_map from the overview section since they belong in the Grid-Stat chapter. Also add direct links for them in the Grid-Stat chapter. * Per #1673, do not reset the beta_value in DMAP::clear(). * Per #1673, log the DMAP options only after the beta_value has been determined. * Per #1673, correct the default beta_value setting as N*N / 2.0. * Per #1673, trying to format G-beta well. * Per #1673, trying to format G-beta well. * Per #1673, beta is formatted as \beta and not \Beta in latex. * Per #1673, beta is formatted as \beta and not \Beta in latex. * Per #1673, switch to defining beta_value as a function of the input grid size. * Per #1673, push initial appendix C updates for GBETA. * Per #1673, working on G and GBETA equations. * Per #1673, working on G and GBETA equations. * Per 1673, update GBETA details in appendix C. * Per #1673, add Eric's info about selecting beta. * Per #1673, update reference for Gilleland-2019 to Gilleland_PartI-2020 and Gilleland_PartII-2020. * Per 1673, fix typo. * Per #1673, making changes requested by Eric. * Per #1673, making changes requested by Eric. * Per #1673, update references based on Eric's feedback. * Update appendixD.rst (#1869) Updated this appendix. * Update appendixD.rst * Per #1673, making the formatting of Gilleland-2020 part I and II references consistent. * Per #1673, update Grid-Stat docs to clarify that GBETA is only computed on the FULL verification domain and not any masking regions. * Per #1673, add DataPlane::n_good_data() function to return a count valid data values. * Per #1673, update vx_statistics library code to only set gbeta/beta_value to bad data for sub-domains. * Per #1673, update grid_stat to mask out bad data between the fcst/obs fields before computing distance maps. This will change existing unit test results! Also store the count of valid data values in the full verification domain. * Per #1673, since GBETA is only reported when VX_MASK = FULL, updating GridStatConfig_all to include the FULL masking region to make this example more meaningful. Co-authored-by: ericgilleland <34106925+ericgilleland@users.noreply.github.com> Co-authored-by: jprestop <jpresto@ucar.edu> Co-authored-by: Howard Soh <hsoh@kiowa.rap.ucar.edu> Co-authored-by: hsoh-u <hsoh@ucar.edu> Co-authored-by: Keith Searight <keith.searight@noaa.gov> Co-authored-by: Seth Linden <linden@ucar.edu> Co-authored-by: Seth Linden <linden@kiowa.rap.ucar.edu> Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> Co-authored-by: lisagoodrich <33230218+lisagoodrich@users.noreply.github.com> Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> Co-authored-by: ericgilleland <34106925+ericgilleland@users.noreply.github.com>
JohnHalleyGotway
added a commit
that referenced
this pull request
Aug 23, 2021
* Update install_met_env.kiowa Commented out items and added text to description * #1817 Added to_north argument at two_to_one() * #1817 Calls two_to_one() with false for the third argument, to_north, if latitude moves north to south * Feature 1819 automation doc warnings (#1836) * Removing "/lib" that should not be there. * #1508 checking stat() methoid returns no error before looking at FILE flag * Enable PDF creation * Modify inputenc, fontenc latex_elements * Fix typo in conf.py * Replace unicode with ascii * Replace unicode exponent with math mode * Change eqnarray to align * Replace gif images with png for PDF * Fix png file names * Escape underscore in variable name in math mode * Escape underscore in variable name in math mode * Remove underscores from variables in math mode * Replace bar with overline in some equations * Test derefencing underscores in tables * Test math changes in grid stat * Test math formatting * Test math formatting * #1838 Added log message for lat/lon values * #1838 Give warning if the first and lat lat/lon is same * #1838 Support the double data type variable on reading float type values * Feature 1833 develop discussions (#1849) * Per #1833, changed references to met_help to Discussions. * Per #1833, fixed typo and removed an unnecessary word * #1852 Removed unused nc_buf_size. Make cur and dim to the same size (called API handles 2D or 3D) * #1852 Check the start and count before calling NetCDF API * Feature 1746 wavelet stat (#1851) * For issue #1746 modified code to allow users to pass in an empty list (or NA) for forecast and observation thresholds in order to skip applying the threhsolds, but it will still compute stats with the raw fields. SL * For issue #1746, added new unit test that uses a config file that has empty lists for the forecast and observation thresholds. SL * For issue #1746 Added some content related to allowing users to set forecast and observation cat thresholds to an empty list in order to skip the binary masking (and consider all grid-points for stats). SL * Per #1746, cleaning up for consistent indentation. * Per #1746, cleaning up for consistent indentation. * Per #1746, add a revision history note, update the plotting range in the postscript output to be [-n,n] where n is the maximum value of the maximum absolute difference and 1.0, and also fix a bug. When the NA threshold comes AFTER a real threshold, the resulting data and difference values were not being updated. * Per #1746, change the Wavelet-Stat config file values in the the wvlt_plot dictionary by setting plot_min = plot_max = 0.0. That enables the default logic of the tool to take effect. Choose the plotting range of the wavelet plots as [-n,n], where n is the maximum of 1.0 and the maximum absolute difference. * Per #1746, used apply_fcst_thresh where it should have been apply_obs_thresh. * Per issue #1746, modified some content related to users being able to skip applying the categorical threhsolds by putting an empty list or NA in the configuration file. SL * Per issue #1746 Added some warnings if the forecast threshold is set to NA but the observation threshold is not NA (a numeric threshold) and vice versa. SL * Per #1746, fix a couple of typos and tweak wording in the wavelet-stat chapter. * Per #1746, loop over each pair of fcst/obs thresholds to check for inconsistent use of the NA threshold type. * Per #1746, a bit of code cleanup replacing calls to n_elements() with n() to make the code more concise. * Per #1746, need to reinitialize apply_fcst_thresh and apply_obs_thresh to true inside the loop since the NA threshold can appear anywhere in the list of thresholds. Co-authored-by: Seth Linden <linden@kiowa.rap.ucar.edu> Co-authored-by: John Halley Gotway <johnhg@ucar.edu> * Add sphinx.ext.pngmath to conf.py * Fix typo in conf.py * Add sphinx.ext.mathjax to conf.py * Add sphinx.ext.imgmath to conf.py * Fix typo in conf.py * Change math formatting in Appendix C * Temporarily disable equations with errors in Appendix C * Remove sphinx.ext.imgmath in conf.py * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Add sphinx.ext.imgmath to conf.py * Modify math formatting in Appendix C * Temporarily disable equations with errors in Appendix C * Remove sphinx.ext.imgmath in conf.py * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Temporarily disable equations with errors * Update README.md * Modify math formatting in Appendix C * Add usepackage{amssymb} to conf.py * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Fix warnings in wavelet-stat.rst * Omit Indices and tables in PDF version of User Guide * Modify math formatting in Appendix G * Modify math formatting in Appendix E * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Change release date in conf.py * #1852 get_string_val: check if 2D variable. Also checking if the variable does not have dimension (#1860) Co-authored-by: Howard Soh <hsoh@kiowa.rap.ucar.edu> * Feature 1453 create pdf user guide (#1859) * Enable PDF creation * Modify inputenc, fontenc latex_elements * Fix typo in conf.py * Replace unicode with ascii * Replace unicode exponent with math mode * Change eqnarray to align * Replace gif images with png for PDF * Fix png file names * Escape underscore in variable name in math mode * Escape underscore in variable name in math mode * Remove underscores from variables in math mode * Replace bar with overline in some equations * Test derefencing underscores in tables * Test math changes in grid stat * Test math formatting * Test math formatting * Add sphinx.ext.pngmath to conf.py * Fix typo in conf.py * Add sphinx.ext.mathjax to conf.py * Add sphinx.ext.imgmath to conf.py * Fix typo in conf.py * Change math formatting in Appendix C * Temporarily disable equations with errors in Appendix C * Remove sphinx.ext.imgmath in conf.py * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Add sphinx.ext.imgmath to conf.py * Modify math formatting in Appendix C * Temporarily disable equations with errors in Appendix C * Remove sphinx.ext.imgmath in conf.py * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Temporarily disable equations with errors * Modify math formatting in Appendix C * Add usepackage{amssymb} to conf.py * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Fix warnings in wavelet-stat.rst * Omit Indices and tables in PDF version of User Guide * Modify math formatting in Appendix G * Modify math formatting in Appendix E * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Change release date in conf.py Co-authored-by: John Halley Gotway <johnhg@ucar.edu> * Feature 1853 rps doc (#1861) * Per #1853, adding RPS docs from Eric. * Per #1853, work in progress getting the formatting correct. * Per #1853, work in progress getting the formatting correct. * Per #1853, work in progress getting the formatting correct. * Per #1853, work in progress getting the formatting correct. * Per #1853, adding a manual page break after the ROC image to get it to stay in the right spot. * Per #1853, backing out last change which didn't make the ROC image stay in the right spot. * #1855 Removed break after return (SobarQube) * #1855 Removed break after exit * 1855 Added return at MetNcPointObsOut::write_to_netcdf * #1855 Avoid releasing memory twice * #1855 Removed unused code * #1855 Corrected syntax error * #1855 Changed the second call of close() to release _ncFile * #1855 Deleted break after exit * #1855 Chamnged while to if * #1855 Added constructor TCLineCounts and initialized the members * #1855 Added break statement back for case '-' * #1855 Check the new size before extending to avoid thereference of the null pointer * #1855 Changed count to processed_count for log messages. Initialized variables at get_filtered_nc_data\*() * #1855 Corected the number of memory copy * #1855 Exit with an error message if both fgi & bgi are null * #1855 Clear Lead_Times before extening. Formatting for error messages * Add definition for the compilation environment on seneca. * #1855 Extend Lead_Times for new times * #1855 Removed break after exit * #1855 define "c" dynamically * #1855 Move the log message within if statement to avoid dereferencing a null pointer * Feature 1843 scatter index (#1863) * Added scatter index variables: SI, SI_BCL, SI_BCU to cnt columns. SL * Per issue #1843 Added Scatter Index variables (SI, SI_BCL, SI_BCU) to the CNT stat type. SL * Added Scatter Index (SI, SI_BCL, SI_BCU) to cnt stat type header. SL * Per issue #1843, added Scatter Index variables to write_cnt_cols(). SL * Per issue #1843 added Scatter Index (si) to CNTinfo class. SL * Per issue #1843 added Scatter Index (SI, si) to CNTInfo::get_stat(). SL * Per issue #1843 added Scatter Index (SI, si) calculation to both versions of compute_cntinfo(). SL * Per issue #1843, forgot semi-colon after SI calculation, fixed. SL * Added Scatter Index (si) to some of the clear() and allocate() functions. SL * Per issue #1843, added Scatter Index (si) to compute_cnt_mean(). SL * Per issue #1843, added Scatter Index (si) to bootstrap interval calculation and to the CNT write function. SL * Per issue #1843 added Scatter Index (SI, SI_BCL, SI_BCU) to store_stat_cnt(). SL * Per issue #1843, in get_stat() moved Scatterd Index (SI) line to be below RMSE for consitency. SL * Added Scatter Index (SI) to the CNT output format table. SL * Added Scatter Index (SI) to CNT statistics definitions. SL * Update appendixC.rst * Update stat_columns.cc * Update compute_ci.cc * Update met_stats.cc * Per #1843, update write_cntinfo() function. Since we're READING the SI value immediately after the RMSE value when computing CI's, we also need to write SI immediately after RMSE. Otherwise, we'll be computing CI's using the wrong statistics replicates. * Per #1843, check for divide by zero when computing SI... this could happen if comparing the same input file to itself with Grid-Stat. * Per #1843, correct definition of SI from RMSE/ME to RMSE/OBAR. * Per #1843, updated the SI definition to divide by OBAR instead of ME, but forgot to update the divide-by-zero check. Co-authored-by: Seth Linden <linden@kiowa.rap.ucar.edu> Co-authored-by: johnhg <johnhg@ucar.edu> * #1855 Corected typo - compare the numnber of U and V * #1855 Check if var_info is hull * #1855 Avoid un-initialized variables: dt at open(), v at lat() & lon(), and local variables at data() * #1855 Check if ldf is null pointer * #1855 Removed break after exit * #1855 Removed break after exit * Update pcp_combine.cc * Feature 1864 config urls (#1868) * Per #1864, remove stale GitHub pages URL for the MET User's Guide. * Per #1864, found stale GitHub IO links in the issue templates. Should make this same fix in the develop and default branches for all the METplus repos. * Per #1864, fix stale GitHub IO link in the PR template. * Move ROC curve in Appendix C * Crop ROC image for better PDF formatting * Shrink ROC image for better PDF formatting * Modify figure alignment in conf.py * Change text placement for ROC curve figure * Feature 1834 faq (#1867) * testing possible new drop-down menus for FAQ * adding sphinx-panels to extensions to be able to use drop down menus in appendixA * Per #1834, adding sphinx-panels to requirements.txt to see if that fixes the error * Per #1834, added sphinx.ext to beginning of sphinx_panels to see if that fixes the error * Per #1834, remove sphinx.ext and add trailing comma to end of extensions to see if that fixed the error * Per #1834, added sphinx-panels to pip install command to see if that fixes the error * Per #1834, added extra space in comment to get docs to build to see if change to documentation.yml will fix an error * Per #1834, added return character to get rid of warning * testing dropdown menus #1 * testing dropdown menus #2 removing lines and dotted lines * testing #3 * testing 4 * fixing numbering, removing 2nd pulldown menu, trying to fix box width * adding code block in dropdown * trying to fix width * trying to fix width with spacing * trying to fix width with spacing 2nd dropdown * removing panels * solved problem, just use dropdown no panels * adding second dropdown menu * removing the dropdown menus and fixing the spacing. * fixing typos * adding in matched pairs * adding in formating files for NetCDF and making FILE_IO a header * fixing spacing * time slice * fixing spacing * UNIX time conversion * fixing spacing * fixed-width * scientific notation * removing which per Julie P * adding Gen_Vx_Mask section * fixing formatting * changing to numbers * complex masking region * neighborhood methods boundaries * Neighborhood Methods to Compute Fraction * adding Grid_Stat - How do I use Config File Setup to Read a NetCDF file * using backticks with asterics * one more backtick with asterics * Grid_Stat - What would be an example of Verifying Probabilities * fixing typos * What is an example of using Grid-Stat with Regridding and Masking Turned On * fixing typos * Grid_Stat - What would be an example of Verifying Probabilities Example2 * trying to fix typos * trying to fix code-block spacing * Per #1834, attempting to fix warning. * How do I use different masks in MET tools using MODE as an example * trying to fix bullet list * trying to fix bullet list 2 * trying to fix bullet list 2 * trying to fix bullet list 3 * Per #1834, attempting formatting changes * Per #1834, attempting formatting changes * Per #1834, attempting formatting changes * Per #1834, attempting formatting changes * Per #1834, attempting formatting changes * Per #1834, modified formatting, including adding backslashes and removing extra returns in command line calls, and adding indentation for config file notation, and adding a return after each 'A.' * Per #1834, changed code-block from ini to none to fix formatting * *Pcp-Combine - What are some examples using -add * Pcp-Combine - How do I add and subtract with Pcp-Combine * Pcp-Combine - How do I Combine 12-hour Accumulated Precipitation from Two Different Initialization Times * Pcp-Combine - How Do I Correct a Precipitation Time Range * Pcp-Combine - What Data Formats does MET Read * Pcp-Combine - How does pass through work * Pcp_Combine - How do I use -pcprx to run a project faster * Pcp-Combine - How Do I Enter the Time Format Correctly * Pcp-Combine - How do I use -subtract * fixing formatting * Pcp_Combine - How Do I Use -sum, -add, and -subtract to Achieve Same Accumulation Interval * Pcp-Combine - What is the difference between -sum vs. -add * Pcp-Combine - How Do I Select a Specific GRIB Record to Use * fixing spacing for original text * Plot-Data-Plane - How Do I Inspect Gen_Vx_Mask Output * fixing typo * Plot-Data-Plane - How Do I Specify GRIB Version * Plot-Data-Plane - last 2 entries * 2 misc. entries * fixing typo * Stat_Analysis - How does -aggregate_stat work * Stat_Analysis - remaining entries * fixing typo * tc-pairs * tc-stat * troubleshooting area * another stat-analysis * another stat-analysis * utilities * fixing dashes * Per #1834, trying out syntax * Per #1834, removing added syntax * fixing returns with bolding * Per #1834, modified formatting * adding italics for directories #1834 * replacing code-block ini with none to remove red lettering * fixing questions to lower case #1834 * linking to section using numref #1834 * linking to section using numref #1834 attempt 2 * linking to section using numref #1834 attempt 3 * linking to section using numref #1834 attempt 4 * linking to section using numref #1834 attempt 5 * linking to section using numref #1834 attempt 6 * linking to section using numref #1834 attempt 7 * linking to section using numref #1834 attempt 8 * linking to anothre section #1834 * linking to another section #1834 * removing old info #1834 * grammar updates #1834 * grammar updates #1834 * fixing typos #1834 * email changes with Julie #1834 * trying to fix warning #1834 * changes #1834 * Per #1834, updating question titles and answers * Per #1834, made updates to the content. * Per #1834, made updates to the content * Per #1834, made updates to the content * Per #1834, made some major changes to the first 3 FAQ's. I don't have the time/bandwidth to review these completely at this time, but wanted to include at least these updates. * testing some of the updates John O. suggested. * updates John O. suggested take 2 #1834. * updates John O. suggested take 3 #1834. * updates John O. suggested take 4 #1834. * updates John O. suggested take 5 #1834. * updates John O. suggested. Some didn't show up even though they've been removed #1834. * trying to fix numbered list indenting #1834 * fixing numbered list indenting #1834 * cleaning up formatting #1834 * fixing broken links #1834 Co-authored-by: Julie.Prestopnik <jpresto@ucar.edu> Co-authored-by: John Halley Gotway <johnhg@ucar.edu> * bugfix 1872 plot_tcmpr.R -lookin directory (develop) (#1874) * Bugfix 1875 develop memory (#1877) * Per #1875, fix set_cdf_ta() to initialize cdf_ta prior to populating it. * Per #1875, add debug(4) log messages whenever defining a new aggregation case in stat-analysis. * Per #1875, switch log messages about new cases from Debug(4) to Debug(3). * Per #1879, porting the same changes over to the develop branch. (#1881) * Feature 1673 gbeta (#1857) * Per #1673, add a distance_map.beta_value entry to all the Grid-Stat config files. * Per #1673, enhance Grid-Stat to parse the new beta_value option and set its value in the library code. * Per #1673, set the user-defined beta_value when processing the distance map scores. * Per #1673, add 4 new DMAP columns write them. Just writing bad data values right now, still need to actually compute them! Also need to update Stat-Analysis to update the parsing/aggregation of these new values. * Per #1673, enable beta_value to be set to a default bad data value. * Per #1673 and @ericgilleland instruction (#1673 (comment)), strip asymmetric G-Beta (AGBETA) from the output. * Per #1673, fix typo that was causing the code not to compile. * Per #1673, update MET User's Guide about G/GBETA. * Per #1673, report BETA_VALUE = TOTAL / 2 in the output instead of NA, as directed by Eric. * Per #1673, remove descriptions of Grid-Stat-specific config options for nbrhd, fourier, gradient, and distance_map from the overview section since they belong in the Grid-Stat chapter. Also add direct links for them in the Grid-Stat chapter. * Per #1673, do not reset the beta_value in DMAP::clear(). * Per #1673, log the DMAP options only after the beta_value has been determined. * Per #1673, correct the default beta_value setting as N*N / 2.0. * Per #1673, trying to format G-beta well. * Per #1673, trying to format G-beta well. * Per #1673, beta is formatted as \beta and not \Beta in latex. * Per #1673, beta is formatted as \beta and not \Beta in latex. * Per #1673, switch to defining beta_value as a function of the input grid size. * Per #1673, push initial appendix C updates for GBETA. * Per #1673, working on G and GBETA equations. * Per #1673, working on G and GBETA equations. * Per 1673, update GBETA details in appendix C. * Per #1673, add Eric's info about selecting beta. * Per #1673, update reference for Gilleland-2019 to Gilleland_PartI-2020 and Gilleland_PartII-2020. * Per 1673, fix typo. * Per #1673, making changes requested by Eric. * Per #1673, making changes requested by Eric. * Per #1673, update references based on Eric's feedback. * Update appendixD.rst (#1869) Updated this appendix. * Update appendixD.rst * Per #1673, making the formatting of Gilleland-2020 part I and II references consistent. * Per #1673, update Grid-Stat docs to clarify that GBETA is only computed on the FULL verification domain and not any masking regions. * Per #1673, add DataPlane::n_good_data() function to return a count valid data values. * Per #1673, update vx_statistics library code to only set gbeta/beta_value to bad data for sub-domains. * Per #1673, update grid_stat to mask out bad data between the fcst/obs fields before computing distance maps. This will change existing unit test results! Also store the count of valid data values in the full verification domain. * Per #1673, since GBETA is only reported when VX_MASK = FULL, updating GridStatConfig_all to include the FULL masking region to make this example more meaningful. Co-authored-by: ericgilleland <34106925+ericgilleland@users.noreply.github.com> * Feature 1764 point stat (#1885) * Per #1764 added write_orank_row() to do_hera_ens(). SL * Per issue #1764, added orank stat type for ensemble output. SL * Per issue #1764, added code to setup_txt_files() for ORANK output. SL * Added some comments for debuging. SL * Per #1764, add orank to the list of outputs created by point_stat. * Per #1764, add functions to compute the number of requested HiRA probabilities and ensemble members. Also update n_txt_row() to keep track of the number of ORANK rows to be written. Not totally sure if I'm counting these ORANK output lines correctly though. * Per #1764, update logic in setup_txt_files() to call get_max_n_hira_prob() and get_max_n_hira_ens() functions. * Per #1764, no real changes. Just fixing spacing. * Per issue #1764, cleaned up print (cout) debug statements. SL * Per issue #1764 added 'orank = NONE' to the config files for compatibility with changes to point_stat for calculating ORANK. SL * Per issue #1764 added 'orank = STAT' to config file to produce ORANK output lines in point_stat. SL * Per issue #1764 added content in all relevant sections for ORANK: observation rank statistic. SL * Added 'orank = NONE' to config file for compatibility with changes to point_stant for ORANK. SL * Added 'orank = NONE' to config file for compatibility with changes to point_stant for ORANK. SL * Apply suggestions from code review Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> * Per #1764, no real changes, just fixing indents. * Per #1764, no real changes, just fixing indents. * Per #1764, no real changes, just fixing indents. * Per #1764, putting the docs for VCNT back into its logical order with the other vector line types. The order that they're listed here does not need to match their order in the enumeration in the code. * Per #1764, fixing a vcnt typo I found in the grid-stat chapter... and making them consistent. * Per issue #1764: After the call to write_orank_row, added lines to reset the observation begin and end times. SL * Per #1764, no code changes, just fixing indents. Co-authored-by: Seth Linden <linden@kiowa.rap.ucar.edu> Co-authored-by: John Halley Gotway <johnhg@ucar.edu> Co-authored-by: John Halley Gotway <johnhg@kiowa.rap.ucar.edu> Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> * Adding Seth to the NB notification list. Co-authored-by: jprestop <jpresto@ucar.edu> Co-authored-by: Howard Soh <hsoh@kiowa.rap.ucar.edu> Co-authored-by: hsoh-u <hsoh@ucar.edu> Co-authored-by: Keith Searight <keith.searight@noaa.gov> Co-authored-by: Seth Linden <linden@ucar.edu> Co-authored-by: Seth Linden <linden@kiowa.rap.ucar.edu> Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> Co-authored-by: lisagoodrich <33230218+lisagoodrich@users.noreply.github.com> Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> Co-authored-by: ericgilleland <34106925+ericgilleland@users.noreply.github.com> Co-authored-by: John Halley Gotway <johnhg@kiowa.rap.ucar.edu> Co-authored-by: MET Tools Test Account <met_test@kiowa.rap.ucar.edu>
JohnHalleyGotway
added a commit
that referenced
this pull request
Aug 24, 2021
* Update install_met_env.kiowa Commented out items and added text to description * #1817 Added to_north argument at two_to_one() * #1817 Calls two_to_one() with false for the third argument, to_north, if latitude moves north to south * Feature 1819 automation doc warnings (#1836) * Removing "/lib" that should not be there. * #1508 checking stat() methoid returns no error before looking at FILE flag * Enable PDF creation * Modify inputenc, fontenc latex_elements * Fix typo in conf.py * Replace unicode with ascii * Replace unicode exponent with math mode * Change eqnarray to align * Replace gif images with png for PDF * Fix png file names * Escape underscore in variable name in math mode * Escape underscore in variable name in math mode * Remove underscores from variables in math mode * Replace bar with overline in some equations * Test derefencing underscores in tables * Test math changes in grid stat * Test math formatting * Test math formatting * #1838 Added log message for lat/lon values * #1838 Give warning if the first and lat lat/lon is same * #1838 Support the double data type variable on reading float type values * Feature 1833 develop discussions (#1849) * Per #1833, changed references to met_help to Discussions. * Per #1833, fixed typo and removed an unnecessary word * #1852 Removed unused nc_buf_size. Make cur and dim to the same size (called API handles 2D or 3D) * #1852 Check the start and count before calling NetCDF API * Feature 1746 wavelet stat (#1851) * For issue #1746 modified code to allow users to pass in an empty list (or NA) for forecast and observation thresholds in order to skip applying the threhsolds, but it will still compute stats with the raw fields. SL * For issue #1746, added new unit test that uses a config file that has empty lists for the forecast and observation thresholds. SL * For issue #1746 Added some content related to allowing users to set forecast and observation cat thresholds to an empty list in order to skip the binary masking (and consider all grid-points for stats). SL * Per #1746, cleaning up for consistent indentation. * Per #1746, cleaning up for consistent indentation. * Per #1746, add a revision history note, update the plotting range in the postscript output to be [-n,n] where n is the maximum value of the maximum absolute difference and 1.0, and also fix a bug. When the NA threshold comes AFTER a real threshold, the resulting data and difference values were not being updated. * Per #1746, change the Wavelet-Stat config file values in the the wvlt_plot dictionary by setting plot_min = plot_max = 0.0. That enables the default logic of the tool to take effect. Choose the plotting range of the wavelet plots as [-n,n], where n is the maximum of 1.0 and the maximum absolute difference. * Per #1746, used apply_fcst_thresh where it should have been apply_obs_thresh. * Per issue #1746, modified some content related to users being able to skip applying the categorical threhsolds by putting an empty list or NA in the configuration file. SL * Per issue #1746 Added some warnings if the forecast threshold is set to NA but the observation threshold is not NA (a numeric threshold) and vice versa. SL * Per #1746, fix a couple of typos and tweak wording in the wavelet-stat chapter. * Per #1746, loop over each pair of fcst/obs thresholds to check for inconsistent use of the NA threshold type. * Per #1746, a bit of code cleanup replacing calls to n_elements() with n() to make the code more concise. * Per #1746, need to reinitialize apply_fcst_thresh and apply_obs_thresh to true inside the loop since the NA threshold can appear anywhere in the list of thresholds. Co-authored-by: Seth Linden <linden@kiowa.rap.ucar.edu> Co-authored-by: John Halley Gotway <johnhg@ucar.edu> * Add sphinx.ext.pngmath to conf.py * Fix typo in conf.py * Add sphinx.ext.mathjax to conf.py * Add sphinx.ext.imgmath to conf.py * Fix typo in conf.py * Change math formatting in Appendix C * Temporarily disable equations with errors in Appendix C * Remove sphinx.ext.imgmath in conf.py * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Add sphinx.ext.imgmath to conf.py * Modify math formatting in Appendix C * Temporarily disable equations with errors in Appendix C * Remove sphinx.ext.imgmath in conf.py * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Temporarily disable equations with errors * Update README.md * Modify math formatting in Appendix C * Add usepackage{amssymb} to conf.py * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Fix warnings in wavelet-stat.rst * Omit Indices and tables in PDF version of User Guide * Modify math formatting in Appendix G * Modify math formatting in Appendix E * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Change release date in conf.py * #1852 get_string_val: check if 2D variable. Also checking if the variable does not have dimension (#1860) Co-authored-by: Howard Soh <hsoh@kiowa.rap.ucar.edu> * Feature 1453 create pdf user guide (#1859) * Enable PDF creation * Modify inputenc, fontenc latex_elements * Fix typo in conf.py * Replace unicode with ascii * Replace unicode exponent with math mode * Change eqnarray to align * Replace gif images with png for PDF * Fix png file names * Escape underscore in variable name in math mode * Escape underscore in variable name in math mode * Remove underscores from variables in math mode * Replace bar with overline in some equations * Test derefencing underscores in tables * Test math changes in grid stat * Test math formatting * Test math formatting * Add sphinx.ext.pngmath to conf.py * Fix typo in conf.py * Add sphinx.ext.mathjax to conf.py * Add sphinx.ext.imgmath to conf.py * Fix typo in conf.py * Change math formatting in Appendix C * Temporarily disable equations with errors in Appendix C * Remove sphinx.ext.imgmath in conf.py * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Add sphinx.ext.imgmath to conf.py * Modify math formatting in Appendix C * Temporarily disable equations with errors in Appendix C * Remove sphinx.ext.imgmath in conf.py * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Temporarily disable equations with errors * Modify math formatting in Appendix C * Add usepackage{amssymb} to conf.py * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Fix warnings in wavelet-stat.rst * Omit Indices and tables in PDF version of User Guide * Modify math formatting in Appendix G * Modify math formatting in Appendix E * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Change release date in conf.py Co-authored-by: John Halley Gotway <johnhg@ucar.edu> * Feature 1853 rps doc (#1861) * Per #1853, adding RPS docs from Eric. * Per #1853, work in progress getting the formatting correct. * Per #1853, work in progress getting the formatting correct. * Per #1853, work in progress getting the formatting correct. * Per #1853, work in progress getting the formatting correct. * Per #1853, adding a manual page break after the ROC image to get it to stay in the right spot. * Per #1853, backing out last change which didn't make the ROC image stay in the right spot. * #1855 Removed break after return (SobarQube) * #1855 Removed break after exit * 1855 Added return at MetNcPointObsOut::write_to_netcdf * #1855 Avoid releasing memory twice * #1855 Removed unused code * #1855 Corrected syntax error * #1855 Changed the second call of close() to release _ncFile * #1855 Deleted break after exit * #1855 Chamnged while to if * #1855 Added constructor TCLineCounts and initialized the members * #1855 Added break statement back for case '-' * #1855 Check the new size before extending to avoid thereference of the null pointer * #1855 Changed count to processed_count for log messages. Initialized variables at get_filtered_nc_data\*() * #1855 Corected the number of memory copy * #1855 Exit with an error message if both fgi & bgi are null * #1855 Clear Lead_Times before extening. Formatting for error messages * Add definition for the compilation environment on seneca. * #1855 Extend Lead_Times for new times * #1855 Removed break after exit * #1855 define "c" dynamically * #1855 Move the log message within if statement to avoid dereferencing a null pointer * Feature 1843 scatter index (#1863) * Added scatter index variables: SI, SI_BCL, SI_BCU to cnt columns. SL * Per issue #1843 Added Scatter Index variables (SI, SI_BCL, SI_BCU) to the CNT stat type. SL * Added Scatter Index (SI, SI_BCL, SI_BCU) to cnt stat type header. SL * Per issue #1843, added Scatter Index variables to write_cnt_cols(). SL * Per issue #1843 added Scatter Index (si) to CNTinfo class. SL * Per issue #1843 added Scatter Index (SI, si) to CNTInfo::get_stat(). SL * Per issue #1843 added Scatter Index (SI, si) calculation to both versions of compute_cntinfo(). SL * Per issue #1843, forgot semi-colon after SI calculation, fixed. SL * Added Scatter Index (si) to some of the clear() and allocate() functions. SL * Per issue #1843, added Scatter Index (si) to compute_cnt_mean(). SL * Per issue #1843, added Scatter Index (si) to bootstrap interval calculation and to the CNT write function. SL * Per issue #1843 added Scatter Index (SI, SI_BCL, SI_BCU) to store_stat_cnt(). SL * Per issue #1843, in get_stat() moved Scatterd Index (SI) line to be below RMSE for consitency. SL * Added Scatter Index (SI) to the CNT output format table. SL * Added Scatter Index (SI) to CNT statistics definitions. SL * Update appendixC.rst * Update stat_columns.cc * Update compute_ci.cc * Update met_stats.cc * Per #1843, update write_cntinfo() function. Since we're READING the SI value immediately after the RMSE value when computing CI's, we also need to write SI immediately after RMSE. Otherwise, we'll be computing CI's using the wrong statistics replicates. * Per #1843, check for divide by zero when computing SI... this could happen if comparing the same input file to itself with Grid-Stat. * Per #1843, correct definition of SI from RMSE/ME to RMSE/OBAR. * Per #1843, updated the SI definition to divide by OBAR instead of ME, but forgot to update the divide-by-zero check. Co-authored-by: Seth Linden <linden@kiowa.rap.ucar.edu> Co-authored-by: johnhg <johnhg@ucar.edu> * #1855 Corected typo - compare the numnber of U and V * #1855 Check if var_info is hull * #1855 Avoid un-initialized variables: dt at open(), v at lat() & lon(), and local variables at data() * #1855 Check if ldf is null pointer * #1855 Removed break after exit * #1855 Removed break after exit * Update pcp_combine.cc * Feature 1864 config urls (#1868) * Per #1864, remove stale GitHub pages URL for the MET User's Guide. * Per #1864, found stale GitHub IO links in the issue templates. Should make this same fix in the develop and default branches for all the METplus repos. * Per #1864, fix stale GitHub IO link in the PR template. * Move ROC curve in Appendix C * Crop ROC image for better PDF formatting * Shrink ROC image for better PDF formatting * Modify figure alignment in conf.py * Change text placement for ROC curve figure * Feature 1834 faq (#1867) * testing possible new drop-down menus for FAQ * adding sphinx-panels to extensions to be able to use drop down menus in appendixA * Per #1834, adding sphinx-panels to requirements.txt to see if that fixes the error * Per #1834, added sphinx.ext to beginning of sphinx_panels to see if that fixes the error * Per #1834, remove sphinx.ext and add trailing comma to end of extensions to see if that fixed the error * Per #1834, added sphinx-panels to pip install command to see if that fixes the error * Per #1834, added extra space in comment to get docs to build to see if change to documentation.yml will fix an error * Per #1834, added return character to get rid of warning * testing dropdown menus #1 * testing dropdown menus #2 removing lines and dotted lines * testing #3 * testing 4 * fixing numbering, removing 2nd pulldown menu, trying to fix box width * adding code block in dropdown * trying to fix width * trying to fix width with spacing * trying to fix width with spacing 2nd dropdown * removing panels * solved problem, just use dropdown no panels * adding second dropdown menu * removing the dropdown menus and fixing the spacing. * fixing typos * adding in matched pairs * adding in formating files for NetCDF and making FILE_IO a header * fixing spacing * time slice * fixing spacing * UNIX time conversion * fixing spacing * fixed-width * scientific notation * removing which per Julie P * adding Gen_Vx_Mask section * fixing formatting * changing to numbers * complex masking region * neighborhood methods boundaries * Neighborhood Methods to Compute Fraction * adding Grid_Stat - How do I use Config File Setup to Read a NetCDF file * using backticks with asterics * one more backtick with asterics * Grid_Stat - What would be an example of Verifying Probabilities * fixing typos * What is an example of using Grid-Stat with Regridding and Masking Turned On * fixing typos * Grid_Stat - What would be an example of Verifying Probabilities Example2 * trying to fix typos * trying to fix code-block spacing * Per #1834, attempting to fix warning. * How do I use different masks in MET tools using MODE as an example * trying to fix bullet list * trying to fix bullet list 2 * trying to fix bullet list 2 * trying to fix bullet list 3 * Per #1834, attempting formatting changes * Per #1834, attempting formatting changes * Per #1834, attempting formatting changes * Per #1834, attempting formatting changes * Per #1834, attempting formatting changes * Per #1834, modified formatting, including adding backslashes and removing extra returns in command line calls, and adding indentation for config file notation, and adding a return after each 'A.' * Per #1834, changed code-block from ini to none to fix formatting * *Pcp-Combine - What are some examples using -add * Pcp-Combine - How do I add and subtract with Pcp-Combine * Pcp-Combine - How do I Combine 12-hour Accumulated Precipitation from Two Different Initialization Times * Pcp-Combine - How Do I Correct a Precipitation Time Range * Pcp-Combine - What Data Formats does MET Read * Pcp-Combine - How does pass through work * Pcp_Combine - How do I use -pcprx to run a project faster * Pcp-Combine - How Do I Enter the Time Format Correctly * Pcp-Combine - How do I use -subtract * fixing formatting * Pcp_Combine - How Do I Use -sum, -add, and -subtract to Achieve Same Accumulation Interval * Pcp-Combine - What is the difference between -sum vs. -add * Pcp-Combine - How Do I Select a Specific GRIB Record to Use * fixing spacing for original text * Plot-Data-Plane - How Do I Inspect Gen_Vx_Mask Output * fixing typo * Plot-Data-Plane - How Do I Specify GRIB Version * Plot-Data-Plane - last 2 entries * 2 misc. entries * fixing typo * Stat_Analysis - How does -aggregate_stat work * Stat_Analysis - remaining entries * fixing typo * tc-pairs * tc-stat * troubleshooting area * another stat-analysis * another stat-analysis * utilities * fixing dashes * Per #1834, trying out syntax * Per #1834, removing added syntax * fixing returns with bolding * Per #1834, modified formatting * adding italics for directories #1834 * replacing code-block ini with none to remove red lettering * fixing questions to lower case #1834 * linking to section using numref #1834 * linking to section using numref #1834 attempt 2 * linking to section using numref #1834 attempt 3 * linking to section using numref #1834 attempt 4 * linking to section using numref #1834 attempt 5 * linking to section using numref #1834 attempt 6 * linking to section using numref #1834 attempt 7 * linking to section using numref #1834 attempt 8 * linking to anothre section #1834 * linking to another section #1834 * removing old info #1834 * grammar updates #1834 * grammar updates #1834 * fixing typos #1834 * email changes with Julie #1834 * trying to fix warning #1834 * changes #1834 * Per #1834, updating question titles and answers * Per #1834, made updates to the content. * Per #1834, made updates to the content * Per #1834, made updates to the content * Per #1834, made some major changes to the first 3 FAQ's. I don't have the time/bandwidth to review these completely at this time, but wanted to include at least these updates. * testing some of the updates John O. suggested. * updates John O. suggested take 2 #1834. * updates John O. suggested take 3 #1834. * updates John O. suggested take 4 #1834. * updates John O. suggested take 5 #1834. * updates John O. suggested. Some didn't show up even though they've been removed #1834. * trying to fix numbered list indenting #1834 * fixing numbered list indenting #1834 * cleaning up formatting #1834 * fixing broken links #1834 Co-authored-by: Julie.Prestopnik <jpresto@ucar.edu> Co-authored-by: John Halley Gotway <johnhg@ucar.edu> * bugfix 1872 plot_tcmpr.R -lookin directory (develop) (#1874) * Bugfix 1875 develop memory (#1877) * Per #1875, fix set_cdf_ta() to initialize cdf_ta prior to populating it. * Per #1875, add debug(4) log messages whenever defining a new aggregation case in stat-analysis. * Per #1875, switch log messages about new cases from Debug(4) to Debug(3). * Per #1879, porting the same changes over to the develop branch. (#1881) * Feature 1673 gbeta (#1857) * Per #1673, add a distance_map.beta_value entry to all the Grid-Stat config files. * Per #1673, enhance Grid-Stat to parse the new beta_value option and set its value in the library code. * Per #1673, set the user-defined beta_value when processing the distance map scores. * Per #1673, add 4 new DMAP columns write them. Just writing bad data values right now, still need to actually compute them! Also need to update Stat-Analysis to update the parsing/aggregation of these new values. * Per #1673, enable beta_value to be set to a default bad data value. * Per #1673 and @ericgilleland instruction (#1673 (comment)), strip asymmetric G-Beta (AGBETA) from the output. * Per #1673, fix typo that was causing the code not to compile. * Per #1673, update MET User's Guide about G/GBETA. * Per #1673, report BETA_VALUE = TOTAL / 2 in the output instead of NA, as directed by Eric. * Per #1673, remove descriptions of Grid-Stat-specific config options for nbrhd, fourier, gradient, and distance_map from the overview section since they belong in the Grid-Stat chapter. Also add direct links for them in the Grid-Stat chapter. * Per #1673, do not reset the beta_value in DMAP::clear(). * Per #1673, log the DMAP options only after the beta_value has been determined. * Per #1673, correct the default beta_value setting as N*N / 2.0. * Per #1673, trying to format G-beta well. * Per #1673, trying to format G-beta well. * Per #1673, beta is formatted as \beta and not \Beta in latex. * Per #1673, beta is formatted as \beta and not \Beta in latex. * Per #1673, switch to defining beta_value as a function of the input grid size. * Per #1673, push initial appendix C updates for GBETA. * Per #1673, working on G and GBETA equations. * Per #1673, working on G and GBETA equations. * Per 1673, update GBETA details in appendix C. * Per #1673, add Eric's info about selecting beta. * Per #1673, update reference for Gilleland-2019 to Gilleland_PartI-2020 and Gilleland_PartII-2020. * Per 1673, fix typo. * Per #1673, making changes requested by Eric. * Per #1673, making changes requested by Eric. * Per #1673, update references based on Eric's feedback. * Update appendixD.rst (#1869) Updated this appendix. * Update appendixD.rst * Per #1673, making the formatting of Gilleland-2020 part I and II references consistent. * Per #1673, update Grid-Stat docs to clarify that GBETA is only computed on the FULL verification domain and not any masking regions. * Per #1673, add DataPlane::n_good_data() function to return a count valid data values. * Per #1673, update vx_statistics library code to only set gbeta/beta_value to bad data for sub-domains. * Per #1673, update grid_stat to mask out bad data between the fcst/obs fields before computing distance maps. This will change existing unit test results! Also store the count of valid data values in the full verification domain. * Per #1673, since GBETA is only reported when VX_MASK = FULL, updating GridStatConfig_all to include the FULL masking region to make this example more meaningful. Co-authored-by: ericgilleland <34106925+ericgilleland@users.noreply.github.com> * Feature 1764 point stat (#1885) * Per #1764 added write_orank_row() to do_hera_ens(). SL * Per issue #1764, added orank stat type for ensemble output. SL * Per issue #1764, added code to setup_txt_files() for ORANK output. SL * Added some comments for debuging. SL * Per #1764, add orank to the list of outputs created by point_stat. * Per #1764, add functions to compute the number of requested HiRA probabilities and ensemble members. Also update n_txt_row() to keep track of the number of ORANK rows to be written. Not totally sure if I'm counting these ORANK output lines correctly though. * Per #1764, update logic in setup_txt_files() to call get_max_n_hira_prob() and get_max_n_hira_ens() functions. * Per #1764, no real changes. Just fixing spacing. * Per issue #1764, cleaned up print (cout) debug statements. SL * Per issue #1764 added 'orank = NONE' to the config files for compatibility with changes to point_stat for calculating ORANK. SL * Per issue #1764 added 'orank = STAT' to config file to produce ORANK output lines in point_stat. SL * Per issue #1764 added content in all relevant sections for ORANK: observation rank statistic. SL * Added 'orank = NONE' to config file for compatibility with changes to point_stant for ORANK. SL * Added 'orank = NONE' to config file for compatibility with changes to point_stant for ORANK. SL * Apply suggestions from code review Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> * Per #1764, no real changes, just fixing indents. * Per #1764, no real changes, just fixing indents. * Per #1764, no real changes, just fixing indents. * Per #1764, putting the docs for VCNT back into its logical order with the other vector line types. The order that they're listed here does not need to match their order in the enumeration in the code. * Per #1764, fixing a vcnt typo I found in the grid-stat chapter... and making them consistent. * Per issue #1764: After the call to write_orank_row, added lines to reset the observation begin and end times. SL * Per #1764, no code changes, just fixing indents. Co-authored-by: Seth Linden <linden@kiowa.rap.ucar.edu> Co-authored-by: John Halley Gotway <johnhg@ucar.edu> Co-authored-by: John Halley Gotway <johnhg@kiowa.rap.ucar.edu> Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> * Adding Seth to the NB notification list. * Bugfix 1886 develop grid_diag (#1888) * Per #1886, port over the bugfixes from main_v10.0 to develop. * Per #1886, make grid_diag error out if to_grid is set to FCST or OBS. Co-authored-by: jprestop <jpresto@ucar.edu> Co-authored-by: Howard Soh <hsoh@kiowa.rap.ucar.edu> Co-authored-by: hsoh-u <hsoh@ucar.edu> Co-authored-by: Keith Searight <keith.searight@noaa.gov> Co-authored-by: Seth Linden <linden@ucar.edu> Co-authored-by: Seth Linden <linden@kiowa.rap.ucar.edu> Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> Co-authored-by: lisagoodrich <33230218+lisagoodrich@users.noreply.github.com> Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> Co-authored-by: ericgilleland <34106925+ericgilleland@users.noreply.github.com> Co-authored-by: John Halley Gotway <johnhg@kiowa.rap.ucar.edu> Co-authored-by: MET Tools Test Account <met_test@kiowa.rap.ucar.edu>
JohnHalleyGotway
added a commit
that referenced
this pull request
Aug 30, 2021
* Update install_met_env.kiowa Commented out items and added text to description * #1817 Added to_north argument at two_to_one() * #1817 Calls two_to_one() with false for the third argument, to_north, if latitude moves north to south * Feature 1819 automation doc warnings (#1836) * Removing "/lib" that should not be there. * #1508 checking stat() methoid returns no error before looking at FILE flag * Enable PDF creation * Modify inputenc, fontenc latex_elements * Fix typo in conf.py * Replace unicode with ascii * Replace unicode exponent with math mode * Change eqnarray to align * Replace gif images with png for PDF * Fix png file names * Escape underscore in variable name in math mode * Escape underscore in variable name in math mode * Remove underscores from variables in math mode * Replace bar with overline in some equations * Test derefencing underscores in tables * Test math changes in grid stat * Test math formatting * Test math formatting * #1838 Added log message for lat/lon values * #1838 Give warning if the first and lat lat/lon is same * #1838 Support the double data type variable on reading float type values * Feature 1833 develop discussions (#1849) * Per #1833, changed references to met_help to Discussions. * Per #1833, fixed typo and removed an unnecessary word * #1852 Removed unused nc_buf_size. Make cur and dim to the same size (called API handles 2D or 3D) * #1852 Check the start and count before calling NetCDF API * Feature 1746 wavelet stat (#1851) * For issue #1746 modified code to allow users to pass in an empty list (or NA) for forecast and observation thresholds in order to skip applying the threhsolds, but it will still compute stats with the raw fields. SL * For issue #1746, added new unit test that uses a config file that has empty lists for the forecast and observation thresholds. SL * For issue #1746 Added some content related to allowing users to set forecast and observation cat thresholds to an empty list in order to skip the binary masking (and consider all grid-points for stats). SL * Per #1746, cleaning up for consistent indentation. * Per #1746, cleaning up for consistent indentation. * Per #1746, add a revision history note, update the plotting range in the postscript output to be [-n,n] where n is the maximum value of the maximum absolute difference and 1.0, and also fix a bug. When the NA threshold comes AFTER a real threshold, the resulting data and difference values were not being updated. * Per #1746, change the Wavelet-Stat config file values in the the wvlt_plot dictionary by setting plot_min = plot_max = 0.0. That enables the default logic of the tool to take effect. Choose the plotting range of the wavelet plots as [-n,n], where n is the maximum of 1.0 and the maximum absolute difference. * Per #1746, used apply_fcst_thresh where it should have been apply_obs_thresh. * Per issue #1746, modified some content related to users being able to skip applying the categorical threhsolds by putting an empty list or NA in the configuration file. SL * Per issue #1746 Added some warnings if the forecast threshold is set to NA but the observation threshold is not NA (a numeric threshold) and vice versa. SL * Per #1746, fix a couple of typos and tweak wording in the wavelet-stat chapter. * Per #1746, loop over each pair of fcst/obs thresholds to check for inconsistent use of the NA threshold type. * Per #1746, a bit of code cleanup replacing calls to n_elements() with n() to make the code more concise. * Per #1746, need to reinitialize apply_fcst_thresh and apply_obs_thresh to true inside the loop since the NA threshold can appear anywhere in the list of thresholds. Co-authored-by: Seth Linden <linden@kiowa.rap.ucar.edu> Co-authored-by: John Halley Gotway <johnhg@ucar.edu> * Add sphinx.ext.pngmath to conf.py * Fix typo in conf.py * Add sphinx.ext.mathjax to conf.py * Add sphinx.ext.imgmath to conf.py * Fix typo in conf.py * Change math formatting in Appendix C * Temporarily disable equations with errors in Appendix C * Remove sphinx.ext.imgmath in conf.py * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Add sphinx.ext.imgmath to conf.py * Modify math formatting in Appendix C * Temporarily disable equations with errors in Appendix C * Remove sphinx.ext.imgmath in conf.py * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Temporarily disable equations with errors * Update README.md * Modify math formatting in Appendix C * Add usepackage{amssymb} to conf.py * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Fix warnings in wavelet-stat.rst * Omit Indices and tables in PDF version of User Guide * Modify math formatting in Appendix G * Modify math formatting in Appendix E * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Change release date in conf.py * #1852 get_string_val: check if 2D variable. Also checking if the variable does not have dimension (#1860) Co-authored-by: Howard Soh <hsoh@kiowa.rap.ucar.edu> * Feature 1453 create pdf user guide (#1859) * Enable PDF creation * Modify inputenc, fontenc latex_elements * Fix typo in conf.py * Replace unicode with ascii * Replace unicode exponent with math mode * Change eqnarray to align * Replace gif images with png for PDF * Fix png file names * Escape underscore in variable name in math mode * Escape underscore in variable name in math mode * Remove underscores from variables in math mode * Replace bar with overline in some equations * Test derefencing underscores in tables * Test math changes in grid stat * Test math formatting * Test math formatting * Add sphinx.ext.pngmath to conf.py * Fix typo in conf.py * Add sphinx.ext.mathjax to conf.py * Add sphinx.ext.imgmath to conf.py * Fix typo in conf.py * Change math formatting in Appendix C * Temporarily disable equations with errors in Appendix C * Remove sphinx.ext.imgmath in conf.py * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Add sphinx.ext.imgmath to conf.py * Modify math formatting in Appendix C * Temporarily disable equations with errors in Appendix C * Remove sphinx.ext.imgmath in conf.py * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Temporarily disable equations with errors * Modify math formatting in Appendix C * Add usepackage{amssymb} to conf.py * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Fix warnings in wavelet-stat.rst * Omit Indices and tables in PDF version of User Guide * Modify math formatting in Appendix G * Modify math formatting in Appendix E * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Change release date in conf.py Co-authored-by: John Halley Gotway <johnhg@ucar.edu> * Feature 1853 rps doc (#1861) * Per #1853, adding RPS docs from Eric. * Per #1853, work in progress getting the formatting correct. * Per #1853, work in progress getting the formatting correct. * Per #1853, work in progress getting the formatting correct. * Per #1853, work in progress getting the formatting correct. * Per #1853, adding a manual page break after the ROC image to get it to stay in the right spot. * Per #1853, backing out last change which didn't make the ROC image stay in the right spot. * #1855 Removed break after return (SobarQube) * #1855 Removed break after exit * 1855 Added return at MetNcPointObsOut::write_to_netcdf * #1855 Avoid releasing memory twice * #1855 Removed unused code * #1855 Corrected syntax error * #1855 Changed the second call of close() to release _ncFile * #1855 Deleted break after exit * #1855 Chamnged while to if * #1855 Added constructor TCLineCounts and initialized the members * #1855 Added break statement back for case '-' * #1855 Check the new size before extending to avoid thereference of the null pointer * #1855 Changed count to processed_count for log messages. Initialized variables at get_filtered_nc_data\*() * #1855 Corected the number of memory copy * #1855 Exit with an error message if both fgi & bgi are null * #1855 Clear Lead_Times before extening. Formatting for error messages * Add definition for the compilation environment on seneca. * #1855 Extend Lead_Times for new times * #1855 Removed break after exit * #1855 define "c" dynamically * #1855 Move the log message within if statement to avoid dereferencing a null pointer * Feature 1843 scatter index (#1863) * Added scatter index variables: SI, SI_BCL, SI_BCU to cnt columns. SL * Per issue #1843 Added Scatter Index variables (SI, SI_BCL, SI_BCU) to the CNT stat type. SL * Added Scatter Index (SI, SI_BCL, SI_BCU) to cnt stat type header. SL * Per issue #1843, added Scatter Index variables to write_cnt_cols(). SL * Per issue #1843 added Scatter Index (si) to CNTinfo class. SL * Per issue #1843 added Scatter Index (SI, si) to CNTInfo::get_stat(). SL * Per issue #1843 added Scatter Index (SI, si) calculation to both versions of compute_cntinfo(). SL * Per issue #1843, forgot semi-colon after SI calculation, fixed. SL * Added Scatter Index (si) to some of the clear() and allocate() functions. SL * Per issue #1843, added Scatter Index (si) to compute_cnt_mean(). SL * Per issue #1843, added Scatter Index (si) to bootstrap interval calculation and to the CNT write function. SL * Per issue #1843 added Scatter Index (SI, SI_BCL, SI_BCU) to store_stat_cnt(). SL * Per issue #1843, in get_stat() moved Scatterd Index (SI) line to be below RMSE for consitency. SL * Added Scatter Index (SI) to the CNT output format table. SL * Added Scatter Index (SI) to CNT statistics definitions. SL * Update appendixC.rst * Update stat_columns.cc * Update compute_ci.cc * Update met_stats.cc * Per #1843, update write_cntinfo() function. Since we're READING the SI value immediately after the RMSE value when computing CI's, we also need to write SI immediately after RMSE. Otherwise, we'll be computing CI's using the wrong statistics replicates. * Per #1843, check for divide by zero when computing SI... this could happen if comparing the same input file to itself with Grid-Stat. * Per #1843, correct definition of SI from RMSE/ME to RMSE/OBAR. * Per #1843, updated the SI definition to divide by OBAR instead of ME, but forgot to update the divide-by-zero check. Co-authored-by: Seth Linden <linden@kiowa.rap.ucar.edu> Co-authored-by: johnhg <johnhg@ucar.edu> * #1855 Corected typo - compare the numnber of U and V * #1855 Check if var_info is hull * #1855 Avoid un-initialized variables: dt at open(), v at lat() & lon(), and local variables at data() * #1855 Check if ldf is null pointer * #1855 Removed break after exit * #1855 Removed break after exit * Update pcp_combine.cc * Feature 1864 config urls (#1868) * Per #1864, remove stale GitHub pages URL for the MET User's Guide. * Per #1864, found stale GitHub IO links in the issue templates. Should make this same fix in the develop and default branches for all the METplus repos. * Per #1864, fix stale GitHub IO link in the PR template. * Move ROC curve in Appendix C * Crop ROC image for better PDF formatting * Shrink ROC image for better PDF formatting * Modify figure alignment in conf.py * Change text placement for ROC curve figure * Feature 1834 faq (#1867) * testing possible new drop-down menus for FAQ * adding sphinx-panels to extensions to be able to use drop down menus in appendixA * Per #1834, adding sphinx-panels to requirements.txt to see if that fixes the error * Per #1834, added sphinx.ext to beginning of sphinx_panels to see if that fixes the error * Per #1834, remove sphinx.ext and add trailing comma to end of extensions to see if that fixed the error * Per #1834, added sphinx-panels to pip install command to see if that fixes the error * Per #1834, added extra space in comment to get docs to build to see if change to documentation.yml will fix an error * Per #1834, added return character to get rid of warning * testing dropdown menus #1 * testing dropdown menus #2 removing lines and dotted lines * testing #3 * testing 4 * fixing numbering, removing 2nd pulldown menu, trying to fix box width * adding code block in dropdown * trying to fix width * trying to fix width with spacing * trying to fix width with spacing 2nd dropdown * removing panels * solved problem, just use dropdown no panels * adding second dropdown menu * removing the dropdown menus and fixing the spacing. * fixing typos * adding in matched pairs * adding in formating files for NetCDF and making FILE_IO a header * fixing spacing * time slice * fixing spacing * UNIX time conversion * fixing spacing * fixed-width * scientific notation * removing which per Julie P * adding Gen_Vx_Mask section * fixing formatting * changing to numbers * complex masking region * neighborhood methods boundaries * Neighborhood Methods to Compute Fraction * adding Grid_Stat - How do I use Config File Setup to Read a NetCDF file * using backticks with asterics * one more backtick with asterics * Grid_Stat - What would be an example of Verifying Probabilities * fixing typos * What is an example of using Grid-Stat with Regridding and Masking Turned On * fixing typos * Grid_Stat - What would be an example of Verifying Probabilities Example2 * trying to fix typos * trying to fix code-block spacing * Per #1834, attempting to fix warning. * How do I use different masks in MET tools using MODE as an example * trying to fix bullet list * trying to fix bullet list 2 * trying to fix bullet list 2 * trying to fix bullet list 3 * Per #1834, attempting formatting changes * Per #1834, attempting formatting changes * Per #1834, attempting formatting changes * Per #1834, attempting formatting changes * Per #1834, attempting formatting changes * Per #1834, modified formatting, including adding backslashes and removing extra returns in command line calls, and adding indentation for config file notation, and adding a return after each 'A.' * Per #1834, changed code-block from ini to none to fix formatting * *Pcp-Combine - What are some examples using -add * Pcp-Combine - How do I add and subtract with Pcp-Combine * Pcp-Combine - How do I Combine 12-hour Accumulated Precipitation from Two Different Initialization Times * Pcp-Combine - How Do I Correct a Precipitation Time Range * Pcp-Combine - What Data Formats does MET Read * Pcp-Combine - How does pass through work * Pcp_Combine - How do I use -pcprx to run a project faster * Pcp-Combine - How Do I Enter the Time Format Correctly * Pcp-Combine - How do I use -subtract * fixing formatting * Pcp_Combine - How Do I Use -sum, -add, and -subtract to Achieve Same Accumulation Interval * Pcp-Combine - What is the difference between -sum vs. -add * Pcp-Combine - How Do I Select a Specific GRIB Record to Use * fixing spacing for original text * Plot-Data-Plane - How Do I Inspect Gen_Vx_Mask Output * fixing typo * Plot-Data-Plane - How Do I Specify GRIB Version * Plot-Data-Plane - last 2 entries * 2 misc. entries * fixing typo * Stat_Analysis - How does -aggregate_stat work * Stat_Analysis - remaining entries * fixing typo * tc-pairs * tc-stat * troubleshooting area * another stat-analysis * another stat-analysis * utilities * fixing dashes * Per #1834, trying out syntax * Per #1834, removing added syntax * fixing returns with bolding * Per #1834, modified formatting * adding italics for directories #1834 * replacing code-block ini with none to remove red lettering * fixing questions to lower case #1834 * linking to section using numref #1834 * linking to section using numref #1834 attempt 2 * linking to section using numref #1834 attempt 3 * linking to section using numref #1834 attempt 4 * linking to section using numref #1834 attempt 5 * linking to section using numref #1834 attempt 6 * linking to section using numref #1834 attempt 7 * linking to section using numref #1834 attempt 8 * linking to anothre section #1834 * linking to another section #1834 * removing old info #1834 * grammar updates #1834 * grammar updates #1834 * fixing typos #1834 * email changes with Julie #1834 * trying to fix warning #1834 * changes #1834 * Per #1834, updating question titles and answers * Per #1834, made updates to the content. * Per #1834, made updates to the content * Per #1834, made updates to the content * Per #1834, made some major changes to the first 3 FAQ's. I don't have the time/bandwidth to review these completely at this time, but wanted to include at least these updates. * testing some of the updates John O. suggested. * updates John O. suggested take 2 #1834. * updates John O. suggested take 3 #1834. * updates John O. suggested take 4 #1834. * updates John O. suggested take 5 #1834. * updates John O. suggested. Some didn't show up even though they've been removed #1834. * trying to fix numbered list indenting #1834 * fixing numbered list indenting #1834 * cleaning up formatting #1834 * fixing broken links #1834 Co-authored-by: Julie.Prestopnik <jpresto@ucar.edu> Co-authored-by: John Halley Gotway <johnhg@ucar.edu> * bugfix 1872 plot_tcmpr.R -lookin directory (develop) (#1874) * Bugfix 1875 develop memory (#1877) * Per #1875, fix set_cdf_ta() to initialize cdf_ta prior to populating it. * Per #1875, add debug(4) log messages whenever defining a new aggregation case in stat-analysis. * Per #1875, switch log messages about new cases from Debug(4) to Debug(3). * Per #1879, porting the same changes over to the develop branch. (#1881) * Feature 1673 gbeta (#1857) * Per #1673, add a distance_map.beta_value entry to all the Grid-Stat config files. * Per #1673, enhance Grid-Stat to parse the new beta_value option and set its value in the library code. * Per #1673, set the user-defined beta_value when processing the distance map scores. * Per #1673, add 4 new DMAP columns write them. Just writing bad data values right now, still need to actually compute them! Also need to update Stat-Analysis to update the parsing/aggregation of these new values. * Per #1673, enable beta_value to be set to a default bad data value. * Per #1673 and @ericgilleland instruction (#1673 (comment)), strip asymmetric G-Beta (AGBETA) from the output. * Per #1673, fix typo that was causing the code not to compile. * Per #1673, update MET User's Guide about G/GBETA. * Per #1673, report BETA_VALUE = TOTAL / 2 in the output instead of NA, as directed by Eric. * Per #1673, remove descriptions of Grid-Stat-specific config options for nbrhd, fourier, gradient, and distance_map from the overview section since they belong in the Grid-Stat chapter. Also add direct links for them in the Grid-Stat chapter. * Per #1673, do not reset the beta_value in DMAP::clear(). * Per #1673, log the DMAP options only after the beta_value has been determined. * Per #1673, correct the default beta_value setting as N*N / 2.0. * Per #1673, trying to format G-beta well. * Per #1673, trying to format G-beta well. * Per #1673, beta is formatted as \beta and not \Beta in latex. * Per #1673, beta is formatted as \beta and not \Beta in latex. * Per #1673, switch to defining beta_value as a function of the input grid size. * Per #1673, push initial appendix C updates for GBETA. * Per #1673, working on G and GBETA equations. * Per #1673, working on G and GBETA equations. * Per 1673, update GBETA details in appendix C. * Per #1673, add Eric's info about selecting beta. * Per #1673, update reference for Gilleland-2019 to Gilleland_PartI-2020 and Gilleland_PartII-2020. * Per 1673, fix typo. * Per #1673, making changes requested by Eric. * Per #1673, making changes requested by Eric. * Per #1673, update references based on Eric's feedback. * Update appendixD.rst (#1869) Updated this appendix. * Update appendixD.rst * Per #1673, making the formatting of Gilleland-2020 part I and II references consistent. * Per #1673, update Grid-Stat docs to clarify that GBETA is only computed on the FULL verification domain and not any masking regions. * Per #1673, add DataPlane::n_good_data() function to return a count valid data values. * Per #1673, update vx_statistics library code to only set gbeta/beta_value to bad data for sub-domains. * Per #1673, update grid_stat to mask out bad data between the fcst/obs fields before computing distance maps. This will change existing unit test results! Also store the count of valid data values in the full verification domain. * Per #1673, since GBETA is only reported when VX_MASK = FULL, updating GridStatConfig_all to include the FULL masking region to make this example more meaningful. Co-authored-by: ericgilleland <34106925+ericgilleland@users.noreply.github.com> * Feature 1764 point stat (#1885) * Per #1764 added write_orank_row() to do_hera_ens(). SL * Per issue #1764, added orank stat type for ensemble output. SL * Per issue #1764, added code to setup_txt_files() for ORANK output. SL * Added some comments for debuging. SL * Per #1764, add orank to the list of outputs created by point_stat. * Per #1764, add functions to compute the number of requested HiRA probabilities and ensemble members. Also update n_txt_row() to keep track of the number of ORANK rows to be written. Not totally sure if I'm counting these ORANK output lines correctly though. * Per #1764, update logic in setup_txt_files() to call get_max_n_hira_prob() and get_max_n_hira_ens() functions. * Per #1764, no real changes. Just fixing spacing. * Per issue #1764, cleaned up print (cout) debug statements. SL * Per issue #1764 added 'orank = NONE' to the config files for compatibility with changes to point_stat for calculating ORANK. SL * Per issue #1764 added 'orank = STAT' to config file to produce ORANK output lines in point_stat. SL * Per issue #1764 added content in all relevant sections for ORANK: observation rank statistic. SL * Added 'orank = NONE' to config file for compatibility with changes to point_stant for ORANK. SL * Added 'orank = NONE' to config file for compatibility with changes to point_stant for ORANK. SL * Apply suggestions from code review Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> * Per #1764, no real changes, just fixing indents. * Per #1764, no real changes, just fixing indents. * Per #1764, no real changes, just fixing indents. * Per #1764, putting the docs for VCNT back into its logical order with the other vector line types. The order that they're listed here does not need to match their order in the enumeration in the code. * Per #1764, fixing a vcnt typo I found in the grid-stat chapter... and making them consistent. * Per issue #1764: After the call to write_orank_row, added lines to reset the observation begin and end times. SL * Per #1764, no code changes, just fixing indents. Co-authored-by: Seth Linden <linden@kiowa.rap.ucar.edu> Co-authored-by: John Halley Gotway <johnhg@ucar.edu> Co-authored-by: John Halley Gotway <johnhg@kiowa.rap.ucar.edu> Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> * Adding Seth to the NB notification list. * Bugfix 1886 develop grid_diag (#1888) * Per #1886, port over the bugfixes from main_v10.0 to develop. * Per #1886, make grid_diag error out if to_grid is set to FCST or OBS. * Added additional default labels * Feature 1870 realtime (#1893) * Per #1870. add TCPairs config entries for valid_inc, valid_exc, and write_valid. * Per #1870, removing 2 files I accidentally added. * Per #1870, add code to parse the new tc_pairs config options. * Per #1870, unrelated... just fixing spacing. * Per #1870, update TrackPairInfo/Array classes and tc_pairs application code to use the write_valid option to subset the tracks by requested valid time before writing them. * Per #1870, update the User's Guide with the new config options. Document valid_inc/valid_exc in the common TC config section and add write_valid to the TC-Pairs chapter. * Per #1870, move the subset_write_valid() functionality to the library code and support it for both track and probability arrays. * Per #1870, update unit_tc_pairs.xml with 2 new calls to exercise the new write_valid functionality for both tracks and probabilities. * Per #1870, correct some typos in the docs... PROBRI and probri should be PROBRIRW and probrirw, resp. * Per #1870, switching match_points from TRUE back to FALSE after I acciedentally committed that change. * Per #1870, just fixing indent. Co-authored-by: John Halley Gotway <johnhg@kiowa.rap.ucar.edu> Co-authored-by: jprestop <jpresto@ucar.edu> Co-authored-by: Howard Soh <hsoh@kiowa.rap.ucar.edu> Co-authored-by: hsoh-u <hsoh@ucar.edu> Co-authored-by: Keith Searight <keith.searight@noaa.gov> Co-authored-by: Seth Linden <linden@ucar.edu> Co-authored-by: Seth Linden <linden@kiowa.rap.ucar.edu> Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> Co-authored-by: lisagoodrich <33230218+lisagoodrich@users.noreply.github.com> Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> Co-authored-by: ericgilleland <34106925+ericgilleland@users.noreply.github.com> Co-authored-by: John Halley Gotway <johnhg@kiowa.rap.ucar.edu> Co-authored-by: MET Tools Test Account <met_test@kiowa.rap.ucar.edu> Co-authored-by: Julie Prestopnik <jpresto@seneca.rap.ucar.edu>
JohnHalleyGotway
added a commit
that referenced
this pull request
Aug 31, 2021
* Update install_met_env.kiowa Commented out items and added text to description * #1817 Added to_north argument at two_to_one() * #1817 Calls two_to_one() with false for the third argument, to_north, if latitude moves north to south * Feature 1819 automation doc warnings (#1836) * Removing "/lib" that should not be there. * #1508 checking stat() methoid returns no error before looking at FILE flag * Enable PDF creation * Modify inputenc, fontenc latex_elements * Fix typo in conf.py * Replace unicode with ascii * Replace unicode exponent with math mode * Change eqnarray to align * Replace gif images with png for PDF * Fix png file names * Escape underscore in variable name in math mode * Escape underscore in variable name in math mode * Remove underscores from variables in math mode * Replace bar with overline in some equations * Test derefencing underscores in tables * Test math changes in grid stat * Test math formatting * Test math formatting * #1838 Added log message for lat/lon values * #1838 Give warning if the first and lat lat/lon is same * #1838 Support the double data type variable on reading float type values * Feature 1833 develop discussions (#1849) * Per #1833, changed references to met_help to Discussions. * Per #1833, fixed typo and removed an unnecessary word * #1852 Removed unused nc_buf_size. Make cur and dim to the same size (called API handles 2D or 3D) * #1852 Check the start and count before calling NetCDF API * Feature 1746 wavelet stat (#1851) * For issue #1746 modified code to allow users to pass in an empty list (or NA) for forecast and observation thresholds in order to skip applying the threhsolds, but it will still compute stats with the raw fields. SL * For issue #1746, added new unit test that uses a config file that has empty lists for the forecast and observation thresholds. SL * For issue #1746 Added some content related to allowing users to set forecast and observation cat thresholds to an empty list in order to skip the binary masking (and consider all grid-points for stats). SL * Per #1746, cleaning up for consistent indentation. * Per #1746, cleaning up for consistent indentation. * Per #1746, add a revision history note, update the plotting range in the postscript output to be [-n,n] where n is the maximum value of the maximum absolute difference and 1.0, and also fix a bug. When the NA threshold comes AFTER a real threshold, the resulting data and difference values were not being updated. * Per #1746, change the Wavelet-Stat config file values in the the wvlt_plot dictionary by setting plot_min = plot_max = 0.0. That enables the default logic of the tool to take effect. Choose the plotting range of the wavelet plots as [-n,n], where n is the maximum of 1.0 and the maximum absolute difference. * Per #1746, used apply_fcst_thresh where it should have been apply_obs_thresh. * Per issue #1746, modified some content related to users being able to skip applying the categorical threhsolds by putting an empty list or NA in the configuration file. SL * Per issue #1746 Added some warnings if the forecast threshold is set to NA but the observation threshold is not NA (a numeric threshold) and vice versa. SL * Per #1746, fix a couple of typos and tweak wording in the wavelet-stat chapter. * Per #1746, loop over each pair of fcst/obs thresholds to check for inconsistent use of the NA threshold type. * Per #1746, a bit of code cleanup replacing calls to n_elements() with n() to make the code more concise. * Per #1746, need to reinitialize apply_fcst_thresh and apply_obs_thresh to true inside the loop since the NA threshold can appear anywhere in the list of thresholds. Co-authored-by: Seth Linden <linden@kiowa.rap.ucar.edu> Co-authored-by: John Halley Gotway <johnhg@ucar.edu> * Add sphinx.ext.pngmath to conf.py * Fix typo in conf.py * Add sphinx.ext.mathjax to conf.py * Add sphinx.ext.imgmath to conf.py * Fix typo in conf.py * Change math formatting in Appendix C * Temporarily disable equations with errors in Appendix C * Remove sphinx.ext.imgmath in conf.py * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Add sphinx.ext.imgmath to conf.py * Modify math formatting in Appendix C * Temporarily disable equations with errors in Appendix C * Remove sphinx.ext.imgmath in conf.py * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Temporarily disable equations with errors * Update README.md * Modify math formatting in Appendix C * Add usepackage{amssymb} to conf.py * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Fix warnings in wavelet-stat.rst * Omit Indices and tables in PDF version of User Guide * Modify math formatting in Appendix G * Modify math formatting in Appendix E * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Change release date in conf.py * #1852 get_string_val: check if 2D variable. Also checking if the variable does not have dimension (#1860) Co-authored-by: Howard Soh <hsoh@kiowa.rap.ucar.edu> * Feature 1453 create pdf user guide (#1859) * Enable PDF creation * Modify inputenc, fontenc latex_elements * Fix typo in conf.py * Replace unicode with ascii * Replace unicode exponent with math mode * Change eqnarray to align * Replace gif images with png for PDF * Fix png file names * Escape underscore in variable name in math mode * Escape underscore in variable name in math mode * Remove underscores from variables in math mode * Replace bar with overline in some equations * Test derefencing underscores in tables * Test math changes in grid stat * Test math formatting * Test math formatting * Add sphinx.ext.pngmath to conf.py * Fix typo in conf.py * Add sphinx.ext.mathjax to conf.py * Add sphinx.ext.imgmath to conf.py * Fix typo in conf.py * Change math formatting in Appendix C * Temporarily disable equations with errors in Appendix C * Remove sphinx.ext.imgmath in conf.py * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Add sphinx.ext.imgmath to conf.py * Modify math formatting in Appendix C * Temporarily disable equations with errors in Appendix C * Remove sphinx.ext.imgmath in conf.py * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Temporarily disable equations with errors * Modify math formatting in Appendix C * Add usepackage{amssymb} to conf.py * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Fix warnings in wavelet-stat.rst * Omit Indices and tables in PDF version of User Guide * Modify math formatting in Appendix G * Modify math formatting in Appendix E * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Change release date in conf.py Co-authored-by: John Halley Gotway <johnhg@ucar.edu> * Feature 1853 rps doc (#1861) * Per #1853, adding RPS docs from Eric. * Per #1853, work in progress getting the formatting correct. * Per #1853, work in progress getting the formatting correct. * Per #1853, work in progress getting the formatting correct. * Per #1853, work in progress getting the formatting correct. * Per #1853, adding a manual page break after the ROC image to get it to stay in the right spot. * Per #1853, backing out last change which didn't make the ROC image stay in the right spot. * #1855 Removed break after return (SobarQube) * #1855 Removed break after exit * 1855 Added return at MetNcPointObsOut::write_to_netcdf * #1855 Avoid releasing memory twice * #1855 Removed unused code * #1855 Corrected syntax error * #1855 Changed the second call of close() to release _ncFile * #1855 Deleted break after exit * #1855 Chamnged while to if * #1855 Added constructor TCLineCounts and initialized the members * #1855 Added break statement back for case '-' * #1855 Check the new size before extending to avoid thereference of the null pointer * #1855 Changed count to processed_count for log messages. Initialized variables at get_filtered_nc_data\*() * #1855 Corected the number of memory copy * #1855 Exit with an error message if both fgi & bgi are null * #1855 Clear Lead_Times before extening. Formatting for error messages * Add definition for the compilation environment on seneca. * #1855 Extend Lead_Times for new times * #1855 Removed break after exit * #1855 define "c" dynamically * #1855 Move the log message within if statement to avoid dereferencing a null pointer * Feature 1843 scatter index (#1863) * Added scatter index variables: SI, SI_BCL, SI_BCU to cnt columns. SL * Per issue #1843 Added Scatter Index variables (SI, SI_BCL, SI_BCU) to the CNT stat type. SL * Added Scatter Index (SI, SI_BCL, SI_BCU) to cnt stat type header. SL * Per issue #1843, added Scatter Index variables to write_cnt_cols(). SL * Per issue #1843 added Scatter Index (si) to CNTinfo class. SL * Per issue #1843 added Scatter Index (SI, si) to CNTInfo::get_stat(). SL * Per issue #1843 added Scatter Index (SI, si) calculation to both versions of compute_cntinfo(). SL * Per issue #1843, forgot semi-colon after SI calculation, fixed. SL * Added Scatter Index (si) to some of the clear() and allocate() functions. SL * Per issue #1843, added Scatter Index (si) to compute_cnt_mean(). SL * Per issue #1843, added Scatter Index (si) to bootstrap interval calculation and to the CNT write function. SL * Per issue #1843 added Scatter Index (SI, SI_BCL, SI_BCU) to store_stat_cnt(). SL * Per issue #1843, in get_stat() moved Scatterd Index (SI) line to be below RMSE for consitency. SL * Added Scatter Index (SI) to the CNT output format table. SL * Added Scatter Index (SI) to CNT statistics definitions. SL * Update appendixC.rst * Update stat_columns.cc * Update compute_ci.cc * Update met_stats.cc * Per #1843, update write_cntinfo() function. Since we're READING the SI value immediately after the RMSE value when computing CI's, we also need to write SI immediately after RMSE. Otherwise, we'll be computing CI's using the wrong statistics replicates. * Per #1843, check for divide by zero when computing SI... this could happen if comparing the same input file to itself with Grid-Stat. * Per #1843, correct definition of SI from RMSE/ME to RMSE/OBAR. * Per #1843, updated the SI definition to divide by OBAR instead of ME, but forgot to update the divide-by-zero check. Co-authored-by: Seth Linden <linden@kiowa.rap.ucar.edu> Co-authored-by: johnhg <johnhg@ucar.edu> * #1855 Corected typo - compare the numnber of U and V * #1855 Check if var_info is hull * #1855 Avoid un-initialized variables: dt at open(), v at lat() & lon(), and local variables at data() * #1855 Check if ldf is null pointer * #1855 Removed break after exit * #1855 Removed break after exit * Update pcp_combine.cc * Feature 1864 config urls (#1868) * Per #1864, remove stale GitHub pages URL for the MET User's Guide. * Per #1864, found stale GitHub IO links in the issue templates. Should make this same fix in the develop and default branches for all the METplus repos. * Per #1864, fix stale GitHub IO link in the PR template. * Move ROC curve in Appendix C * Crop ROC image for better PDF formatting * Shrink ROC image for better PDF formatting * Modify figure alignment in conf.py * Change text placement for ROC curve figure * Feature 1834 faq (#1867) * testing possible new drop-down menus for FAQ * adding sphinx-panels to extensions to be able to use drop down menus in appendixA * Per #1834, adding sphinx-panels to requirements.txt to see if that fixes the error * Per #1834, added sphinx.ext to beginning of sphinx_panels to see if that fixes the error * Per #1834, remove sphinx.ext and add trailing comma to end of extensions to see if that fixed the error * Per #1834, added sphinx-panels to pip install command to see if that fixes the error * Per #1834, added extra space in comment to get docs to build to see if change to documentation.yml will fix an error * Per #1834, added return character to get rid of warning * testing dropdown menus #1 * testing dropdown menus #2 removing lines and dotted lines * testing #3 * testing 4 * fixing numbering, removing 2nd pulldown menu, trying to fix box width * adding code block in dropdown * trying to fix width * trying to fix width with spacing * trying to fix width with spacing 2nd dropdown * removing panels * solved problem, just use dropdown no panels * adding second dropdown menu * removing the dropdown menus and fixing the spacing. * fixing typos * adding in matched pairs * adding in formating files for NetCDF and making FILE_IO a header * fixing spacing * time slice * fixing spacing * UNIX time conversion * fixing spacing * fixed-width * scientific notation * removing which per Julie P * adding Gen_Vx_Mask section * fixing formatting * changing to numbers * complex masking region * neighborhood methods boundaries * Neighborhood Methods to Compute Fraction * adding Grid_Stat - How do I use Config File Setup to Read a NetCDF file * using backticks with asterics * one more backtick with asterics * Grid_Stat - What would be an example of Verifying Probabilities * fixing typos * What is an example of using Grid-Stat with Regridding and Masking Turned On * fixing typos * Grid_Stat - What would be an example of Verifying Probabilities Example2 * trying to fix typos * trying to fix code-block spacing * Per #1834, attempting to fix warning. * How do I use different masks in MET tools using MODE as an example * trying to fix bullet list * trying to fix bullet list 2 * trying to fix bullet list 2 * trying to fix bullet list 3 * Per #1834, attempting formatting changes * Per #1834, attempting formatting changes * Per #1834, attempting formatting changes * Per #1834, attempting formatting changes * Per #1834, attempting formatting changes * Per #1834, modified formatting, including adding backslashes and removing extra returns in command line calls, and adding indentation for config file notation, and adding a return after each 'A.' * Per #1834, changed code-block from ini to none to fix formatting * *Pcp-Combine - What are some examples using -add * Pcp-Combine - How do I add and subtract with Pcp-Combine * Pcp-Combine - How do I Combine 12-hour Accumulated Precipitation from Two Different Initialization Times * Pcp-Combine - How Do I Correct a Precipitation Time Range * Pcp-Combine - What Data Formats does MET Read * Pcp-Combine - How does pass through work * Pcp_Combine - How do I use -pcprx to run a project faster * Pcp-Combine - How Do I Enter the Time Format Correctly * Pcp-Combine - How do I use -subtract * fixing formatting * Pcp_Combine - How Do I Use -sum, -add, and -subtract to Achieve Same Accumulation Interval * Pcp-Combine - What is the difference between -sum vs. -add * Pcp-Combine - How Do I Select a Specific GRIB Record to Use * fixing spacing for original text * Plot-Data-Plane - How Do I Inspect Gen_Vx_Mask Output * fixing typo * Plot-Data-Plane - How Do I Specify GRIB Version * Plot-Data-Plane - last 2 entries * 2 misc. entries * fixing typo * Stat_Analysis - How does -aggregate_stat work * Stat_Analysis - remaining entries * fixing typo * tc-pairs * tc-stat * troubleshooting area * another stat-analysis * another stat-analysis * utilities * fixing dashes * Per #1834, trying out syntax * Per #1834, removing added syntax * fixing returns with bolding * Per #1834, modified formatting * adding italics for directories #1834 * replacing code-block ini with none to remove red lettering * fixing questions to lower case #1834 * linking to section using numref #1834 * linking to section using numref #1834 attempt 2 * linking to section using numref #1834 attempt 3 * linking to section using numref #1834 attempt 4 * linking to section using numref #1834 attempt 5 * linking to section using numref #1834 attempt 6 * linking to section using numref #1834 attempt 7 * linking to section using numref #1834 attempt 8 * linking to anothre section #1834 * linking to another section #1834 * removing old info #1834 * grammar updates #1834 * grammar updates #1834 * fixing typos #1834 * email changes with Julie #1834 * trying to fix warning #1834 * changes #1834 * Per #1834, updating question titles and answers * Per #1834, made updates to the content. * Per #1834, made updates to the content * Per #1834, made updates to the content * Per #1834, made some major changes to the first 3 FAQ's. I don't have the time/bandwidth to review these completely at this time, but wanted to include at least these updates. * testing some of the updates John O. suggested. * updates John O. suggested take 2 #1834. * updates John O. suggested take 3 #1834. * updates John O. suggested take 4 #1834. * updates John O. suggested take 5 #1834. * updates John O. suggested. Some didn't show up even though they've been removed #1834. * trying to fix numbered list indenting #1834 * fixing numbered list indenting #1834 * cleaning up formatting #1834 * fixing broken links #1834 Co-authored-by: Julie.Prestopnik <jpresto@ucar.edu> Co-authored-by: John Halley Gotway <johnhg@ucar.edu> * bugfix 1872 plot_tcmpr.R -lookin directory (develop) (#1874) * Bugfix 1875 develop memory (#1877) * Per #1875, fix set_cdf_ta() to initialize cdf_ta prior to populating it. * Per #1875, add debug(4) log messages whenever defining a new aggregation case in stat-analysis. * Per #1875, switch log messages about new cases from Debug(4) to Debug(3). * Per #1879, porting the same changes over to the develop branch. (#1881) * Feature 1673 gbeta (#1857) * Per #1673, add a distance_map.beta_value entry to all the Grid-Stat config files. * Per #1673, enhance Grid-Stat to parse the new beta_value option and set its value in the library code. * Per #1673, set the user-defined beta_value when processing the distance map scores. * Per #1673, add 4 new DMAP columns write them. Just writing bad data values right now, still need to actually compute them! Also need to update Stat-Analysis to update the parsing/aggregation of these new values. * Per #1673, enable beta_value to be set to a default bad data value. * Per #1673 and @ericgilleland instruction (#1673 (comment)), strip asymmetric G-Beta (AGBETA) from the output. * Per #1673, fix typo that was causing the code not to compile. * Per #1673, update MET User's Guide about G/GBETA. * Per #1673, report BETA_VALUE = TOTAL / 2 in the output instead of NA, as directed by Eric. * Per #1673, remove descriptions of Grid-Stat-specific config options for nbrhd, fourier, gradient, and distance_map from the overview section since they belong in the Grid-Stat chapter. Also add direct links for them in the Grid-Stat chapter. * Per #1673, do not reset the beta_value in DMAP::clear(). * Per #1673, log the DMAP options only after the beta_value has been determined. * Per #1673, correct the default beta_value setting as N*N / 2.0. * Per #1673, trying to format G-beta well. * Per #1673, trying to format G-beta well. * Per #1673, beta is formatted as \beta and not \Beta in latex. * Per #1673, beta is formatted as \beta and not \Beta in latex. * Per #1673, switch to defining beta_value as a function of the input grid size. * Per #1673, push initial appendix C updates for GBETA. * Per #1673, working on G and GBETA equations. * Per #1673, working on G and GBETA equations. * Per 1673, update GBETA details in appendix C. * Per #1673, add Eric's info about selecting beta. * Per #1673, update reference for Gilleland-2019 to Gilleland_PartI-2020 and Gilleland_PartII-2020. * Per 1673, fix typo. * Per #1673, making changes requested by Eric. * Per #1673, making changes requested by Eric. * Per #1673, update references based on Eric's feedback. * Update appendixD.rst (#1869) Updated this appendix. * Update appendixD.rst * Per #1673, making the formatting of Gilleland-2020 part I and II references consistent. * Per #1673, update Grid-Stat docs to clarify that GBETA is only computed on the FULL verification domain and not any masking regions. * Per #1673, add DataPlane::n_good_data() function to return a count valid data values. * Per #1673, update vx_statistics library code to only set gbeta/beta_value to bad data for sub-domains. * Per #1673, update grid_stat to mask out bad data between the fcst/obs fields before computing distance maps. This will change existing unit test results! Also store the count of valid data values in the full verification domain. * Per #1673, since GBETA is only reported when VX_MASK = FULL, updating GridStatConfig_all to include the FULL masking region to make this example more meaningful. Co-authored-by: ericgilleland <34106925+ericgilleland@users.noreply.github.com> * Feature 1764 point stat (#1885) * Per #1764 added write_orank_row() to do_hera_ens(). SL * Per issue #1764, added orank stat type for ensemble output. SL * Per issue #1764, added code to setup_txt_files() for ORANK output. SL * Added some comments for debuging. SL * Per #1764, add orank to the list of outputs created by point_stat. * Per #1764, add functions to compute the number of requested HiRA probabilities and ensemble members. Also update n_txt_row() to keep track of the number of ORANK rows to be written. Not totally sure if I'm counting these ORANK output lines correctly though. * Per #1764, update logic in setup_txt_files() to call get_max_n_hira_prob() and get_max_n_hira_ens() functions. * Per #1764, no real changes. Just fixing spacing. * Per issue #1764, cleaned up print (cout) debug statements. SL * Per issue #1764 added 'orank = NONE' to the config files for compatibility with changes to point_stat for calculating ORANK. SL * Per issue #1764 added 'orank = STAT' to config file to produce ORANK output lines in point_stat. SL * Per issue #1764 added content in all relevant sections for ORANK: observation rank statistic. SL * Added 'orank = NONE' to config file for compatibility with changes to point_stant for ORANK. SL * Added 'orank = NONE' to config file for compatibility with changes to point_stant for ORANK. SL * Apply suggestions from code review Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> * Per #1764, no real changes, just fixing indents. * Per #1764, no real changes, just fixing indents. * Per #1764, no real changes, just fixing indents. * Per #1764, putting the docs for VCNT back into its logical order with the other vector line types. The order that they're listed here does not need to match their order in the enumeration in the code. * Per #1764, fixing a vcnt typo I found in the grid-stat chapter... and making them consistent. * Per issue #1764: After the call to write_orank_row, added lines to reset the observation begin and end times. SL * Per #1764, no code changes, just fixing indents. Co-authored-by: Seth Linden <linden@kiowa.rap.ucar.edu> Co-authored-by: John Halley Gotway <johnhg@ucar.edu> Co-authored-by: John Halley Gotway <johnhg@kiowa.rap.ucar.edu> Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> * Adding Seth to the NB notification list. * Bugfix 1886 develop grid_diag (#1888) * Per #1886, port over the bugfixes from main_v10.0 to develop. * Per #1886, make grid_diag error out if to_grid is set to FCST or OBS. * Added additional default labels * Feature 1870 realtime (#1893) * Per #1870. add TCPairs config entries for valid_inc, valid_exc, and write_valid. * Per #1870, removing 2 files I accidentally added. * Per #1870, add code to parse the new tc_pairs config options. * Per #1870, unrelated... just fixing spacing. * Per #1870, update TrackPairInfo/Array classes and tc_pairs application code to use the write_valid option to subset the tracks by requested valid time before writing them. * Per #1870, update the User's Guide with the new config options. Document valid_inc/valid_exc in the common TC config section and add write_valid to the TC-Pairs chapter. * Per #1870, move the subset_write_valid() functionality to the library code and support it for both track and probability arrays. * Per #1870, update unit_tc_pairs.xml with 2 new calls to exercise the new write_valid functionality for both tracks and probabilities. * Per #1870, correct some typos in the docs... PROBRI and probri should be PROBRIRW and probrirw, resp. * Per #1870, switching match_points from TRUE back to FALSE after I acciedentally committed that change. * Per #1870, just fixing indent. Co-authored-by: John Halley Gotway <johnhg@kiowa.rap.ucar.edu> * Feature 1788 ssidx (#1892) * Per #1788, add initial definition of the SSIDX line type. * Per #1788, add a STAT-Analysis config file for computing the CBS Score. This is currently just a copy of the GO Index. * Per #1788, add hooks for computing -job cbs_score. * Per #1788, since we're adding a new SSIDX line type, the columns for that line type replace the job_ss_columns and job_go_columns arrays. * Per #1788, rename cbs_score to cbs_index per UK Met Office instruction on 7/22/21. This convention is also consistent with the existing go_index and ss_index jobs. * Per #1788 rename STATAnalysisConfig_CBS_Score to STATAnalysisConfig_CBS_Index * Per #1788, update Makefile to handle name change. * Per #1788, add support for SSIDX stat line type. * Per #1788, define 4 output SSIDX columns. * Per #1788, define an SSIndexInfo struct for stashing/writing SS Index values. * Per #1788, write the SSIDX columns. However we don't need to function to write the whole row since it's only written by Stat-Analysis. * Per #1788, create an AggrSSIndexInfo struct to keep track of unique stat header column input values. * Per #1788, update the stat-analysis job code to make use of the new structs. * Per #1788 fix typo. * Per #1788, remove GO Index defintion from the code comments. That detail belongs elsewhere. * Per #1788, since coding up the GO Index several CNT and CTC columns have been added. Adding them here to let them be included as part of the Skill Score index computations. * Per #1788, update check_hdr_str() to print the name of the problematic header columns. * Per #1788, add a set(one) member function to multiple array classes. This calls clear or erase on the array prior to adding a single element. Add this for StringArray, NumArray, and ThreshArray, and CRCArray. * Per #1788, update the compute_ss_index() function in stat_analysis to make the code more concise by calling the newly adding singular set functions for arrays. * Per #1788, add FCST_MODEL and REF_MODEL output columns to the SSIDX line type. * Per #1788, add a new N_INIT column to the SSIDX line type to report the number of initialization times represented by this skill score. Also define the SSIDXData struct to store the skill score index job output. * Per #1788, update STATAnalysisJob to explicitly set the job type directly rather than parsing a string. * Per #1788, major change here adding skill_score_index_job.h/.cc to handle the computation of the skill score indices. * Per #1788, update stat_analysis source code to make use of the new classes in skill_score_index_job.h/.cc. * Per #1788, write a log message listing the model initialization times over which the skill score index is computed. * Per #1788, tweak log message. * Per #1788, compute n_term as the max lenght of the required arrays. * Per #1788, drop in the ACTUAL definition of the GO index provide by the DTC AF project team. Also simplify the existing arrays down to lenght one if the value remains constant. * Per #1788, correct CBS Index config file. Some parameters had 40 elements while others only had 32. * Per #1788, do not write SSIDX output if there are no valid terms found. * Per #1788, format the values in the skill score index log messages, replacing -9999 with NA. * Per #1788, update the get_stat() member functions. For 0 pairs, return bad data for all statistics other than TOTAL. * Per #1788, working on readability of the warning and error messages. * Per #1788, define the ss_index_name directly in the STATAnalysis config file to make it really easy to modify. * Per #1788, store the ss_index_name configuration option in the StatJob class. Also support a command line -ss_index_name option to override the config file value. * Per #1788, change from name to ss_index_name variable to be a bit more descriptive in the variable names. * Per #1788, since ss_index_name is in the StatJob class we no longer need to pass it around as an argument. * Per #1788, simply the skill score index job handling logic a bit. We only need to update the default job once, not twice.: * Revert "Per #1788, since ss_index_name is in the StatJob class we no longer need to pass it around as an argument." This reverts commit cde0be0. * Revert "Revert "Per #1788, since ss_index_name is in the StatJob class we no longer need to pass it around as an argument."" This reverts commit bbfd838. * Per #1788, I'd pruned too much code from stat_analysis.cc which caused the processing of config file jobs to fail. Fixing that here. * Per #1788, switch Z0 to L0 to match the sample data for computing CBS Index. * Per #1788, make it so that cbs_index and go_index work from config file. * Per #1788, update the documentation for the skill score index changes. * Per #1788, just move some variable definitions further down since they're only used sometimes. * Per #1788, make job output description into a numbered section. * Per #1788, little tweak. * Per #1788, update the STATAnalysisConfig files to add ss_index_name and hss_ec_value, as needed. * Per #1788, add new unit tests to exercise recent ss_index changes and update the Stat-Analysis documentation. * Update met/docs/Users_Guide/stat-analysis.rst Co-authored-by: jprestop <jpresto@ucar.edu> * Update met/docs/Users_Guide/stat-analysis.rst Sorry for the typos! Thanks for catching them. Co-authored-by: jprestop <jpresto@ucar.edu> * Update met/docs/Users_Guide/stat-analysis.rst Co-authored-by: jprestop <jpresto@ucar.edu> * Per #1788, fix typo in error message. * Per #1788, fix a bug for ss_index... changing 'add' to 'set'. Also set the line_type seperately for each term, if requested. * Per #1788, tweak the ss_index logic slightly to make it more efficient. For each input STATLine, instead of checking all the index terms, stop searching after the first match. This assumes that each input line should be used for only one of the terms... and I think that's a pretty reasonable assumption. * Per #1788, minor tweaks to the STAT-Analysis docs. * Per #1788, add ss_index_vld_thresh option to existing STAT-Analysis config files. * Per #1788, add StatAnalysisJob ss_index_vld_thresh entry and parse it from the STAT-Analysis config file. * Per #1788, update stat_analysis code to check ss_index_vld_thresh setting when computing the skill score index. Also update the Stat-Analysis chapter of the User's Guide. * Per #1788, tweak one test to exercise a non-default ss_index_vld_thresh value. Co-authored-by: jprestop <jpresto@ucar.edu> Co-authored-by: jprestop <jpresto@ucar.edu> Co-authored-by: Howard Soh <hsoh@kiowa.rap.ucar.edu> Co-authored-by: hsoh-u <hsoh@ucar.edu> Co-authored-by: Keith Searight <keith.searight@noaa.gov> Co-authored-by: Seth Linden <linden@ucar.edu> Co-authored-by: Seth Linden <linden@kiowa.rap.ucar.edu> Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> Co-authored-by: lisagoodrich <33230218+lisagoodrich@users.noreply.github.com> Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> Co-authored-by: ericgilleland <34106925+ericgilleland@users.noreply.github.com> Co-authored-by: John Halley Gotway <johnhg@kiowa.rap.ucar.edu> Co-authored-by: MET Tools Test Account <met_test@kiowa.rap.ucar.edu> Co-authored-by: Julie Prestopnik <jpresto@seneca.rap.ucar.edu>
JohnHalleyGotway
added a commit
that referenced
this pull request
Aug 31, 2021
* Update install_met_env.kiowa Commented out items and added text to description * #1817 Added to_north argument at two_to_one() * #1817 Calls two_to_one() with false for the third argument, to_north, if latitude moves north to south * Feature 1819 automation doc warnings (#1836) * Removing "/lib" that should not be there. * #1508 checking stat() methoid returns no error before looking at FILE flag * Enable PDF creation * Modify inputenc, fontenc latex_elements * Fix typo in conf.py * Replace unicode with ascii * Replace unicode exponent with math mode * Change eqnarray to align * Replace gif images with png for PDF * Fix png file names * Escape underscore in variable name in math mode * Escape underscore in variable name in math mode * Remove underscores from variables in math mode * Replace bar with overline in some equations * Test derefencing underscores in tables * Test math changes in grid stat * Test math formatting * Test math formatting * #1838 Added log message for lat/lon values * #1838 Give warning if the first and lat lat/lon is same * #1838 Support the double data type variable on reading float type values * Feature 1833 develop discussions (#1849) * Per #1833, changed references to met_help to Discussions. * Per #1833, fixed typo and removed an unnecessary word * #1852 Removed unused nc_buf_size. Make cur and dim to the same size (called API handles 2D or 3D) * #1852 Check the start and count before calling NetCDF API * Feature 1746 wavelet stat (#1851) * For issue #1746 modified code to allow users to pass in an empty list (or NA) for forecast and observation thresholds in order to skip applying the threhsolds, but it will still compute stats with the raw fields. SL * For issue #1746, added new unit test that uses a config file that has empty lists for the forecast and observation thresholds. SL * For issue #1746 Added some content related to allowing users to set forecast and observation cat thresholds to an empty list in order to skip the binary masking (and consider all grid-points for stats). SL * Per #1746, cleaning up for consistent indentation. * Per #1746, cleaning up for consistent indentation. * Per #1746, add a revision history note, update the plotting range in the postscript output to be [-n,n] where n is the maximum value of the maximum absolute difference and 1.0, and also fix a bug. When the NA threshold comes AFTER a real threshold, the resulting data and difference values were not being updated. * Per #1746, change the Wavelet-Stat config file values in the the wvlt_plot dictionary by setting plot_min = plot_max = 0.0. That enables the default logic of the tool to take effect. Choose the plotting range of the wavelet plots as [-n,n], where n is the maximum of 1.0 and the maximum absolute difference. * Per #1746, used apply_fcst_thresh where it should have been apply_obs_thresh. * Per issue #1746, modified some content related to users being able to skip applying the categorical threhsolds by putting an empty list or NA in the configuration file. SL * Per issue #1746 Added some warnings if the forecast threshold is set to NA but the observation threshold is not NA (a numeric threshold) and vice versa. SL * Per #1746, fix a couple of typos and tweak wording in the wavelet-stat chapter. * Per #1746, loop over each pair of fcst/obs thresholds to check for inconsistent use of the NA threshold type. * Per #1746, a bit of code cleanup replacing calls to n_elements() with n() to make the code more concise. * Per #1746, need to reinitialize apply_fcst_thresh and apply_obs_thresh to true inside the loop since the NA threshold can appear anywhere in the list of thresholds. Co-authored-by: Seth Linden <linden@kiowa.rap.ucar.edu> Co-authored-by: John Halley Gotway <johnhg@ucar.edu> * Add sphinx.ext.pngmath to conf.py * Fix typo in conf.py * Add sphinx.ext.mathjax to conf.py * Add sphinx.ext.imgmath to conf.py * Fix typo in conf.py * Change math formatting in Appendix C * Temporarily disable equations with errors in Appendix C * Remove sphinx.ext.imgmath in conf.py * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Add sphinx.ext.imgmath to conf.py * Modify math formatting in Appendix C * Temporarily disable equations with errors in Appendix C * Remove sphinx.ext.imgmath in conf.py * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Temporarily disable equations with errors * Update README.md * Modify math formatting in Appendix C * Add usepackage{amssymb} to conf.py * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Fix warnings in wavelet-stat.rst * Omit Indices and tables in PDF version of User Guide * Modify math formatting in Appendix G * Modify math formatting in Appendix E * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Change release date in conf.py * #1852 get_string_val: check if 2D variable. Also checking if the variable does not have dimension (#1860) Co-authored-by: Howard Soh <hsoh@kiowa.rap.ucar.edu> * Feature 1453 create pdf user guide (#1859) * Enable PDF creation * Modify inputenc, fontenc latex_elements * Fix typo in conf.py * Replace unicode with ascii * Replace unicode exponent with math mode * Change eqnarray to align * Replace gif images with png for PDF * Fix png file names * Escape underscore in variable name in math mode * Escape underscore in variable name in math mode * Remove underscores from variables in math mode * Replace bar with overline in some equations * Test derefencing underscores in tables * Test math changes in grid stat * Test math formatting * Test math formatting * Add sphinx.ext.pngmath to conf.py * Fix typo in conf.py * Add sphinx.ext.mathjax to conf.py * Add sphinx.ext.imgmath to conf.py * Fix typo in conf.py * Change math formatting in Appendix C * Temporarily disable equations with errors in Appendix C * Remove sphinx.ext.imgmath in conf.py * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Add sphinx.ext.imgmath to conf.py * Modify math formatting in Appendix C * Temporarily disable equations with errors in Appendix C * Remove sphinx.ext.imgmath in conf.py * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Temporarily disable equations with errors * Modify math formatting in Appendix C * Add usepackage{amssymb} to conf.py * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Fix warnings in wavelet-stat.rst * Omit Indices and tables in PDF version of User Guide * Modify math formatting in Appendix G * Modify math formatting in Appendix E * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Change release date in conf.py Co-authored-by: John Halley Gotway <johnhg@ucar.edu> * Feature 1853 rps doc (#1861) * Per #1853, adding RPS docs from Eric. * Per #1853, work in progress getting the formatting correct. * Per #1853, work in progress getting the formatting correct. * Per #1853, work in progress getting the formatting correct. * Per #1853, work in progress getting the formatting correct. * Per #1853, adding a manual page break after the ROC image to get it to stay in the right spot. * Per #1853, backing out last change which didn't make the ROC image stay in the right spot. * #1855 Removed break after return (SobarQube) * #1855 Removed break after exit * 1855 Added return at MetNcPointObsOut::write_to_netcdf * #1855 Avoid releasing memory twice * #1855 Removed unused code * #1855 Corrected syntax error * #1855 Changed the second call of close() to release _ncFile * #1855 Deleted break after exit * #1855 Chamnged while to if * #1855 Added constructor TCLineCounts and initialized the members * #1855 Added break statement back for case '-' * #1855 Check the new size before extending to avoid thereference of the null pointer * #1855 Changed count to processed_count for log messages. Initialized variables at get_filtered_nc_data\*() * #1855 Corected the number of memory copy * #1855 Exit with an error message if both fgi & bgi are null * #1855 Clear Lead_Times before extening. Formatting for error messages * Add definition for the compilation environment on seneca. * #1855 Extend Lead_Times for new times * #1855 Removed break after exit * #1855 define "c" dynamically * #1855 Move the log message within if statement to avoid dereferencing a null pointer * Feature 1843 scatter index (#1863) * Added scatter index variables: SI, SI_BCL, SI_BCU to cnt columns. SL * Per issue #1843 Added Scatter Index variables (SI, SI_BCL, SI_BCU) to the CNT stat type. SL * Added Scatter Index (SI, SI_BCL, SI_BCU) to cnt stat type header. SL * Per issue #1843, added Scatter Index variables to write_cnt_cols(). SL * Per issue #1843 added Scatter Index (si) to CNTinfo class. SL * Per issue #1843 added Scatter Index (SI, si) to CNTInfo::get_stat(). SL * Per issue #1843 added Scatter Index (SI, si) calculation to both versions of compute_cntinfo(). SL * Per issue #1843, forgot semi-colon after SI calculation, fixed. SL * Added Scatter Index (si) to some of the clear() and allocate() functions. SL * Per issue #1843, added Scatter Index (si) to compute_cnt_mean(). SL * Per issue #1843, added Scatter Index (si) to bootstrap interval calculation and to the CNT write function. SL * Per issue #1843 added Scatter Index (SI, SI_BCL, SI_BCU) to store_stat_cnt(). SL * Per issue #1843, in get_stat() moved Scatterd Index (SI) line to be below RMSE for consitency. SL * Added Scatter Index (SI) to the CNT output format table. SL * Added Scatter Index (SI) to CNT statistics definitions. SL * Update appendixC.rst * Update stat_columns.cc * Update compute_ci.cc * Update met_stats.cc * Per #1843, update write_cntinfo() function. Since we're READING the SI value immediately after the RMSE value when computing CI's, we also need to write SI immediately after RMSE. Otherwise, we'll be computing CI's using the wrong statistics replicates. * Per #1843, check for divide by zero when computing SI... this could happen if comparing the same input file to itself with Grid-Stat. * Per #1843, correct definition of SI from RMSE/ME to RMSE/OBAR. * Per #1843, updated the SI definition to divide by OBAR instead of ME, but forgot to update the divide-by-zero check. Co-authored-by: Seth Linden <linden@kiowa.rap.ucar.edu> Co-authored-by: johnhg <johnhg@ucar.edu> * #1855 Corected typo - compare the numnber of U and V * #1855 Check if var_info is hull * #1855 Avoid un-initialized variables: dt at open(), v at lat() & lon(), and local variables at data() * #1855 Check if ldf is null pointer * #1855 Removed break after exit * #1855 Removed break after exit * Update pcp_combine.cc * Feature 1864 config urls (#1868) * Per #1864, remove stale GitHub pages URL for the MET User's Guide. * Per #1864, found stale GitHub IO links in the issue templates. Should make this same fix in the develop and default branches for all the METplus repos. * Per #1864, fix stale GitHub IO link in the PR template. * Move ROC curve in Appendix C * Crop ROC image for better PDF formatting * Shrink ROC image for better PDF formatting * Modify figure alignment in conf.py * Change text placement for ROC curve figure * Feature 1834 faq (#1867) * testing possible new drop-down menus for FAQ * adding sphinx-panels to extensions to be able to use drop down menus in appendixA * Per #1834, adding sphinx-panels to requirements.txt to see if that fixes the error * Per #1834, added sphinx.ext to beginning of sphinx_panels to see if that fixes the error * Per #1834, remove sphinx.ext and add trailing comma to end of extensions to see if that fixed the error * Per #1834, added sphinx-panels to pip install command to see if that fixes the error * Per #1834, added extra space in comment to get docs to build to see if change to documentation.yml will fix an error * Per #1834, added return character to get rid of warning * testing dropdown menus #1 * testing dropdown menus #2 removing lines and dotted lines * testing #3 * testing 4 * fixing numbering, removing 2nd pulldown menu, trying to fix box width * adding code block in dropdown * trying to fix width * trying to fix width with spacing * trying to fix width with spacing 2nd dropdown * removing panels * solved problem, just use dropdown no panels * adding second dropdown menu * removing the dropdown menus and fixing the spacing. * fixing typos * adding in matched pairs * adding in formating files for NetCDF and making FILE_IO a header * fixing spacing * time slice * fixing spacing * UNIX time conversion * fixing spacing * fixed-width * scientific notation * removing which per Julie P * adding Gen_Vx_Mask section * fixing formatting * changing to numbers * complex masking region * neighborhood methods boundaries * Neighborhood Methods to Compute Fraction * adding Grid_Stat - How do I use Config File Setup to Read a NetCDF file * using backticks with asterics * one more backtick with asterics * Grid_Stat - What would be an example of Verifying Probabilities * fixing typos * What is an example of using Grid-Stat with Regridding and Masking Turned On * fixing typos * Grid_Stat - What would be an example of Verifying Probabilities Example2 * trying to fix typos * trying to fix code-block spacing * Per #1834, attempting to fix warning. * How do I use different masks in MET tools using MODE as an example * trying to fix bullet list * trying to fix bullet list 2 * trying to fix bullet list 2 * trying to fix bullet list 3 * Per #1834, attempting formatting changes * Per #1834, attempting formatting changes * Per #1834, attempting formatting changes * Per #1834, attempting formatting changes * Per #1834, attempting formatting changes * Per #1834, modified formatting, including adding backslashes and removing extra returns in command line calls, and adding indentation for config file notation, and adding a return after each 'A.' * Per #1834, changed code-block from ini to none to fix formatting * *Pcp-Combine - What are some examples using -add * Pcp-Combine - How do I add and subtract with Pcp-Combine * Pcp-Combine - How do I Combine 12-hour Accumulated Precipitation from Two Different Initialization Times * Pcp-Combine - How Do I Correct a Precipitation Time Range * Pcp-Combine - What Data Formats does MET Read * Pcp-Combine - How does pass through work * Pcp_Combine - How do I use -pcprx to run a project faster * Pcp-Combine - How Do I Enter the Time Format Correctly * Pcp-Combine - How do I use -subtract * fixing formatting * Pcp_Combine - How Do I Use -sum, -add, and -subtract to Achieve Same Accumulation Interval * Pcp-Combine - What is the difference between -sum vs. -add * Pcp-Combine - How Do I Select a Specific GRIB Record to Use * fixing spacing for original text * Plot-Data-Plane - How Do I Inspect Gen_Vx_Mask Output * fixing typo * Plot-Data-Plane - How Do I Specify GRIB Version * Plot-Data-Plane - last 2 entries * 2 misc. entries * fixing typo * Stat_Analysis - How does -aggregate_stat work * Stat_Analysis - remaining entries * fixing typo * tc-pairs * tc-stat * troubleshooting area * another stat-analysis * another stat-analysis * utilities * fixing dashes * Per #1834, trying out syntax * Per #1834, removing added syntax * fixing returns with bolding * Per #1834, modified formatting * adding italics for directories #1834 * replacing code-block ini with none to remove red lettering * fixing questions to lower case #1834 * linking to section using numref #1834 * linking to section using numref #1834 attempt 2 * linking to section using numref #1834 attempt 3 * linking to section using numref #1834 attempt 4 * linking to section using numref #1834 attempt 5 * linking to section using numref #1834 attempt 6 * linking to section using numref #1834 attempt 7 * linking to section using numref #1834 attempt 8 * linking to anothre section #1834 * linking to another section #1834 * removing old info #1834 * grammar updates #1834 * grammar updates #1834 * fixing typos #1834 * email changes with Julie #1834 * trying to fix warning #1834 * changes #1834 * Per #1834, updating question titles and answers * Per #1834, made updates to the content. * Per #1834, made updates to the content * Per #1834, made updates to the content * Per #1834, made some major changes to the first 3 FAQ's. I don't have the time/bandwidth to review these completely at this time, but wanted to include at least these updates. * testing some of the updates John O. suggested. * updates John O. suggested take 2 #1834. * updates John O. suggested take 3 #1834. * updates John O. suggested take 4 #1834. * updates John O. suggested take 5 #1834. * updates John O. suggested. Some didn't show up even though they've been removed #1834. * trying to fix numbered list indenting #1834 * fixing numbered list indenting #1834 * cleaning up formatting #1834 * fixing broken links #1834 Co-authored-by: Julie.Prestopnik <jpresto@ucar.edu> Co-authored-by: John Halley Gotway <johnhg@ucar.edu> * bugfix 1872 plot_tcmpr.R -lookin directory (develop) (#1874) * Bugfix 1875 develop memory (#1877) * Per #1875, fix set_cdf_ta() to initialize cdf_ta prior to populating it. * Per #1875, add debug(4) log messages whenever defining a new aggregation case in stat-analysis. * Per #1875, switch log messages about new cases from Debug(4) to Debug(3). * Per #1879, porting the same changes over to the develop branch. (#1881) * Feature 1673 gbeta (#1857) * Per #1673, add a distance_map.beta_value entry to all the Grid-Stat config files. * Per #1673, enhance Grid-Stat to parse the new beta_value option and set its value in the library code. * Per #1673, set the user-defined beta_value when processing the distance map scores. * Per #1673, add 4 new DMAP columns write them. Just writing bad data values right now, still need to actually compute them! Also need to update Stat-Analysis to update the parsing/aggregation of these new values. * Per #1673, enable beta_value to be set to a default bad data value. * Per #1673 and @ericgilleland instruction (#1673 (comment)), strip asymmetric G-Beta (AGBETA) from the output. * Per #1673, fix typo that was causing the code not to compile. * Per #1673, update MET User's Guide about G/GBETA. * Per #1673, report BETA_VALUE = TOTAL / 2 in the output instead of NA, as directed by Eric. * Per #1673, remove descriptions of Grid-Stat-specific config options for nbrhd, fourier, gradient, and distance_map from the overview section since they belong in the Grid-Stat chapter. Also add direct links for them in the Grid-Stat chapter. * Per #1673, do not reset the beta_value in DMAP::clear(). * Per #1673, log the DMAP options only after the beta_value has been determined. * Per #1673, correct the default beta_value setting as N*N / 2.0. * Per #1673, trying to format G-beta well. * Per #1673, trying to format G-beta well. * Per #1673, beta is formatted as \beta and not \Beta in latex. * Per #1673, beta is formatted as \beta and not \Beta in latex. * Per #1673, switch to defining beta_value as a function of the input grid size. * Per #1673, push initial appendix C updates for GBETA. * Per #1673, working on G and GBETA equations. * Per #1673, working on G and GBETA equations. * Per 1673, update GBETA details in appendix C. * Per #1673, add Eric's info about selecting beta. * Per #1673, update reference for Gilleland-2019 to Gilleland_PartI-2020 and Gilleland_PartII-2020. * Per 1673, fix typo. * Per #1673, making changes requested by Eric. * Per #1673, making changes requested by Eric. * Per #1673, update references based on Eric's feedback. * Update appendixD.rst (#1869) Updated this appendix. * Update appendixD.rst * Per #1673, making the formatting of Gilleland-2020 part I and II references consistent. * Per #1673, update Grid-Stat docs to clarify that GBETA is only computed on the FULL verification domain and not any masking regions. * Per #1673, add DataPlane::n_good_data() function to return a count valid data values. * Per #1673, update vx_statistics library code to only set gbeta/beta_value to bad data for sub-domains. * Per #1673, update grid_stat to mask out bad data between the fcst/obs fields before computing distance maps. This will change existing unit test results! Also store the count of valid data values in the full verification domain. * Per #1673, since GBETA is only reported when VX_MASK = FULL, updating GridStatConfig_all to include the FULL masking region to make this example more meaningful. Co-authored-by: ericgilleland <34106925+ericgilleland@users.noreply.github.com> * Feature 1764 point stat (#1885) * Per #1764 added write_orank_row() to do_hera_ens(). SL * Per issue #1764, added orank stat type for ensemble output. SL * Per issue #1764, added code to setup_txt_files() for ORANK output. SL * Added some comments for debuging. SL * Per #1764, add orank to the list of outputs created by point_stat. * Per #1764, add functions to compute the number of requested HiRA probabilities and ensemble members. Also update n_txt_row() to keep track of the number of ORANK rows to be written. Not totally sure if I'm counting these ORANK output lines correctly though. * Per #1764, update logic in setup_txt_files() to call get_max_n_hira_prob() and get_max_n_hira_ens() functions. * Per #1764, no real changes. Just fixing spacing. * Per issue #1764, cleaned up print (cout) debug statements. SL * Per issue #1764 added 'orank = NONE' to the config files for compatibility with changes to point_stat for calculating ORANK. SL * Per issue #1764 added 'orank = STAT' to config file to produce ORANK output lines in point_stat. SL * Per issue #1764 added content in all relevant sections for ORANK: observation rank statistic. SL * Added 'orank = NONE' to config file for compatibility with changes to point_stant for ORANK. SL * Added 'orank = NONE' to config file for compatibility with changes to point_stant for ORANK. SL * Apply suggestions from code review Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> * Per #1764, no real changes, just fixing indents. * Per #1764, no real changes, just fixing indents. * Per #1764, no real changes, just fixing indents. * Per #1764, putting the docs for VCNT back into its logical order with the other vector line types. The order that they're listed here does not need to match their order in the enumeration in the code. * Per #1764, fixing a vcnt typo I found in the grid-stat chapter... and making them consistent. * Per issue #1764: After the call to write_orank_row, added lines to reset the observation begin and end times. SL * Per #1764, no code changes, just fixing indents. Co-authored-by: Seth Linden <linden@kiowa.rap.ucar.edu> Co-authored-by: John Halley Gotway <johnhg@ucar.edu> Co-authored-by: John Halley Gotway <johnhg@kiowa.rap.ucar.edu> Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> * Adding Seth to the NB notification list. * Bugfix 1886 develop grid_diag (#1888) * Per #1886, port over the bugfixes from main_v10.0 to develop. * Per #1886, make grid_diag error out if to_grid is set to FCST or OBS. * Added additional default labels * Feature 1870 realtime (#1893) * Per #1870. add TCPairs config entries for valid_inc, valid_exc, and write_valid. * Per #1870, removing 2 files I accidentally added. * Per #1870, add code to parse the new tc_pairs config options. * Per #1870, unrelated... just fixing spacing. * Per #1870, update TrackPairInfo/Array classes and tc_pairs application code to use the write_valid option to subset the tracks by requested valid time before writing them. * Per #1870, update the User's Guide with the new config options. Document valid_inc/valid_exc in the common TC config section and add write_valid to the TC-Pairs chapter. * Per #1870, move the subset_write_valid() functionality to the library code and support it for both track and probability arrays. * Per #1870, update unit_tc_pairs.xml with 2 new calls to exercise the new write_valid functionality for both tracks and probabilities. * Per #1870, correct some typos in the docs... PROBRI and probri should be PROBRIRW and probrirw, resp. * Per #1870, switching match_points from TRUE back to FALSE after I acciedentally committed that change. * Per #1870, just fixing indent. Co-authored-by: John Halley Gotway <johnhg@kiowa.rap.ucar.edu> * Feature 1788 ssidx (#1892) * Per #1788, add initial definition of the SSIDX line type. * Per #1788, add a STAT-Analysis config file for computing the CBS Score. This is currently just a copy of the GO Index. * Per #1788, add hooks for computing -job cbs_score. * Per #1788, since we're adding a new SSIDX line type, the columns for that line type replace the job_ss_columns and job_go_columns arrays. * Per #1788, rename cbs_score to cbs_index per UK Met Office instruction on 7/22/21. This convention is also consistent with the existing go_index and ss_index jobs. * Per #1788 rename STATAnalysisConfig_CBS_Score to STATAnalysisConfig_CBS_Index * Per #1788, update Makefile to handle name change. * Per #1788, add support for SSIDX stat line type. * Per #1788, define 4 output SSIDX columns. * Per #1788, define an SSIndexInfo struct for stashing/writing SS Index values. * Per #1788, write the SSIDX columns. However we don't need to function to write the whole row since it's only written by Stat-Analysis. * Per #1788, create an AggrSSIndexInfo struct to keep track of unique stat header column input values. * Per #1788, update the stat-analysis job code to make use of the new structs. * Per #1788 fix typo. * Per #1788, remove GO Index defintion from the code comments. That detail belongs elsewhere. * Per #1788, since coding up the GO Index several CNT and CTC columns have been added. Adding them here to let them be included as part of the Skill Score index computations. * Per #1788, update check_hdr_str() to print the name of the problematic header columns. * Per #1788, add a set(one) member function to multiple array classes. This calls clear or erase on the array prior to adding a single element. Add this for StringArray, NumArray, and ThreshArray, and CRCArray. * Per #1788, update the compute_ss_index() function in stat_analysis to make the code more concise by calling the newly adding singular set functions for arrays. * Per #1788, add FCST_MODEL and REF_MODEL output columns to the SSIDX line type. * Per #1788, add a new N_INIT column to the SSIDX line type to report the number of initialization times represented by this skill score. Also define the SSIDXData struct to store the skill score index job output. * Per #1788, update STATAnalysisJob to explicitly set the job type directly rather than parsing a string. * Per #1788, major change here adding skill_score_index_job.h/.cc to handle the computation of the skill score indices. * Per #1788, update stat_analysis source code to make use of the new classes in skill_score_index_job.h/.cc. * Per #1788, write a log message listing the model initialization times over which the skill score index is computed. * Per #1788, tweak log message. * Per #1788, compute n_term as the max lenght of the required arrays. * Per #1788, drop in the ACTUAL definition of the GO index provide by the DTC AF project team. Also simplify the existing arrays down to lenght one if the value remains constant. * Per #1788, correct CBS Index config file. Some parameters had 40 elements while others only had 32. * Per #1788, do not write SSIDX output if there are no valid terms found. * Per #1788, format the values in the skill score index log messages, replacing -9999 with NA. * Per #1788, update the get_stat() member functions. For 0 pairs, return bad data for all statistics other than TOTAL. * Per #1788, working on readability of the warning and error messages. * Per #1788, define the ss_index_name directly in the STATAnalysis config file to make it really easy to modify. * Per #1788, store the ss_index_name configuration option in the StatJob class. Also support a command line -ss_index_name option to override the config file value. * Per #1788, change from name to ss_index_name variable to be a bit more descriptive in the variable names. * Per #1788, since ss_index_name is in the StatJob class we no longer need to pass it around as an argument. * Per #1788, simply the skill score index job handling logic a bit. We only need to update the default job once, not twice.: * Revert "Per #1788, since ss_index_name is in the StatJob class we no longer need to pass it around as an argument." This reverts commit cde0be0. * Revert "Revert "Per #1788, since ss_index_name is in the StatJob class we no longer need to pass it around as an argument."" This reverts commit bbfd838. * Per #1788, I'd pruned too much code from stat_analysis.cc which caused the processing of config file jobs to fail. Fixing that here. * Per #1788, switch Z0 to L0 to match the sample data for computing CBS Index. * Per #1788, make it so that cbs_index and go_index work from config file. * Per #1788, update the documentation for the skill score index changes. * Per #1788, just move some variable definitions further down since they're only used sometimes. * Per #1788, make job output description into a numbered section. * Per #1788, little tweak. * Per #1788, update the STATAnalysisConfig files to add ss_index_name and hss_ec_value, as needed. * Per #1788, add new unit tests to exercise recent ss_index changes and update the Stat-Analysis documentation. * Update met/docs/Users_Guide/stat-analysis.rst Co-authored-by: jprestop <jpresto@ucar.edu> * Update met/docs/Users_Guide/stat-analysis.rst Sorry for the typos! Thanks for catching them. Co-authored-by: jprestop <jpresto@ucar.edu> * Update met/docs/Users_Guide/stat-analysis.rst Co-authored-by: jprestop <jpresto@ucar.edu> * Per #1788, fix typo in error message. * Per #1788, fix a bug for ss_index... changing 'add' to 'set'. Also set the line_type seperately for each term, if requested. * Per #1788, tweak the ss_index logic slightly to make it more efficient. For each input STATLine, instead of checking all the index terms, stop searching after the first match. This assumes that each input line should be used for only one of the terms... and I think that's a pretty reasonable assumption. * Per #1788, minor tweaks to the STAT-Analysis docs. * Per #1788, add ss_index_vld_thresh option to existing STAT-Analysis config files. * Per #1788, add StatAnalysisJob ss_index_vld_thresh entry and parse it from the STAT-Analysis config file. * Per #1788, update stat_analysis code to check ss_index_vld_thresh setting when computing the skill score index. Also update the Stat-Analysis chapter of the User's Guide. * Per #1788, tweak one test to exercise a non-default ss_index_vld_thresh value. Co-authored-by: jprestop <jpresto@ucar.edu> * Bugfix 1891 gen_vx_mask (#1895) * Per #1891, tighten up the gen_vx_mask logic a bit. Use a global config for the default NC compression level. Then read the input and mask fields using local variables instead of the global one to ensure they don't affect eachother. Also took the opportunity to refine the logic so that we only create a mtddf object in one spot, in the get_data_plane() function. * Per #1891, add a new test in unit_gen_vx_mask.xml that fails prior to #1891 and succeeds with these changes. * Per #1891, file name typo in unit_gen_vx_mask.xml. Co-authored-by: jprestop <jpresto@ucar.edu> Co-authored-by: Howard Soh <hsoh@kiowa.rap.ucar.edu> Co-authored-by: hsoh-u <hsoh@ucar.edu> Co-authored-by: Keith Searight <keith.searight@noaa.gov> Co-authored-by: Seth Linden <linden@ucar.edu> Co-authored-by: Seth Linden <linden@kiowa.rap.ucar.edu> Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> Co-authored-by: lisagoodrich <33230218+lisagoodrich@users.noreply.github.com> Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> Co-authored-by: ericgilleland <34106925+ericgilleland@users.noreply.github.com> Co-authored-by: John Halley Gotway <johnhg@kiowa.rap.ucar.edu> Co-authored-by: MET Tools Test Account <met_test@kiowa.rap.ucar.edu> Co-authored-by: Julie Prestopnik <jpresto@seneca.rap.ucar.edu>
JohnHalleyGotway
added a commit
that referenced
this pull request
Sep 17, 2021
* Update install_met_env.kiowa Commented out items and added text to description * #1817 Added to_north argument at two_to_one() * #1817 Calls two_to_one() with false for the third argument, to_north, if latitude moves north to south * Feature 1819 automation doc warnings (#1836) * Removing "/lib" that should not be there. * #1508 checking stat() methoid returns no error before looking at FILE flag * Enable PDF creation * Modify inputenc, fontenc latex_elements * Fix typo in conf.py * Replace unicode with ascii * Replace unicode exponent with math mode * Change eqnarray to align * Replace gif images with png for PDF * Fix png file names * Escape underscore in variable name in math mode * Escape underscore in variable name in math mode * Remove underscores from variables in math mode * Replace bar with overline in some equations * Test derefencing underscores in tables * Test math changes in grid stat * Test math formatting * Test math formatting * #1838 Added log message for lat/lon values * #1838 Give warning if the first and lat lat/lon is same * #1838 Support the double data type variable on reading float type values * Feature 1833 develop discussions (#1849) * Per #1833, changed references to met_help to Discussions. * Per #1833, fixed typo and removed an unnecessary word * #1852 Removed unused nc_buf_size. Make cur and dim to the same size (called API handles 2D or 3D) * #1852 Check the start and count before calling NetCDF API * Feature 1746 wavelet stat (#1851) * For issue #1746 modified code to allow users to pass in an empty list (or NA) for forecast and observation thresholds in order to skip applying the threhsolds, but it will still compute stats with the raw fields. SL * For issue #1746, added new unit test that uses a config file that has empty lists for the forecast and observation thresholds. SL * For issue #1746 Added some content related to allowing users to set forecast and observation cat thresholds to an empty list in order to skip the binary masking (and consider all grid-points for stats). SL * Per #1746, cleaning up for consistent indentation. * Per #1746, cleaning up for consistent indentation. * Per #1746, add a revision history note, update the plotting range in the postscript output to be [-n,n] where n is the maximum value of the maximum absolute difference and 1.0, and also fix a bug. When the NA threshold comes AFTER a real threshold, the resulting data and difference values were not being updated. * Per #1746, change the Wavelet-Stat config file values in the the wvlt_plot dictionary by setting plot_min = plot_max = 0.0. That enables the default logic of the tool to take effect. Choose the plotting range of the wavelet plots as [-n,n], where n is the maximum of 1.0 and the maximum absolute difference. * Per #1746, used apply_fcst_thresh where it should have been apply_obs_thresh. * Per issue #1746, modified some content related to users being able to skip applying the categorical threhsolds by putting an empty list or NA in the configuration file. SL * Per issue #1746 Added some warnings if the forecast threshold is set to NA but the observation threshold is not NA (a numeric threshold) and vice versa. SL * Per #1746, fix a couple of typos and tweak wording in the wavelet-stat chapter. * Per #1746, loop over each pair of fcst/obs thresholds to check for inconsistent use of the NA threshold type. * Per #1746, a bit of code cleanup replacing calls to n_elements() with n() to make the code more concise. * Per #1746, need to reinitialize apply_fcst_thresh and apply_obs_thresh to true inside the loop since the NA threshold can appear anywhere in the list of thresholds. Co-authored-by: Seth Linden <linden@kiowa.rap.ucar.edu> Co-authored-by: John Halley Gotway <johnhg@ucar.edu> * Add sphinx.ext.pngmath to conf.py * Fix typo in conf.py * Add sphinx.ext.mathjax to conf.py * Add sphinx.ext.imgmath to conf.py * Fix typo in conf.py * Change math formatting in Appendix C * Temporarily disable equations with errors in Appendix C * Remove sphinx.ext.imgmath in conf.py * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Add sphinx.ext.imgmath to conf.py * Modify math formatting in Appendix C * Temporarily disable equations with errors in Appendix C * Remove sphinx.ext.imgmath in conf.py * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Temporarily disable equations with errors * Update README.md * Modify math formatting in Appendix C * Add usepackage{amssymb} to conf.py * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Fix warnings in wavelet-stat.rst * Omit Indices and tables in PDF version of User Guide * Modify math formatting in Appendix G * Modify math formatting in Appendix E * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Change release date in conf.py * #1852 get_string_val: check if 2D variable. Also checking if the variable does not have dimension (#1860) Co-authored-by: Howard Soh <hsoh@kiowa.rap.ucar.edu> * Feature 1453 create pdf user guide (#1859) * Enable PDF creation * Modify inputenc, fontenc latex_elements * Fix typo in conf.py * Replace unicode with ascii * Replace unicode exponent with math mode * Change eqnarray to align * Replace gif images with png for PDF * Fix png file names * Escape underscore in variable name in math mode * Escape underscore in variable name in math mode * Remove underscores from variables in math mode * Replace bar with overline in some equations * Test derefencing underscores in tables * Test math changes in grid stat * Test math formatting * Test math formatting * Add sphinx.ext.pngmath to conf.py * Fix typo in conf.py * Add sphinx.ext.mathjax to conf.py * Add sphinx.ext.imgmath to conf.py * Fix typo in conf.py * Change math formatting in Appendix C * Temporarily disable equations with errors in Appendix C * Remove sphinx.ext.imgmath in conf.py * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Add sphinx.ext.imgmath to conf.py * Modify math formatting in Appendix C * Temporarily disable equations with errors in Appendix C * Remove sphinx.ext.imgmath in conf.py * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Temporarily disable equations with errors * Modify math formatting in Appendix C * Add usepackage{amssymb} to conf.py * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Fix warnings in wavelet-stat.rst * Omit Indices and tables in PDF version of User Guide * Modify math formatting in Appendix G * Modify math formatting in Appendix E * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Change release date in conf.py Co-authored-by: John Halley Gotway <johnhg@ucar.edu> * Feature 1853 rps doc (#1861) * Per #1853, adding RPS docs from Eric. * Per #1853, work in progress getting the formatting correct. * Per #1853, work in progress getting the formatting correct. * Per #1853, work in progress getting the formatting correct. * Per #1853, work in progress getting the formatting correct. * Per #1853, adding a manual page break after the ROC image to get it to stay in the right spot. * Per #1853, backing out last change which didn't make the ROC image stay in the right spot. * #1855 Removed break after return (SobarQube) * #1855 Removed break after exit * 1855 Added return at MetNcPointObsOut::write_to_netcdf * #1855 Avoid releasing memory twice * #1855 Removed unused code * #1855 Corrected syntax error * #1855 Changed the second call of close() to release _ncFile * #1855 Deleted break after exit * #1855 Chamnged while to if * #1855 Added constructor TCLineCounts and initialized the members * #1855 Added break statement back for case '-' * #1855 Check the new size before extending to avoid thereference of the null pointer * #1855 Changed count to processed_count for log messages. Initialized variables at get_filtered_nc_data\*() * #1855 Corected the number of memory copy * #1855 Exit with an error message if both fgi & bgi are null * #1855 Clear Lead_Times before extening. Formatting for error messages * Add definition for the compilation environment on seneca. * #1855 Extend Lead_Times for new times * #1855 Removed break after exit * #1855 define "c" dynamically * #1855 Move the log message within if statement to avoid dereferencing a null pointer * Feature 1843 scatter index (#1863) * Added scatter index variables: SI, SI_BCL, SI_BCU to cnt columns. SL * Per issue #1843 Added Scatter Index variables (SI, SI_BCL, SI_BCU) to the CNT stat type. SL * Added Scatter Index (SI, SI_BCL, SI_BCU) to cnt stat type header. SL * Per issue #1843, added Scatter Index variables to write_cnt_cols(). SL * Per issue #1843 added Scatter Index (si) to CNTinfo class. SL * Per issue #1843 added Scatter Index (SI, si) to CNTInfo::get_stat(). SL * Per issue #1843 added Scatter Index (SI, si) calculation to both versions of compute_cntinfo(). SL * Per issue #1843, forgot semi-colon after SI calculation, fixed. SL * Added Scatter Index (si) to some of the clear() and allocate() functions. SL * Per issue #1843, added Scatter Index (si) to compute_cnt_mean(). SL * Per issue #1843, added Scatter Index (si) to bootstrap interval calculation and to the CNT write function. SL * Per issue #1843 added Scatter Index (SI, SI_BCL, SI_BCU) to store_stat_cnt(). SL * Per issue #1843, in get_stat() moved Scatterd Index (SI) line to be below RMSE for consitency. SL * Added Scatter Index (SI) to the CNT output format table. SL * Added Scatter Index (SI) to CNT statistics definitions. SL * Update appendixC.rst * Update stat_columns.cc * Update compute_ci.cc * Update met_stats.cc * Per #1843, update write_cntinfo() function. Since we're READING the SI value immediately after the RMSE value when computing CI's, we also need to write SI immediately after RMSE. Otherwise, we'll be computing CI's using the wrong statistics replicates. * Per #1843, check for divide by zero when computing SI... this could happen if comparing the same input file to itself with Grid-Stat. * Per #1843, correct definition of SI from RMSE/ME to RMSE/OBAR. * Per #1843, updated the SI definition to divide by OBAR instead of ME, but forgot to update the divide-by-zero check. Co-authored-by: Seth Linden <linden@kiowa.rap.ucar.edu> Co-authored-by: johnhg <johnhg@ucar.edu> * #1855 Corected typo - compare the numnber of U and V * #1855 Check if var_info is hull * #1855 Avoid un-initialized variables: dt at open(), v at lat() & lon(), and local variables at data() * #1855 Check if ldf is null pointer * #1855 Removed break after exit * #1855 Removed break after exit * Update pcp_combine.cc * Feature 1864 config urls (#1868) * Per #1864, remove stale GitHub pages URL for the MET User's Guide. * Per #1864, found stale GitHub IO links in the issue templates. Should make this same fix in the develop and default branches for all the METplus repos. * Per #1864, fix stale GitHub IO link in the PR template. * Move ROC curve in Appendix C * Crop ROC image for better PDF formatting * Shrink ROC image for better PDF formatting * Modify figure alignment in conf.py * Change text placement for ROC curve figure * Feature 1834 faq (#1867) * testing possible new drop-down menus for FAQ * adding sphinx-panels to extensions to be able to use drop down menus in appendixA * Per #1834, adding sphinx-panels to requirements.txt to see if that fixes the error * Per #1834, added sphinx.ext to beginning of sphinx_panels to see if that fixes the error * Per #1834, remove sphinx.ext and add trailing comma to end of extensions to see if that fixed the error * Per #1834, added sphinx-panels to pip install command to see if that fixes the error * Per #1834, added extra space in comment to get docs to build to see if change to documentation.yml will fix an error * Per #1834, added return character to get rid of warning * testing dropdown menus #1 * testing dropdown menus #2 removing lines and dotted lines * testing #3 * testing 4 * fixing numbering, removing 2nd pulldown menu, trying to fix box width * adding code block in dropdown * trying to fix width * trying to fix width with spacing * trying to fix width with spacing 2nd dropdown * removing panels * solved problem, just use dropdown no panels * adding second dropdown menu * removing the dropdown menus and fixing the spacing. * fixing typos * adding in matched pairs * adding in formating files for NetCDF and making FILE_IO a header * fixing spacing * time slice * fixing spacing * UNIX time conversion * fixing spacing * fixed-width * scientific notation * removing which per Julie P * adding Gen_Vx_Mask section * fixing formatting * changing to numbers * complex masking region * neighborhood methods boundaries * Neighborhood Methods to Compute Fraction * adding Grid_Stat - How do I use Config File Setup to Read a NetCDF file * using backticks with asterics * one more backtick with asterics * Grid_Stat - What would be an example of Verifying Probabilities * fixing typos * What is an example of using Grid-Stat with Regridding and Masking Turned On * fixing typos * Grid_Stat - What would be an example of Verifying Probabilities Example2 * trying to fix typos * trying to fix code-block spacing * Per #1834, attempting to fix warning. * How do I use different masks in MET tools using MODE as an example * trying to fix bullet list * trying to fix bullet list 2 * trying to fix bullet list 2 * trying to fix bullet list 3 * Per #1834, attempting formatting changes * Per #1834, attempting formatting changes * Per #1834, attempting formatting changes * Per #1834, attempting formatting changes * Per #1834, attempting formatting changes * Per #1834, modified formatting, including adding backslashes and removing extra returns in command line calls, and adding indentation for config file notation, and adding a return after each 'A.' * Per #1834, changed code-block from ini to none to fix formatting * *Pcp-Combine - What are some examples using -add * Pcp-Combine - How do I add and subtract with Pcp-Combine * Pcp-Combine - How do I Combine 12-hour Accumulated Precipitation from Two Different Initialization Times * Pcp-Combine - How Do I Correct a Precipitation Time Range * Pcp-Combine - What Data Formats does MET Read * Pcp-Combine - How does pass through work * Pcp_Combine - How do I use -pcprx to run a project faster * Pcp-Combine - How Do I Enter the Time Format Correctly * Pcp-Combine - How do I use -subtract * fixing formatting * Pcp_Combine - How Do I Use -sum, -add, and -subtract to Achieve Same Accumulation Interval * Pcp-Combine - What is the difference between -sum vs. -add * Pcp-Combine - How Do I Select a Specific GRIB Record to Use * fixing spacing for original text * Plot-Data-Plane - How Do I Inspect Gen_Vx_Mask Output * fixing typo * Plot-Data-Plane - How Do I Specify GRIB Version * Plot-Data-Plane - last 2 entries * 2 misc. entries * fixing typo * Stat_Analysis - How does -aggregate_stat work * Stat_Analysis - remaining entries * fixing typo * tc-pairs * tc-stat * troubleshooting area * another stat-analysis * another stat-analysis * utilities * fixing dashes * Per #1834, trying out syntax * Per #1834, removing added syntax * fixing returns with bolding * Per #1834, modified formatting * adding italics for directories #1834 * replacing code-block ini with none to remove red lettering * fixing questions to lower case #1834 * linking to section using numref #1834 * linking to section using numref #1834 attempt 2 * linking to section using numref #1834 attempt 3 * linking to section using numref #1834 attempt 4 * linking to section using numref #1834 attempt 5 * linking to section using numref #1834 attempt 6 * linking to section using numref #1834 attempt 7 * linking to section using numref #1834 attempt 8 * linking to anothre section #1834 * linking to another section #1834 * removing old info #1834 * grammar updates #1834 * grammar updates #1834 * fixing typos #1834 * email changes with Julie #1834 * trying to fix warning #1834 * changes #1834 * Per #1834, updating question titles and answers * Per #1834, made updates to the content. * Per #1834, made updates to the content * Per #1834, made updates to the content * Per #1834, made some major changes to the first 3 FAQ's. I don't have the time/bandwidth to review these completely at this time, but wanted to include at least these updates. * testing some of the updates John O. suggested. * updates John O. suggested take 2 #1834. * updates John O. suggested take 3 #1834. * updates John O. suggested take 4 #1834. * updates John O. suggested take 5 #1834. * updates John O. suggested. Some didn't show up even though they've been removed #1834. * trying to fix numbered list indenting #1834 * fixing numbered list indenting #1834 * cleaning up formatting #1834 * fixing broken links #1834 Co-authored-by: Julie.Prestopnik <jpresto@ucar.edu> Co-authored-by: John Halley Gotway <johnhg@ucar.edu> * bugfix 1872 plot_tcmpr.R -lookin directory (develop) (#1874) * Bugfix 1875 develop memory (#1877) * Per #1875, fix set_cdf_ta() to initialize cdf_ta prior to populating it. * Per #1875, add debug(4) log messages whenever defining a new aggregation case in stat-analysis. * Per #1875, switch log messages about new cases from Debug(4) to Debug(3). * Per #1879, porting the same changes over to the develop branch. (#1881) * Feature 1673 gbeta (#1857) * Per #1673, add a distance_map.beta_value entry to all the Grid-Stat config files. * Per #1673, enhance Grid-Stat to parse the new beta_value option and set its value in the library code. * Per #1673, set the user-defined beta_value when processing the distance map scores. * Per #1673, add 4 new DMAP columns write them. Just writing bad data values right now, still need to actually compute them! Also need to update Stat-Analysis to update the parsing/aggregation of these new values. * Per #1673, enable beta_value to be set to a default bad data value. * Per #1673 and @ericgilleland instruction (#1673 (comment)), strip asymmetric G-Beta (AGBETA) from the output. * Per #1673, fix typo that was causing the code not to compile. * Per #1673, update MET User's Guide about G/GBETA. * Per #1673, report BETA_VALUE = TOTAL / 2 in the output instead of NA, as directed by Eric. * Per #1673, remove descriptions of Grid-Stat-specific config options for nbrhd, fourier, gradient, and distance_map from the overview section since they belong in the Grid-Stat chapter. Also add direct links for them in the Grid-Stat chapter. * Per #1673, do not reset the beta_value in DMAP::clear(). * Per #1673, log the DMAP options only after the beta_value has been determined. * Per #1673, correct the default beta_value setting as N*N / 2.0. * Per #1673, trying to format G-beta well. * Per #1673, trying to format G-beta well. * Per #1673, beta is formatted as \beta and not \Beta in latex. * Per #1673, beta is formatted as \beta and not \Beta in latex. * Per #1673, switch to defining beta_value as a function of the input grid size. * Per #1673, push initial appendix C updates for GBETA. * Per #1673, working on G and GBETA equations. * Per #1673, working on G and GBETA equations. * Per 1673, update GBETA details in appendix C. * Per #1673, add Eric's info about selecting beta. * Per #1673, update reference for Gilleland-2019 to Gilleland_PartI-2020 and Gilleland_PartII-2020. * Per 1673, fix typo. * Per #1673, making changes requested by Eric. * Per #1673, making changes requested by Eric. * Per #1673, update references based on Eric's feedback. * Update appendixD.rst (#1869) Updated this appendix. * Update appendixD.rst * Per #1673, making the formatting of Gilleland-2020 part I and II references consistent. * Per #1673, update Grid-Stat docs to clarify that GBETA is only computed on the FULL verification domain and not any masking regions. * Per #1673, add DataPlane::n_good_data() function to return a count valid data values. * Per #1673, update vx_statistics library code to only set gbeta/beta_value to bad data for sub-domains. * Per #1673, update grid_stat to mask out bad data between the fcst/obs fields before computing distance maps. This will change existing unit test results! Also store the count of valid data values in the full verification domain. * Per #1673, since GBETA is only reported when VX_MASK = FULL, updating GridStatConfig_all to include the FULL masking region to make this example more meaningful. Co-authored-by: ericgilleland <34106925+ericgilleland@users.noreply.github.com> * Feature 1764 point stat (#1885) * Per #1764 added write_orank_row() to do_hera_ens(). SL * Per issue #1764, added orank stat type for ensemble output. SL * Per issue #1764, added code to setup_txt_files() for ORANK output. SL * Added some comments for debuging. SL * Per #1764, add orank to the list of outputs created by point_stat. * Per #1764, add functions to compute the number of requested HiRA probabilities and ensemble members. Also update n_txt_row() to keep track of the number of ORANK rows to be written. Not totally sure if I'm counting these ORANK output lines correctly though. * Per #1764, update logic in setup_txt_files() to call get_max_n_hira_prob() and get_max_n_hira_ens() functions. * Per #1764, no real changes. Just fixing spacing. * Per issue #1764, cleaned up print (cout) debug statements. SL * Per issue #1764 added 'orank = NONE' to the config files for compatibility with changes to point_stat for calculating ORANK. SL * Per issue #1764 added 'orank = STAT' to config file to produce ORANK output lines in point_stat. SL * Per issue #1764 added content in all relevant sections for ORANK: observation rank statistic. SL * Added 'orank = NONE' to config file for compatibility with changes to point_stant for ORANK. SL * Added 'orank = NONE' to config file for compatibility with changes to point_stant for ORANK. SL * Apply suggestions from code review Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> * Per #1764, no real changes, just fixing indents. * Per #1764, no real changes, just fixing indents. * Per #1764, no real changes, just fixing indents. * Per #1764, putting the docs for VCNT back into its logical order with the other vector line types. The order that they're listed here does not need to match their order in the enumeration in the code. * Per #1764, fixing a vcnt typo I found in the grid-stat chapter... and making them consistent. * Per issue #1764: After the call to write_orank_row, added lines to reset the observation begin and end times. SL * Per #1764, no code changes, just fixing indents. Co-authored-by: Seth Linden <linden@kiowa.rap.ucar.edu> Co-authored-by: John Halley Gotway <johnhg@ucar.edu> Co-authored-by: John Halley Gotway <johnhg@kiowa.rap.ucar.edu> Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> * Adding Seth to the NB notification list. * Bugfix 1886 develop grid_diag (#1888) * Per #1886, port over the bugfixes from main_v10.0 to develop. * Per #1886, make grid_diag error out if to_grid is set to FCST or OBS. * Added additional default labels * Feature 1870 realtime (#1893) * Per #1870. add TCPairs config entries for valid_inc, valid_exc, and write_valid. * Per #1870, removing 2 files I accidentally added. * Per #1870, add code to parse the new tc_pairs config options. * Per #1870, unrelated... just fixing spacing. * Per #1870, update TrackPairInfo/Array classes and tc_pairs application code to use the write_valid option to subset the tracks by requested valid time before writing them. * Per #1870, update the User's Guide with the new config options. Document valid_inc/valid_exc in the common TC config section and add write_valid to the TC-Pairs chapter. * Per #1870, move the subset_write_valid() functionality to the library code and support it for both track and probability arrays. * Per #1870, update unit_tc_pairs.xml with 2 new calls to exercise the new write_valid functionality for both tracks and probabilities. * Per #1870, correct some typos in the docs... PROBRI and probri should be PROBRIRW and probrirw, resp. * Per #1870, switching match_points from TRUE back to FALSE after I acciedentally committed that change. * Per #1870, just fixing indent. Co-authored-by: John Halley Gotway <johnhg@kiowa.rap.ucar.edu> * Feature 1788 ssidx (#1892) * Per #1788, add initial definition of the SSIDX line type. * Per #1788, add a STAT-Analysis config file for computing the CBS Score. This is currently just a copy of the GO Index. * Per #1788, add hooks for computing -job cbs_score. * Per #1788, since we're adding a new SSIDX line type, the columns for that line type replace the job_ss_columns and job_go_columns arrays. * Per #1788, rename cbs_score to cbs_index per UK Met Office instruction on 7/22/21. This convention is also consistent with the existing go_index and ss_index jobs. * Per #1788 rename STATAnalysisConfig_CBS_Score to STATAnalysisConfig_CBS_Index * Per #1788, update Makefile to handle name change. * Per #1788, add support for SSIDX stat line type. * Per #1788, define 4 output SSIDX columns. * Per #1788, define an SSIndexInfo struct for stashing/writing SS Index values. * Per #1788, write the SSIDX columns. However we don't need to function to write the whole row since it's only written by Stat-Analysis. * Per #1788, create an AggrSSIndexInfo struct to keep track of unique stat header column input values. * Per #1788, update the stat-analysis job code to make use of the new structs. * Per #1788 fix typo. * Per #1788, remove GO Index defintion from the code comments. That detail belongs elsewhere. * Per #1788, since coding up the GO Index several CNT and CTC columns have been added. Adding them here to let them be included as part of the Skill Score index computations. * Per #1788, update check_hdr_str() to print the name of the problematic header columns. * Per #1788, add a set(one) member function to multiple array classes. This calls clear or erase on the array prior to adding a single element. Add this for StringArray, NumArray, and ThreshArray, and CRCArray. * Per #1788, update the compute_ss_index() function in stat_analysis to make the code more concise by calling the newly adding singular set functions for arrays. * Per #1788, add FCST_MODEL and REF_MODEL output columns to the SSIDX line type. * Per #1788, add a new N_INIT column to the SSIDX line type to report the number of initialization times represented by this skill score. Also define the SSIDXData struct to store the skill score index job output. * Per #1788, update STATAnalysisJob to explicitly set the job type directly rather than parsing a string. * Per #1788, major change here adding skill_score_index_job.h/.cc to handle the computation of the skill score indices. * Per #1788, update stat_analysis source code to make use of the new classes in skill_score_index_job.h/.cc. * Per #1788, write a log message listing the model initialization times over which the skill score index is computed. * Per #1788, tweak log message. * Per #1788, compute n_term as the max lenght of the required arrays. * Per #1788, drop in the ACTUAL definition of the GO index provide by the DTC AF project team. Also simplify the existing arrays down to lenght one if the value remains constant. * Per #1788, correct CBS Index config file. Some parameters had 40 elements while others only had 32. * Per #1788, do not write SSIDX output if there are no valid terms found. * Per #1788, format the values in the skill score index log messages, replacing -9999 with NA. * Per #1788, update the get_stat() member functions. For 0 pairs, return bad data for all statistics other than TOTAL. * Per #1788, working on readability of the warning and error messages. * Per #1788, define the ss_index_name directly in the STATAnalysis config file to make it really easy to modify. * Per #1788, store the ss_index_name configuration option in the StatJob class. Also support a command line -ss_index_name option to override the config file value. * Per #1788, change from name to ss_index_name variable to be a bit more descriptive in the variable names. * Per #1788, since ss_index_name is in the StatJob class we no longer need to pass it around as an argument. * Per #1788, simply the skill score index job handling logic a bit. We only need to update the default job once, not twice.: * Revert "Per #1788, since ss_index_name is in the StatJob class we no longer need to pass it around as an argument." This reverts commit cde0be0. * Revert "Revert "Per #1788, since ss_index_name is in the StatJob class we no longer need to pass it around as an argument."" This reverts commit bbfd838. * Per #1788, I'd pruned too much code from stat_analysis.cc which caused the processing of config file jobs to fail. Fixing that here. * Per #1788, switch Z0 to L0 to match the sample data for computing CBS Index. * Per #1788, make it so that cbs_index and go_index work from config file. * Per #1788, update the documentation for the skill score index changes. * Per #1788, just move some variable definitions further down since they're only used sometimes. * Per #1788, make job output description into a numbered section. * Per #1788, little tweak. * Per #1788, update the STATAnalysisConfig files to add ss_index_name and hss_ec_value, as needed. * Per #1788, add new unit tests to exercise recent ss_index changes and update the Stat-Analysis documentation. * Update met/docs/Users_Guide/stat-analysis.rst Co-authored-by: jprestop <jpresto@ucar.edu> * Update met/docs/Users_Guide/stat-analysis.rst Sorry for the typos! Thanks for catching them. Co-authored-by: jprestop <jpresto@ucar.edu> * Update met/docs/Users_Guide/stat-analysis.rst Co-authored-by: jprestop <jpresto@ucar.edu> * Per #1788, fix typo in error message. * Per #1788, fix a bug for ss_index... changing 'add' to 'set'. Also set the line_type seperately for each term, if requested. * Per #1788, tweak the ss_index logic slightly to make it more efficient. For each input STATLine, instead of checking all the index terms, stop searching after the first match. This assumes that each input line should be used for only one of the terms... and I think that's a pretty reasonable assumption. * Per #1788, minor tweaks to the STAT-Analysis docs. * Per #1788, add ss_index_vld_thresh option to existing STAT-Analysis config files. * Per #1788, add StatAnalysisJob ss_index_vld_thresh entry and parse it from the STAT-Analysis config file. * Per #1788, update stat_analysis code to check ss_index_vld_thresh setting when computing the skill score index. Also update the Stat-Analysis chapter of the User's Guide. * Per #1788, tweak one test to exercise a non-default ss_index_vld_thresh value. Co-authored-by: jprestop <jpresto@ucar.edu> * #1855 Initialize AllocInc * 855 call snprintf instead of sprintf * Avoid the same for loops (SobarQube warns this) * #1855 Checking the minimum rows (3) for formatting. Initialize left & right array * #1855 Removed the unreachable return statement * #1855 Avoide out of index for v_miss * #1855 Keep buf_size for for-loop (avoid changing it in for loop). The buffer size for strncpy is decided by the targetbuffer size or the input length, not the existing data at the target buffer. The size of existing data at the target buffer is used to erase the previous data * #1855 Make sure the variuble c has enough data to avoid out of index error * #1855 nake sure the index is in range (avoid out of index error) * #1855 Make sure no negative offset for s array * #1855 Allocated line before calling getline'. Simplify the code to check the first level only. Check pbl_level before provcessing * #1855 Added m_strlen, m_strcpy, m_strcpy2, and m_strncpy * #1855 Added m_strlen, m_strcpy, m_strcpy2, and m_strncpy> Check if null pointer * #1855 Calls m_strcpy intead of strcpy * #1855 Calls m_strcpy2 intead of strcpy * #1855 Calls m_strcpy intead of strcpy * #1855 Removed spaces at the empty line * Bugfix 1891 gen_vx_mask (#1895) * Per #1891, tighten up the gen_vx_mask logic a bit. Use a global config for the default NC compression level. Then read the input and mask fields using local variables instead of the global one to ensure they don't affect eachother. Also took the opportunity to refine the logic so that we only create a mtddf object in one spot, in the get_data_plane() function. * Per #1891, add a new test in unit_gen_vx_mask.xml that fails prior to #1891 and succeeds with these changes. * Per #1891, file name typo in unit_gen_vx_mask.xml. * #1855 sonarqube: replaced strlen to m_strlen * #1855 sonarqube: replaced strcpy to m_strcpy * #1855 sonarqube: replaced strlen to m_strlen * #1855 sonarqube: replaced strlen to m_strlen * #1855 sonarqube: replaced strlen to m_strlen * #1855 Replaced strncpy to m_strncpy and enhance m_strcpy & m_strncpy (#1901) Co-authored-by: Howard Soh <hsoh@kiowa.rap.ucar.edu> * #1855 Turn off new strcpy functions * #1855 Corretced typo (arguments were swapped at MetPythonDataFile::open) * #1855 Make sure the to_str is null terminated after strncpy * Feature 1827 v10.1.0 beta2 (#1902) * Per #1827, updated release notes * Per #1827, Added extra ~ under header * Update met/docs/Users_Guide/release-notes.rst Co-authored-by: johnhg <johnhg@ucar.edu> * Update met/docs/Users_Guide/release-notes.rst Co-authored-by: johnhg <johnhg@ucar.edu> * Update met/docs/Users_Guide/release-notes.rst Co-authored-by: johnhg <johnhg@ucar.edu> * Per #1827, rereferenced GitHub issue number Co-authored-by: Julie Prestopnik <jpresto@seneca.rap.ucar.edu> Co-authored-by: johnhg <johnhg@ucar.edu> * #1903 Added str_wrappers.cc & str_wrappers.h. They are moved from string_fxns * #1903 Added enum_to_string.h. They are moved from enum_to_string.cc * #1903 Moved include statements to the top. Included enum_to_string.h & str_wrappers.h. Changed 0 to NULL to reset the pointer * #1903 Moved m_strXXX to str_wrappers. Includes str_wrappers.h instead of string_fxns.h * #1903 Changed strXXX to m_strXXX * #1903 Chaged dst_info to const * #1903 Separated two || conditions to avoid potential side effect * #1903 Avoid the negative offset if the string length is 0 * #1903 make sure the pointer is not NULL before releasing * #1903 Replaced strXXX to m_strXXX. Separated two || conditions to avoid the potential side effects * #1903 Allow to truncate the string on calling m_strncpy * #1903 Replaced strXXX calls to m_strXXX * #1903 Avoid the negative offset if the string length is 0 * Separated third AND condition to avoid a possible side effect (assigned lead) * #1903 Renamed import (which is a keyword) to import_image) * Initialize data at get_filtered_nc_data_2d * #1903 Initialize r, g, b at dhsv_to_drgb * Modified to avoid unreachable return statement * #1903 Separated the second AND condition to avoid a potential side effect (assignment at the second condition) * #1903 Removed unreachable break (after exit) * #1903 Removed a unused offset variable * #1903 Initialize fcst variable * Renamed module (keyword) to py_module * Removed the second argument at m_strlen * #1903 Deleted extra two empty lines * #1903 Formatting (tab to spaces) * #1903 mstcpy is changed to m_strcpy2 * Deleted the commented out code * #1903 Calls sizeof instead of m_strlen at m_strcpy * #1903 Calls m_strncpy instead of m_strcpy * #1903 Replaced m_strcpy to m_strncpy * #1903 Do not use sizeof * Feature 1912 missing (#1915) * Per #1912, update logic of Met2dDataFile::process_data_plane(). Rather than returning bad status for a field of all bad data, print a warning message and continue processing. * Per #1912, fixing typo in error message that I ran into. Doesn't actually apply to this issue. * Per #1912, making an unrelated change here to fix up the MODE-Analysis log output. It was reporting (nul) for the number of lines being read/kept. Simplify the log messages to print the correct numbers. * Testing RTD updates * Added sphinx-panels * Testing double hyphen change to code-block * Testing double hyphen change to code-block * Changed bolding of options to code-blocks * Feature 1858 obs quality (#1919) * Per issue #1858: changed parse_conf_obs_qty to parse_conf_obs_qty_inc. Added new function parse_conf_obs_qty_exc. SL * Per issue #1858: changed conf_key_obs_qty to conf_key_obs_qty_inc. Added conf_key_obs_qty_exc. SL * Per issue #1858: modified vx_pd.set_obs_qty_filt() to use parse_conf_obs_qty_inc() instead of parse_conf_obs_qty(). SL * Per issue #1858: added back the original parse_conf_obs_qty() function for backward compatibility. SL * Per issue #1858: added back config_constants.h for backwards compatibility. SL * Per issue #1858: added code to read in obs_qty_inc instead of obs_qty. Also added in code for backwards compatibility to check for obs_qty (obs_quality) and use that for obs_qty_inc. SL * Changed obs_qty_filt to obs_qty_inc_filt. Added obs_qty_exc_filt, including relevant function. SL * Per issue #1858 added vx_pd.set_obs_qty_exc_filt. SL * In add_point_obs() added if block to apply obs_qty_exc_filt. SL * Per issue #1858, in add_point_obs(), cleaned up sections that apply obs_qty_inc and obs_qty_exc filters. Fixed spacing issue. SL * Per issue #1858 in process_config(), cleaned up sections that set obs_qty_inc_filt and obs_qty_exc_filt. Fixed spacing issue. SL * Per issue #1858 in process_obs_file() cleaned up code. SL * Per issue #1858: changed obs_qty_filt to obs_qty_inc_filt. Added obs_qty_exc_filt. Modified add_point_obs() to use obs_qty_exc_filt. SL * Per issue #1858: in process_config(): added code to set_obs_qty_inc_filt and set_obs_qty_exc_filt. Also added code to check if old obs_quality is being used, if so, set it to obs_qty_inc_filt. SL * Per issue #1858: in add_point_obs() modified section that applies the obs_qty_inc and obs_qty_exc filters. Combined the if-blocks into one concise if-block that applies both filters. SL * Per issue #1858: replaced obs_quality with obs_quality_inc and added obs_quality_inc. SL * Per issue #1858: replaced obs_quality with obs_quality_inc and added obs_quality_exc. SL * Per issue #1858: For both the point_stat and ensemble_stat tests: modified tests that use OBS_QTY to use OBS_QTY_INC. Added unit test for OBS_QTY_EXC (obs_quality_exclude). SL * Per issue #1858: just added comment at top of file indicating what changed. SL * Per issue #1858: just added comment at top of file indicating what changed. SL * Per issue #1858: removed parse_conf_obs_qty. Modified parse_conf_obs_qty_inc to check for old 'obs_quality' entry. SL * Per issue #1858: in process_config(), modified section that calls set_obs_qty_inc_filt...this new function can now handle old 'obs_quality' entry as well as the new 'obs_quality_inc'. SL * Modified parse_conf_obs_qty_inc and parse_conf_obs_qty_exc. SL * Per issue #1858: re-updated parse_conf_obs_qty_exc and reverted back to the simplier version of this function. SL * Per issue #1858: replaced obs_quality with obs_quality_inc and added obs_quality_exc. SL * Per issue #1858: replaced obs_quality with obs_quality_inc and added obs_quality_exc. SL * Per issue #1858: replaced obs_quality with obs_quality_inc and added obs_quality_exc. SL * Per issue #1858: replaced obs_quality with obs_quality_inc and added obs_quality_exc. SL * Per issue #1858: replaced obs_quality with obs_quality_inc and added obs_quality_inc. SL * Per issue #1858: replaced obs_quality with obs_quality_inc. Added documentation for obs_quality_exc. SL * Per issue #1858: checking in latest version. Modified section that calls set_obs_qty_inc_filt...this new function can now handle old 'obs_quality' entry as well as the new 'obs_quality_inc'. SL * Per issue #1858: cleaned up some extra line spaces. SL * Per #1858, removed the deprecated obs_quality configuration entry from 3 EnsembleStat and 5 PointStat configuration files. * Per issue #1858: duplicated fcst/obs fields to run/test with both obs_quality_inc and obs_quality_exc. SL * Per issue #1858: duplicated fcst/obs fields to run/test with both obs_quality_inc and obs_quality_exc. SL * Per issue #1858. Deleted these files git repository. These were replaced with: PointStatConfig_qty_inc_exc. SL * Per issue #1858. Deleted these files from the git repository. These were replaced with: EnsembleStatConfig_qty_inc_exc. SL * Per issue #1858. Deleted these files from the git repository. These were replaced with: PointStatConfig_qty_inc_exc. SL * Per issue #1858: Modified: Don't need to run separate tests for obs_quality_inc and obs_quality_exc, these are now combined into one test via the updated config files: PointStatConfig_qty_inc_exc and EnsembleStatConfig_qty_inc_exc. SL * Per #1858, update the logic in parse_conf_obs_qty_inc() to use last_lookup_status(). See forthcoming GitHub issue comment about this. * Per #1858, tweak the warning message. * Per #1858, made the existing unit_quality_filter.xml tests a bit more concise. Call Point-Stat and Ensemble-Stat only once each time. But for each, test 3 things: all qty, obs_quality_inc, and obs_quality_exc. * Per #1858, remove the _ON from the job name since the _OFF has been included in the same configuration. Co-authored-by: Seth Linden <linden@kiowa.rap.ucar.edu> Co-authored-by: John Halley Gotway <johnhg@ucar.edu> Co-authored-by: John Halley Gotway <johnhg@kiowa.rap.ucar.edu> * Feature 1894 pb2nc var all (#1917) * #1894 Added station ID 14008 to pb2nc_NDAS_var_all unit test * #1894 Check the existance of the valid data up to 16 levels Co-authored-by: Howard Soh <hsoh@kiowa.rap.ucar.edu> * Feature 1913 MAX_PBL (#1916) * #1913 Changed MAX_PBL to 10000. Filter out if the PBL input data is invalid * #1912 Replaced is_eq to is_bad_data * #1913 Changed is_eq to is_bad_data Co-authored-by: Howard Soh <hsoh@kiowa.rap.ucar.edu> * Add bolding the MET version 10.1.0 beta2 release notes to indicate emphasis prior to the METplus Governance meeting. Co-authored-by: jprestop <jpresto@ucar.edu> Co-authored-by: Howard Soh <hsoh@kiowa.rap.ucar.edu> Co-authored-by: hsoh-u <hsoh@ucar.edu> Co-authored-by: Keith Searight <keith.searight@noaa.gov> Co-authored-by: Seth Linden <linden@ucar.edu> Co-authored-by: Seth Linden <linden@kiowa.rap.ucar.edu> Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> Co-authored-by: lisagoodrich <33230218+lisagoodrich@users.noreply.github.com> Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> Co-authored-by: ericgilleland <34106925+ericgilleland@users.noreply.github.com> Co-authored-by: John Halley Gotway <johnhg@kiowa.rap.ucar.edu> Co-authored-by: MET Tools Test Account <met_test@kiowa.rap.ucar.edu> Co-authored-by: Julie Prestopnik <jpresto@seneca.rap.ucar.edu>
JohnHalleyGotway
added a commit
that referenced
this pull request
Sep 18, 2021
* Update install_met_env.kiowa Commented out items and added text to description * #1817 Added to_north argument at two_to_one() * #1817 Calls two_to_one() with false for the third argument, to_north, if latitude moves north to south * Feature 1819 automation doc warnings (#1836) * Removing "/lib" that should not be there. * #1508 checking stat() methoid returns no error before looking at FILE flag * Enable PDF creation * Modify inputenc, fontenc latex_elements * Fix typo in conf.py * Replace unicode with ascii * Replace unicode exponent with math mode * Change eqnarray to align * Replace gif images with png for PDF * Fix png file names * Escape underscore in variable name in math mode * Escape underscore in variable name in math mode * Remove underscores from variables in math mode * Replace bar with overline in some equations * Test derefencing underscores in tables * Test math changes in grid stat * Test math formatting * Test math formatting * #1838 Added log message for lat/lon values * #1838 Give warning if the first and lat lat/lon is same * #1838 Support the double data type variable on reading float type values * Feature 1833 develop discussions (#1849) * Per #1833, changed references to met_help to Discussions. * Per #1833, fixed typo and removed an unnecessary word * #1852 Removed unused nc_buf_size. Make cur and dim to the same size (called API handles 2D or 3D) * #1852 Check the start and count before calling NetCDF API * Feature 1746 wavelet stat (#1851) * For issue #1746 modified code to allow users to pass in an empty list (or NA) for forecast and observation thresholds in order to skip applying the threhsolds, but it will still compute stats with the raw fields. SL * For issue #1746, added new unit test that uses a config file that has empty lists for the forecast and observation thresholds. SL * For issue #1746 Added some content related to allowing users to set forecast and observation cat thresholds to an empty list in order to skip the binary masking (and consider all grid-points for stats). SL * Per #1746, cleaning up for consistent indentation. * Per #1746, cleaning up for consistent indentation. * Per #1746, add a revision history note, update the plotting range in the postscript output to be [-n,n] where n is the maximum value of the maximum absolute difference and 1.0, and also fix a bug. When the NA threshold comes AFTER a real threshold, the resulting data and difference values were not being updated. * Per #1746, change the Wavelet-Stat config file values in the the wvlt_plot dictionary by setting plot_min = plot_max = 0.0. That enables the default logic of the tool to take effect. Choose the plotting range of the wavelet plots as [-n,n], where n is the maximum of 1.0 and the maximum absolute difference. * Per #1746, used apply_fcst_thresh where it should have been apply_obs_thresh. * Per issue #1746, modified some content related to users being able to skip applying the categorical threhsolds by putting an empty list or NA in the configuration file. SL * Per issue #1746 Added some warnings if the forecast threshold is set to NA but the observation threshold is not NA (a numeric threshold) and vice versa. SL * Per #1746, fix a couple of typos and tweak wording in the wavelet-stat chapter. * Per #1746, loop over each pair of fcst/obs thresholds to check for inconsistent use of the NA threshold type. * Per #1746, a bit of code cleanup replacing calls to n_elements() with n() to make the code more concise. * Per #1746, need to reinitialize apply_fcst_thresh and apply_obs_thresh to true inside the loop since the NA threshold can appear anywhere in the list of thresholds. Co-authored-by: Seth Linden <linden@kiowa.rap.ucar.edu> Co-authored-by: John Halley Gotway <johnhg@ucar.edu> * Add sphinx.ext.pngmath to conf.py * Fix typo in conf.py * Add sphinx.ext.mathjax to conf.py * Add sphinx.ext.imgmath to conf.py * Fix typo in conf.py * Change math formatting in Appendix C * Temporarily disable equations with errors in Appendix C * Remove sphinx.ext.imgmath in conf.py * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Add sphinx.ext.imgmath to conf.py * Modify math formatting in Appendix C * Temporarily disable equations with errors in Appendix C * Remove sphinx.ext.imgmath in conf.py * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Temporarily disable equations with errors * Update README.md * Modify math formatting in Appendix C * Add usepackage{amssymb} to conf.py * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Fix warnings in wavelet-stat.rst * Omit Indices and tables in PDF version of User Guide * Modify math formatting in Appendix G * Modify math formatting in Appendix E * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Change release date in conf.py * #1852 get_string_val: check if 2D variable. Also checking if the variable does not have dimension (#1860) Co-authored-by: Howard Soh <hsoh@kiowa.rap.ucar.edu> * Feature 1453 create pdf user guide (#1859) * Enable PDF creation * Modify inputenc, fontenc latex_elements * Fix typo in conf.py * Replace unicode with ascii * Replace unicode exponent with math mode * Change eqnarray to align * Replace gif images with png for PDF * Fix png file names * Escape underscore in variable name in math mode * Escape underscore in variable name in math mode * Remove underscores from variables in math mode * Replace bar with overline in some equations * Test derefencing underscores in tables * Test math changes in grid stat * Test math formatting * Test math formatting * Add sphinx.ext.pngmath to conf.py * Fix typo in conf.py * Add sphinx.ext.mathjax to conf.py * Add sphinx.ext.imgmath to conf.py * Fix typo in conf.py * Change math formatting in Appendix C * Temporarily disable equations with errors in Appendix C * Remove sphinx.ext.imgmath in conf.py * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Add sphinx.ext.imgmath to conf.py * Modify math formatting in Appendix C * Temporarily disable equations with errors in Appendix C * Remove sphinx.ext.imgmath in conf.py * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Temporarily disable equations with errors * Modify math formatting in Appendix C * Add usepackage{amssymb} to conf.py * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Modify math formatting in Appendix C * Fix warnings in wavelet-stat.rst * Omit Indices and tables in PDF version of User Guide * Modify math formatting in Appendix G * Modify math formatting in Appendix E * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Modify math formatting in Appendix G * Change release date in conf.py Co-authored-by: John Halley Gotway <johnhg@ucar.edu> * Feature 1853 rps doc (#1861) * Per #1853, adding RPS docs from Eric. * Per #1853, work in progress getting the formatting correct. * Per #1853, work in progress getting the formatting correct. * Per #1853, work in progress getting the formatting correct. * Per #1853, work in progress getting the formatting correct. * Per #1853, adding a manual page break after the ROC image to get it to stay in the right spot. * Per #1853, backing out last change which didn't make the ROC image stay in the right spot. * #1855 Removed break after return (SobarQube) * #1855 Removed break after exit * 1855 Added return at MetNcPointObsOut::write_to_netcdf * #1855 Avoid releasing memory twice * #1855 Removed unused code * #1855 Corrected syntax error * #1855 Changed the second call of close() to release _ncFile * #1855 Deleted break after exit * #1855 Chamnged while to if * #1855 Added constructor TCLineCounts and initialized the members * #1855 Added break statement back for case '-' * #1855 Check the new size before extending to avoid thereference of the null pointer * #1855 Changed count to processed_count for log messages. Initialized variables at get_filtered_nc_data\*() * #1855 Corected the number of memory copy * #1855 Exit with an error message if both fgi & bgi are null * #1855 Clear Lead_Times before extening. Formatting for error messages * Add definition for the compilation environment on seneca. * #1855 Extend Lead_Times for new times * #1855 Removed break after exit * #1855 define "c" dynamically * #1855 Move the log message within if statement to avoid dereferencing a null pointer * Feature 1843 scatter index (#1863) * Added scatter index variables: SI, SI_BCL, SI_BCU to cnt columns. SL * Per issue #1843 Added Scatter Index variables (SI, SI_BCL, SI_BCU) to the CNT stat type. SL * Added Scatter Index (SI, SI_BCL, SI_BCU) to cnt stat type header. SL * Per issue #1843, added Scatter Index variables to write_cnt_cols(). SL * Per issue #1843 added Scatter Index (si) to CNTinfo class. SL * Per issue #1843 added Scatter Index (SI, si) to CNTInfo::get_stat(). SL * Per issue #1843 added Scatter Index (SI, si) calculation to both versions of compute_cntinfo(). SL * Per issue #1843, forgot semi-colon after SI calculation, fixed. SL * Added Scatter Index (si) to some of the clear() and allocate() functions. SL * Per issue #1843, added Scatter Index (si) to compute_cnt_mean(). SL * Per issue #1843, added Scatter Index (si) to bootstrap interval calculation and to the CNT write function. SL * Per issue #1843 added Scatter Index (SI, SI_BCL, SI_BCU) to store_stat_cnt(). SL * Per issue #1843, in get_stat() moved Scatterd Index (SI) line to be below RMSE for consitency. SL * Added Scatter Index (SI) to the CNT output format table. SL * Added Scatter Index (SI) to CNT statistics definitions. SL * Update appendixC.rst * Update stat_columns.cc * Update compute_ci.cc * Update met_stats.cc * Per #1843, update write_cntinfo() function. Since we're READING the SI value immediately after the RMSE value when computing CI's, we also need to write SI immediately after RMSE. Otherwise, we'll be computing CI's using the wrong statistics replicates. * Per #1843, check for divide by zero when computing SI... this could happen if comparing the same input file to itself with Grid-Stat. * Per #1843, correct definition of SI from RMSE/ME to RMSE/OBAR. * Per #1843, updated the SI definition to divide by OBAR instead of ME, but forgot to update the divide-by-zero check. Co-authored-by: Seth Linden <linden@kiowa.rap.ucar.edu> Co-authored-by: johnhg <johnhg@ucar.edu> * #1855 Corected typo - compare the numnber of U and V * #1855 Check if var_info is hull * #1855 Avoid un-initialized variables: dt at open(), v at lat() & lon(), and local variables at data() * #1855 Check if ldf is null pointer * #1855 Removed break after exit * #1855 Removed break after exit * Update pcp_combine.cc * Feature 1864 config urls (#1868) * Per #1864, remove stale GitHub pages URL for the MET User's Guide. * Per #1864, found stale GitHub IO links in the issue templates. Should make this same fix in the develop and default branches for all the METplus repos. * Per #1864, fix stale GitHub IO link in the PR template. * Move ROC curve in Appendix C * Crop ROC image for better PDF formatting * Shrink ROC image for better PDF formatting * Modify figure alignment in conf.py * Change text placement for ROC curve figure * Feature 1834 faq (#1867) * testing possible new drop-down menus for FAQ * adding sphinx-panels to extensions to be able to use drop down menus in appendixA * Per #1834, adding sphinx-panels to requirements.txt to see if that fixes the error * Per #1834, added sphinx.ext to beginning of sphinx_panels to see if that fixes the error * Per #1834, remove sphinx.ext and add trailing comma to end of extensions to see if that fixed the error * Per #1834, added sphinx-panels to pip install command to see if that fixes the error * Per #1834, added extra space in comment to get docs to build to see if change to documentation.yml will fix an error * Per #1834, added return character to get rid of warning * testing dropdown menus #1 * testing dropdown menus #2 removing lines and dotted lines * testing #3 * testing 4 * fixing numbering, removing 2nd pulldown menu, trying to fix box width * adding code block in dropdown * trying to fix width * trying to fix width with spacing * trying to fix width with spacing 2nd dropdown * removing panels * solved problem, just use dropdown no panels * adding second dropdown menu * removing the dropdown menus and fixing the spacing. * fixing typos * adding in matched pairs * adding in formating files for NetCDF and making FILE_IO a header * fixing spacing * time slice * fixing spacing * UNIX time conversion * fixing spacing * fixed-width * scientific notation * removing which per Julie P * adding Gen_Vx_Mask section * fixing formatting * changing to numbers * complex masking region * neighborhood methods boundaries * Neighborhood Methods to Compute Fraction * adding Grid_Stat - How do I use Config File Setup to Read a NetCDF file * using backticks with asterics * one more backtick with asterics * Grid_Stat - What would be an example of Verifying Probabilities * fixing typos * What is an example of using Grid-Stat with Regridding and Masking Turned On * fixing typos * Grid_Stat - What would be an example of Verifying Probabilities Example2 * trying to fix typos * trying to fix code-block spacing * Per #1834, attempting to fix warning. * How do I use different masks in MET tools using MODE as an example * trying to fix bullet list * trying to fix bullet list 2 * trying to fix bullet list 2 * trying to fix bullet list 3 * Per #1834, attempting formatting changes * Per #1834, attempting formatting changes * Per #1834, attempting formatting changes * Per #1834, attempting formatting changes * Per #1834, attempting formatting changes * Per #1834, modified formatting, including adding backslashes and removing extra returns in command line calls, and adding indentation for config file notation, and adding a return after each 'A.' * Per #1834, changed code-block from ini to none to fix formatting * *Pcp-Combine - What are some examples using -add * Pcp-Combine - How do I add and subtract with Pcp-Combine * Pcp-Combine - How do I Combine 12-hour Accumulated Precipitation from Two Different Initialization Times * Pcp-Combine - How Do I Correct a Precipitation Time Range * Pcp-Combine - What Data Formats does MET Read * Pcp-Combine - How does pass through work * Pcp_Combine - How do I use -pcprx to run a project faster * Pcp-Combine - How Do I Enter the Time Format Correctly * Pcp-Combine - How do I use -subtract * fixing formatting * Pcp_Combine - How Do I Use -sum, -add, and -subtract to Achieve Same Accumulation Interval * Pcp-Combine - What is the difference between -sum vs. -add * Pcp-Combine - How Do I Select a Specific GRIB Record to Use * fixing spacing for original text * Plot-Data-Plane - How Do I Inspect Gen_Vx_Mask Output * fixing typo * Plot-Data-Plane - How Do I Specify GRIB Version * Plot-Data-Plane - last 2 entries * 2 misc. entries * fixing typo * Stat_Analysis - How does -aggregate_stat work * Stat_Analysis - remaining entries * fixing typo * tc-pairs * tc-stat * troubleshooting area * another stat-analysis * another stat-analysis * utilities * fixing dashes * Per #1834, trying out syntax * Per #1834, removing added syntax * fixing returns with bolding * Per #1834, modified formatting * adding italics for directories #1834 * replacing code-block ini with none to remove red lettering * fixing questions to lower case #1834 * linking to section using numref #1834 * linking to section using numref #1834 attempt 2 * linking to section using numref #1834 attempt 3 * linking to section using numref #1834 attempt 4 * linking to section using numref #1834 attempt 5 * linking to section using numref #1834 attempt 6 * linking to section using numref #1834 attempt 7 * linking to section using numref #1834 attempt 8 * linking to anothre section #1834 * linking to another section #1834 * removing old info #1834 * grammar updates #1834 * grammar updates #1834 * fixing typos #1834 * email changes with Julie #1834 * trying to fix warning #1834 * changes #1834 * Per #1834, updating question titles and answers * Per #1834, made updates to the content. * Per #1834, made updates to the content * Per #1834, made updates to the content * Per #1834, made some major changes to the first 3 FAQ's. I don't have the time/bandwidth to review these completely at this time, but wanted to include at least these updates. * testing some of the updates John O. suggested. * updates John O. suggested take 2 #1834. * updates John O. suggested take 3 #1834. * updates John O. suggested take 4 #1834. * updates John O. suggested take 5 #1834. * updates John O. suggested. Some didn't show up even though they've been removed #1834. * trying to fix numbered list indenting #1834 * fixing numbered list indenting #1834 * cleaning up formatting #1834 * fixing broken links #1834 Co-authored-by: Julie.Prestopnik <jpresto@ucar.edu> Co-authored-by: John Halley Gotway <johnhg@ucar.edu> * bugfix 1872 plot_tcmpr.R -lookin directory (develop) (#1874) * Bugfix 1875 develop memory (#1877) * Per #1875, fix set_cdf_ta() to initialize cdf_ta prior to populating it. * Per #1875, add debug(4) log messages whenever defining a new aggregation case in stat-analysis. * Per #1875, switch log messages about new cases from Debug(4) to Debug(3). * Per #1879, porting the same changes over to the develop branch. (#1881) * Feature 1673 gbeta (#1857) * Per #1673, add a distance_map.beta_value entry to all the Grid-Stat config files. * Per #1673, enhance Grid-Stat to parse the new beta_value option and set its value in the library code. * Per #1673, set the user-defined beta_value when processing the distance map scores. * Per #1673, add 4 new DMAP columns write them. Just writing bad data values right now, still need to actually compute them! Also need to update Stat-Analysis to update the parsing/aggregation of these new values. * Per #1673, enable beta_value to be set to a default bad data value. * Per #1673 and @ericgilleland instruction (#1673 (comment)), strip asymmetric G-Beta (AGBETA) from the output. * Per #1673, fix typo that was causing the code not to compile. * Per #1673, update MET User's Guide about G/GBETA. * Per #1673, report BETA_VALUE = TOTAL / 2 in the output instead of NA, as directed by Eric. * Per #1673, remove descriptions of Grid-Stat-specific config options for nbrhd, fourier, gradient, and distance_map from the overview section since they belong in the Grid-Stat chapter. Also add direct links for them in the Grid-Stat chapter. * Per #1673, do not reset the beta_value in DMAP::clear(). * Per #1673, log the DMAP options only after the beta_value has been determined. * Per #1673, correct the default beta_value setting as N*N / 2.0. * Per #1673, trying to format G-beta well. * Per #1673, trying to format G-beta well. * Per #1673, beta is formatted as \beta and not \Beta in latex. * Per #1673, beta is formatted as \beta and not \Beta in latex. * Per #1673, switch to defining beta_value as a function of the input grid size. * Per #1673, push initial appendix C updates for GBETA. * Per #1673, working on G and GBETA equations. * Per #1673, working on G and GBETA equations. * Per 1673, update GBETA details in appendix C. * Per #1673, add Eric's info about selecting beta. * Per #1673, update reference for Gilleland-2019 to Gilleland_PartI-2020 and Gilleland_PartII-2020. * Per 1673, fix typo. * Per #1673, making changes requested by Eric. * Per #1673, making changes requested by Eric. * Per #1673, update references based on Eric's feedback. * Update appendixD.rst (#1869) Updated this appendix. * Update appendixD.rst * Per #1673, making the formatting of Gilleland-2020 part I and II references consistent. * Per #1673, update Grid-Stat docs to clarify that GBETA is only computed on the FULL verification domain and not any masking regions. * Per #1673, add DataPlane::n_good_data() function to return a count valid data values. * Per #1673, update vx_statistics library code to only set gbeta/beta_value to bad data for sub-domains. * Per #1673, update grid_stat to mask out bad data between the fcst/obs fields before computing distance maps. This will change existing unit test results! Also store the count of valid data values in the full verification domain. * Per #1673, since GBETA is only reported when VX_MASK = FULL, updating GridStatConfig_all to include the FULL masking region to make this example more meaningful. Co-authored-by: ericgilleland <34106925+ericgilleland@users.noreply.github.com> * Feature 1764 point stat (#1885) * Per #1764 added write_orank_row() to do_hera_ens(). SL * Per issue #1764, added orank stat type for ensemble output. SL * Per issue #1764, added code to setup_txt_files() for ORANK output. SL * Added some comments for debuging. SL * Per #1764, add orank to the list of outputs created by point_stat. * Per #1764, add functions to compute the number of requested HiRA probabilities and ensemble members. Also update n_txt_row() to keep track of the number of ORANK rows to be written. Not totally sure if I'm counting these ORANK output lines correctly though. * Per #1764, update logic in setup_txt_files() to call get_max_n_hira_prob() and get_max_n_hira_ens() functions. * Per #1764, no real changes. Just fixing spacing. * Per issue #1764, cleaned up print (cout) debug statements. SL * Per issue #1764 added 'orank = NONE' to the config files for compatibility with changes to point_stat for calculating ORANK. SL * Per issue #1764 added 'orank = STAT' to config file to produce ORANK output lines in point_stat. SL * Per issue #1764 added content in all relevant sections for ORANK: observation rank statistic. SL * Added 'orank = NONE' to config file for compatibility with changes to point_stant for ORANK. SL * Added 'orank = NONE' to config file for compatibility with changes to point_stant for ORANK. SL * Apply suggestions from code review Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> * Per #1764, no real changes, just fixing indents. * Per #1764, no real changes, just fixing indents. * Per #1764, no real changes, just fixing indents. * Per #1764, putting the docs for VCNT back into its logical order with the other vector line types. The order that they're listed here does not need to match their order in the enumeration in the code. * Per #1764, fixing a vcnt typo I found in the grid-stat chapter... and making them consistent. * Per issue #1764: After the call to write_orank_row, added lines to reset the observation begin and end times. SL * Per #1764, no code changes, just fixing indents. Co-authored-by: Seth Linden <linden@kiowa.rap.ucar.edu> Co-authored-by: John Halley Gotway <johnhg@ucar.edu> Co-authored-by: John Halley Gotway <johnhg@kiowa.rap.ucar.edu> Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> * Adding Seth to the NB notification list. * Bugfix 1886 develop grid_diag (#1888) * Per #1886, port over the bugfixes from main_v10.0 to develop. * Per #1886, make grid_diag error out if to_grid is set to FCST or OBS. * Added additional default labels * Feature 1870 realtime (#1893) * Per #1870. add TCPairs config entries for valid_inc, valid_exc, and write_valid. * Per #1870, removing 2 files I accidentally added. * Per #1870, add code to parse the new tc_pairs config options. * Per #1870, unrelated... just fixing spacing. * Per #1870, update TrackPairInfo/Array classes and tc_pairs application code to use the write_valid option to subset the tracks by requested valid time before writing them. * Per #1870, update the User's Guide with the new config options. Document valid_inc/valid_exc in the common TC config section and add write_valid to the TC-Pairs chapter. * Per #1870, move the subset_write_valid() functionality to the library code and support it for both track and probability arrays. * Per #1870, update unit_tc_pairs.xml with 2 new calls to exercise the new write_valid functionality for both tracks and probabilities. * Per #1870, correct some typos in the docs... PROBRI and probri should be PROBRIRW and probrirw, resp. * Per #1870, switching match_points from TRUE back to FALSE after I acciedentally committed that change. * Per #1870, just fixing indent. Co-authored-by: John Halley Gotway <johnhg@kiowa.rap.ucar.edu> * Feature 1788 ssidx (#1892) * Per #1788, add initial definition of the SSIDX line type. * Per #1788, add a STAT-Analysis config file for computing the CBS Score. This is currently just a copy of the GO Index. * Per #1788, add hooks for computing -job cbs_score. * Per #1788, since we're adding a new SSIDX line type, the columns for that line type replace the job_ss_columns and job_go_columns arrays. * Per #1788, rename cbs_score to cbs_index per UK Met Office instruction on 7/22/21. This convention is also consistent with the existing go_index and ss_index jobs. * Per #1788 rename STATAnalysisConfig_CBS_Score to STATAnalysisConfig_CBS_Index * Per #1788, update Makefile to handle name change. * Per #1788, add support for SSIDX stat line type. * Per #1788, define 4 output SSIDX columns. * Per #1788, define an SSIndexInfo struct for stashing/writing SS Index values. * Per #1788, write the SSIDX columns. However we don't need to function to write the whole row since it's only written by Stat-Analysis. * Per #1788, create an AggrSSIndexInfo struct to keep track of unique stat header column input values. * Per #1788, update the stat-analysis job code to make use of the new structs. * Per #1788 fix typo. * Per #1788, remove GO Index defintion from the code comments. That detail belongs elsewhere. * Per #1788, since coding up the GO Index several CNT and CTC columns have been added. Adding them here to let them be included as part of the Skill Score index computations. * Per #1788, update check_hdr_str() to print the name of the problematic header columns. * Per #1788, add a set(one) member function to multiple array classes. This calls clear or erase on the array prior to adding a single element. Add this for StringArray, NumArray, and ThreshArray, and CRCArray. * Per #1788, update the compute_ss_index() function in stat_analysis to make the code more concise by calling the newly adding singular set functions for arrays. * Per #1788, add FCST_MODEL and REF_MODEL output columns to the SSIDX line type. * Per #1788, add a new N_INIT column to the SSIDX line type to report the number of initialization times represented by this skill score. Also define the SSIDXData struct to store the skill score index job output. * Per #1788, update STATAnalysisJob to explicitly set the job type directly rather than parsing a string. * Per #1788, major change here adding skill_score_index_job.h/.cc to handle the computation of the skill score indices. * Per #1788, update stat_analysis source code to make use of the new classes in skill_score_index_job.h/.cc. * Per #1788, write a log message listing the model initialization times over which the skill score index is computed. * Per #1788, tweak log message. * Per #1788, compute n_term as the max lenght of the required arrays. * Per #1788, drop in the ACTUAL definition of the GO index provide by the DTC AF project team. Also simplify the existing arrays down to lenght one if the value remains constant. * Per #1788, correct CBS Index config file. Some parameters had 40 elements while others only had 32. * Per #1788, do not write SSIDX output if there are no valid terms found. * Per #1788, format the values in the skill score index log messages, replacing -9999 with NA. * Per #1788, update the get_stat() member functions. For 0 pairs, return bad data for all statistics other than TOTAL. * Per #1788, working on readability of the warning and error messages. * Per #1788, define the ss_index_name directly in the STATAnalysis config file to make it really easy to modify. * Per #1788, store the ss_index_name configuration option in the StatJob class. Also support a command line -ss_index_name option to override the config file value. * Per #1788, change from name to ss_index_name variable to be a bit more descriptive in the variable names. * Per #1788, since ss_index_name is in the StatJob class we no longer need to pass it around as an argument. * Per #1788, simply the skill score index job handling logic a bit. We only need to update the default job once, not twice.: * Revert "Per #1788, since ss_index_name is in the StatJob class we no longer need to pass it around as an argument." This reverts commit cde0be0. * Revert "Revert "Per #1788, since ss_index_name is in the StatJob class we no longer need to pass it around as an argument."" This reverts commit bbfd838. * Per #1788, I'd pruned too much code from stat_analysis.cc which caused the processing of config file jobs to fail. Fixing that here. * Per #1788, switch Z0 to L0 to match the sample data for computing CBS Index. * Per #1788, make it so that cbs_index and go_index work from config file. * Per #1788, update the documentation for the skill score index changes. * Per #1788, just move some variable definitions further down since they're only used sometimes. * Per #1788, make job output description into a numbered section. * Per #1788, little tweak. * Per #1788, update the STATAnalysisConfig files to add ss_index_name and hss_ec_value, as needed. * Per #1788, add new unit tests to exercise recent ss_index changes and update the Stat-Analysis documentation. * Update met/docs/Users_Guide/stat-analysis.rst Co-authored-by: jprestop <jpresto@ucar.edu> * Update met/docs/Users_Guide/stat-analysis.rst Sorry for the typos! Thanks for catching them. Co-authored-by: jprestop <jpresto@ucar.edu> * Update met/docs/Users_Guide/stat-analysis.rst Co-authored-by: jprestop <jpresto@ucar.edu> * Per #1788, fix typo in error message. * Per #1788, fix a bug for ss_index... changing 'add' to 'set'. Also set the line_type seperately for each term, if requested. * Per #1788, tweak the ss_index logic slightly to make it more efficient. For each input STATLine, instead of checking all the index terms, stop searching after the first match. This assumes that each input line should be used for only one of the terms... and I think that's a pretty reasonable assumption. * Per #1788, minor tweaks to the STAT-Analysis docs. * Per #1788, add ss_index_vld_thresh option to existing STAT-Analysis config files. * Per #1788, add StatAnalysisJob ss_index_vld_thresh entry and parse it from the STAT-Analysis config file. * Per #1788, update stat_analysis code to check ss_index_vld_thresh setting when computing the skill score index. Also update the Stat-Analysis chapter of the User's Guide. * Per #1788, tweak one test to exercise a non-default ss_index_vld_thresh value. Co-authored-by: jprestop <jpresto@ucar.edu> * #1855 Initialize AllocInc * 855 call snprintf instead of sprintf * Avoid the same for loops (SobarQube warns this) * #1855 Checking the minimum rows (3) for formatting. Initialize left & right array * #1855 Removed the unreachable return statement * #1855 Avoide out of index for v_miss * #1855 Keep buf_size for for-loop (avoid changing it in for loop). The buffer size for strncpy is decided by the targetbuffer size or the input length, not the existing data at the target buffer. The size of existing data at the target buffer is used to erase the previous data * #1855 Make sure the variuble c has enough data to avoid out of index error * #1855 nake sure the index is in range (avoid out of index error) * #1855 Make sure no negative offset for s array * #1855 Allocated line before calling getline'. Simplify the code to check the first level only. Check pbl_level before provcessing * #1855 Added m_strlen, m_strcpy, m_strcpy2, and m_strncpy * #1855 Added m_strlen, m_strcpy, m_strcpy2, and m_strncpy> Check if null pointer * #1855 Calls m_strcpy intead of strcpy * #1855 Calls m_strcpy2 intead of strcpy * #1855 Calls m_strcpy intead of strcpy * #1855 Removed spaces at the empty line * Bugfix 1891 gen_vx_mask (#1895) * Per #1891, tighten up the gen_vx_mask logic a bit. Use a global config for the default NC compression level. Then read the input and mask fields using local variables instead of the global one to ensure they don't affect eachother. Also took the opportunity to refine the logic so that we only create a mtddf object in one spot, in the get_data_plane() function. * Per #1891, add a new test in unit_gen_vx_mask.xml that fails prior to #1891 and succeeds with these changes. * Per #1891, file name typo in unit_gen_vx_mask.xml. * #1855 sonarqube: replaced strlen to m_strlen * #1855 sonarqube: replaced strcpy to m_strcpy * #1855 sonarqube: replaced strlen to m_strlen * #1855 sonarqube: replaced strlen to m_strlen * #1855 sonarqube: replaced strlen to m_strlen * #1855 Replaced strncpy to m_strncpy and enhance m_strcpy & m_strncpy (#1901) Co-authored-by: Howard Soh <hsoh@kiowa.rap.ucar.edu> * #1855 Turn off new strcpy functions * #1855 Corretced typo (arguments were swapped at MetPythonDataFile::open) * #1855 Make sure the to_str is null terminated after strncpy * Feature 1827 v10.1.0 beta2 (#1902) * Per #1827, updated release notes * Per #1827, Added extra ~ under header * Update met/docs/Users_Guide/release-notes.rst Co-authored-by: johnhg <johnhg@ucar.edu> * Update met/docs/Users_Guide/release-notes.rst Co-authored-by: johnhg <johnhg@ucar.edu> * Update met/docs/Users_Guide/release-notes.rst Co-authored-by: johnhg <johnhg@ucar.edu> * Per #1827, rereferenced GitHub issue number Co-authored-by: Julie Prestopnik <jpresto@seneca.rap.ucar.edu> Co-authored-by: johnhg <johnhg@ucar.edu> * #1903 Added str_wrappers.cc & str_wrappers.h. They are moved from string_fxns * #1903 Added enum_to_string.h. They are moved from enum_to_string.cc * #1903 Moved include statements to the top. Included enum_to_string.h & str_wrappers.h. Changed 0 to NULL to reset the pointer * #1903 Moved m_strXXX to str_wrappers. Includes str_wrappers.h instead of string_fxns.h * #1903 Changed strXXX to m_strXXX * #1903 Chaged dst_info to const * #1903 Separated two || conditions to avoid potential side effect * #1903 Avoid the negative offset if the string length is 0 * #1903 make sure the pointer is not NULL before releasing * #1903 Replaced strXXX to m_strXXX. Separated two || conditions to avoid the potential side effects * #1903 Allow to truncate the string on calling m_strncpy * #1903 Replaced strXXX calls to m_strXXX * #1903 Avoid the negative offset if the string length is 0 * Separated third AND condition to avoid a possible side effect (assigned lead) * #1903 Renamed import (which is a keyword) to import_image) * Initialize data at get_filtered_nc_data_2d * #1903 Initialize r, g, b at dhsv_to_drgb * Modified to avoid unreachable return statement * #1903 Separated the second AND condition to avoid a potential side effect (assignment at the second condition) * #1903 Removed unreachable break (after exit) * #1903 Removed a unused offset variable * #1903 Initialize fcst variable * Renamed module (keyword) to py_module * Removed the second argument at m_strlen * #1903 Deleted extra two empty lines * #1903 Formatting (tab to spaces) * #1903 mstcpy is changed to m_strcpy2 * Deleted the commented out code * #1903 Calls sizeof instead of m_strlen at m_strcpy * #1903 Calls m_strncpy instead of m_strcpy * #1903 Replaced m_strcpy to m_strncpy * #1903 Do not use sizeof * Feature 1912 missing (#1915) * Per #1912, update logic of Met2dDataFile::process_data_plane(). Rather than returning bad status for a field of all bad data, print a warning message and continue processing. * Per #1912, fixing typo in error message that I ran into. Doesn't actually apply to this issue. * Per #1912, making an unrelated change here to fix up the MODE-Analysis log output. It was reporting (nul) for the number of lines being read/kept. Simplify the log messages to print the correct numbers. * Testing RTD updates * Added sphinx-panels * Testing double hyphen change to code-block * Testing double hyphen change to code-block * Changed bolding of options to code-blocks * Feature 1858 obs quality (#1919) * Per issue #1858: changed parse_conf_obs_qty to parse_conf_obs_qty_inc. Added new function parse_conf_obs_qty_exc. SL * Per issue #1858: changed conf_key_obs_qty to conf_key_obs_qty_inc. Added conf_key_obs_qty_exc. SL * Per issue #1858: modified vx_pd.set_obs_qty_filt() to use parse_conf_obs_qty_inc() instead of parse_conf_obs_qty(). SL * Per issue #1858: added back the original parse_conf_obs_qty() function for backward compatibility. SL * Per issue #1858: added back config_constants.h for backwards compatibility. SL * Per issue #1858: added code to read in obs_qty_inc instead of obs_qty. Also added in code for backwards compatibility to check for obs_qty (obs_quality) and use that for obs_qty_inc. SL * Changed obs_qty_filt to obs_qty_inc_filt. Added obs_qty_exc_filt, including relevant function. SL * Per issue #1858 added vx_pd.set_obs_qty_exc_filt. SL * In add_point_obs() added if block to apply obs_qty_exc_filt. SL * Per issue #1858, in add_point_obs(), cleaned up sections that apply obs_qty_inc and obs_qty_exc filters. Fixed spacing issue. SL * Per issue #1858 in process_config(), cleaned up sections that set obs_qty_inc_filt and obs_qty_exc_filt. Fixed spacing issue. SL * Per issue #1858 in process_obs_file() cleaned up code. SL * Per issue #1858: changed obs_qty_filt to obs_qty_inc_filt. Added obs_qty_exc_filt. Modified add_point_obs() to use obs_qty_exc_filt. SL * Per issue #1858: in process_config(): added code to set_obs_qty_inc_filt and set_obs_qty_exc_filt. Also added code to check if old obs_quality is being used, if so, set it to obs_qty_inc_filt. SL * Per issue #1858: in add_point_obs() modified section that applies the obs_qty_inc and obs_qty_exc filters. Combined the if-blocks into one concise if-block that applies both filters. SL * Per issue #1858: replaced obs_quality with obs_quality_inc and added obs_quality_inc. SL * Per issue #1858: replaced obs_quality with obs_quality_inc and added obs_quality_exc. SL * Per issue #1858: For both the point_stat and ensemble_stat tests: modified tests that use OBS_QTY to use OBS_QTY_INC. Added unit test for OBS_QTY_EXC (obs_quality_exclude). SL * Per issue #1858: just added comment at top of file indicating what changed. SL * Per issue #1858: just added comment at top of file indicating what changed. SL * Per issue #1858: removed parse_conf_obs_qty. Modified parse_conf_obs_qty_inc to check for old 'obs_quality' entry. SL * Per issue #1858: in process_config(), modified section that calls set_obs_qty_inc_filt...this new function can now handle old 'obs_quality' entry as well as the new 'obs_quality_inc'. SL * Modified parse_conf_obs_qty_inc and parse_conf_obs_qty_exc. SL * Per issue #1858: re-updated parse_conf_obs_qty_exc and reverted back to the simplier version of this function. SL * Per issue #1858: replaced obs_quality with obs_quality_inc and added obs_quality_exc. SL * Per issue #1858: replaced obs_quality with obs_quality_inc and added obs_quality_exc. SL * Per issue #1858: replaced obs_quality with obs_quality_inc and added obs_quality_exc. SL * Per issue #1858: replaced obs_quality with obs_quality_inc and added obs_quality_exc. SL * Per issue #1858: replaced obs_quality with obs_quality_inc and added obs_quality_inc. SL * Per issue #1858: replaced obs_quality with obs_quality_inc. Added documentation for obs_quality_exc. SL * Per issue #1858: checking in latest version. Modified section that calls set_obs_qty_inc_filt...this new function can now handle old 'obs_quality' entry as well as the new 'obs_quality_inc'. SL * Per issue #1858: cleaned up some extra line spaces. SL * Per #1858, removed the deprecated obs_quality configuration entry from 3 EnsembleStat and 5 PointStat configuration files. * Per issue #1858: duplicated fcst/obs fields to run/test with both obs_quality_inc and obs_quality_exc. SL * Per issue #1858: duplicated fcst/obs fields to run/test with both obs_quality_inc and obs_quality_exc. SL * Per issue #1858. Deleted these files git repository. These were replaced with: PointStatConfig_qty_inc_exc. SL * Per issue #1858. Deleted these files from the git repository. These were replaced with: EnsembleStatConfig_qty_inc_exc. SL * Per issue #1858. Deleted these files from the git repository. These were replaced with: PointStatConfig_qty_inc_exc. SL * Per issue #1858: Modified: Don't need to run separate tests for obs_quality_inc and obs_quality_exc, these are now combined into one test via the updated config files: PointStatConfig_qty_inc_exc and EnsembleStatConfig_qty_inc_exc. SL * Per #1858, update the logic in parse_conf_obs_qty_inc() to use last_lookup_status(). See forthcoming GitHub issue comment about this. * Per #1858, tweak the warning message. * Per #1858, made the existing unit_quality_filter.xml tests a bit more concise. Call Point-Stat and Ensemble-Stat only once each time. But for each, test 3 things: all qty, obs_quality_inc, and obs_quality_exc. * Per #1858, remove the _ON from the job name since the _OFF has been included in the same configuration. Co-authored-by: Seth Linden <linden@kiowa.rap.ucar.edu> Co-authored-by: John Halley Gotway <johnhg@ucar.edu> Co-authored-by: John Halley Gotway <johnhg@kiowa.rap.ucar.edu> * Feature 1894 pb2nc var all (#1917) * #1894 Added station ID 14008 to pb2nc_NDAS_var_all unit test * #1894 Check the existance of the valid data up to 16 levels Co-authored-by: Howard Soh <hsoh@kiowa.rap.ucar.edu> * Feature 1913 MAX_PBL (#1916) * #1913 Changed MAX_PBL to 10000. Filter out if the PBL input data is invalid * #1912 Replaced is_eq to is_bad_data * #1913 Changed is_eq to is_bad_data Co-authored-by: Howard Soh <hsoh@kiowa.rap.ucar.edu> * Add bolding the MET version 10.1.0 beta2 release notes to indicate emphasis prior to the METplus Governance meeting. * feature 1922 iodas2nc - Removed trailing spaces based on the string length (#1923) * #1922 Removed trailing spaces based on the string lenth than buffer length * #1922 Renamed cleanup_hdr_bfr to m_rstrip and moved to str_wrappers.cc Co-authored-by: Howard Soh <hsoh@kiowa.rap.ucar.edu> Co-authored-by: jprestop <jpresto@ucar.edu> Co-authored-by: Howard Soh <hsoh@kiowa.rap.ucar.edu> Co-authored-by: hsoh-u <hsoh@ucar.edu> Co-authored-by: Keith Searight <keith.searight@noaa.gov> Co-authored-by: Seth Linden <linden@ucar.edu> Co-authored-by: Seth Linden <linden@kiowa.rap.ucar.edu> Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> Co-authored-by: lisagoodrich <33230218+lisagoodrich@users.noreply.github.com> Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> Co-authored-by: ericgilleland <34106925+ericgilleland@users.noreply.github.com> Co-authored-by: John Halley Gotway <johnhg@kiowa.rap.ucar.edu> Co-authored-by: MET Tools Test Account <met_test@kiowa.rap.ucar.edu> Co-authored-by: Julie Prestopnik <jpresto@seneca.rap.ucar.edu>
JohnHalleyGotway
added a commit
that referenced
this pull request
Nov 3, 2022
* Initialze obs_data at constructor to avoid warninjg on building * Initialze obs_data at constructor to avoid a warning by compiler * #1824 ci-run-test Removed the duplicated code * Feature 1810 expand asciitable (#2086) * Feature 1583 skip_mean (#2090) * Feature 1275 MODE Object Count (#2091) * for #1275, fix issue with more than 1000 object for mode, ci-run-unit * Per #1275, just deleting stale, commented-out code. Co-authored-by: Randy Bullock <bullock@seneca.rap.ucar.edu> * Feature 1184 dryline (#2088) Co-authored-by: davidfillmore <fillmore.winslow.david@gmail.com> Co-authored-by: rgbullock <bullock@ucar.edu> Co-authored-by: John Halley Gotway <johnhg@kiowa.rap.ucar.edu> * Feature 1184 replace_env (#2093) * Feature 2092 v10.1.0-rc1 (#2094) * Hotfix to the develop branch to fix the MODE Makefile for the MET-10.1.0-rc1 release. * Second hotfix for the MODE Makefile in the develop branch. * Feature 1844 docs (#2096) * Feature 2097 v10.1.0 (#2099) * Feature 2098 gen ens prod doc (#2100) * update job control rules for GHA testing workflow to use version specific input data if running tests on main_v branch * Migrating a small tweak added to the v10.1.0 release into the develop branch. * Adding SonarQube location to development.seneca file. * Bugfix #2102 develop initialize modified_hdr_typ (#2104) Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu> Co-authored-by: johnhg <johnhg@ucar.edu> * added logic to manually trigger a workflow via the GitHub… (develop) (#2109) Co-authored-by: John Halley Gotway <johnhg@ucar.edu> * Feature update installation info (#2114) * Updating configuration and modulefiles * Removed jet files * create DockerHub tag that ends with -lite to use when triggering the METplus testing workflow * Bugfix #2115 develop Rotated LatLon (#2117) * Feature 1950 sort station list (#2121) * Update the top-level README file to list ioda2nc, but also to trigger GHA to run for this new main_v10.1-ref branch. * Added bool Sorted variable, set to true or false in relevant functions. Updated sort() function. SL * In parse_sid_mask(), added the line: mask_sid.sort(). SL * Per issue #1950, put in some cout statements for testing. SL * Per issue #1950, working in has() function, checking for Sort and put in initial cout's for testing. SL * Per issue #1950: Cleaned up some cout (print) statements. Left one in there that is commented out for future testing. SL * Per issue #1950: modified has() function. Added lower_bound search for sorted arrays. Left some print (cout) statements in there for further testing. SL * Per issue #1950: In has(), I commented out the print (cout) statements for time testing. SL * added logic to manually trigger a workflow via the GitHub… (#2107) * Per issue #1950, put back in some print (cout) statements for testing. SL * Per issue #1950: in has() function, experimenting using the upper_bound() search instead of lower_bound() for sorted vectors. In progress. SL * Per issue #1950: added a new has function that uses binary_search. Re-working the other has() functions related to this. SL * Per issue #1950: re-worked original has(), basically reverted back to what it was...but it's now only used for unsorted vectors or for a case insensitive search. Cleaned up. SL * Per issue #1950: in section that checks Obs Station Id in SID masking list, changed cout to a mlog (debug) statement. SL * Per issue #1950: created new mask station id list (CONUS METARs) and modified the PointStatConfig_MASK_SID config file to use this list along with the existing lists. SL * Per issue #1950: reverted back to orginal sid mask site-lists. SL * Per issue #1950: checking in this site-list that was used for testing point_stat. SL * Per issue #1950, modified and added better debugging to figure out current issue with ascii2nc. SL * Per #1950, small tweak in ascii2nc to make it slightly more efficient. After adding the new obs varable name to the end of the list, we can compute var_index direclty without needing to call StringArray::has() a second time. * Per #1950: removed mlog and cout print statements from has() functions. Cleaned up. SL * Update met/src/basic/vx_log/string_array.cc Co-authored-by: johnhg <johnhg@ucar.edu> * Update met/src/basic/vx_log/string_array.cc Co-authored-by: johnhg <johnhg@ucar.edu> * Update met/src/basic/vx_log/string_array.cc Co-authored-by: johnhg <johnhg@ucar.edu> * Update met/src/basic/vx_log/string_array.cc Co-authored-by: johnhg <johnhg@ucar.edu> * Update met/src/basic/vx_log/string_array.cc Co-authored-by: johnhg <johnhg@ucar.edu> * Update met/src/basic/vx_log/string_array.h Co-authored-by: johnhg <johnhg@ucar.edu> * Delete SID_CONUS.txt * Delete SID_CONUS_times11.txt Co-authored-by: John Halley Gotway <johnhg@ucar.edu> Co-authored-by: Seth Linden <linden@seneca.rap.ucar.edu> Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> * Bugfix #2118 develop grib1_rotll (#2130) Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> * Bugfix 2123 nccf valid time (#2127) Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu> Co-authored-by: johnhg <johnhg@ucar.edu> * Feature #2132 version 11.0.0 (#2133) * Bugfix #2106 develop gcc (#2135) Co-authored-by: Julie Prestopnik <jpresto@seneca.rap.ucar.edu> Co-authored-by: johnhg <johnhg@ucar.edu> Co-authored-by: Seth Linden <linden@kiowa.rap.ucar.edu> Co-authored-by: John Halley Gotway <johnhg@ucar.edu> Co-authored-by: j-opatz <59586397+j-opatz@users.noreply.github.com> Co-authored-by: Howard Soh <hsoh@kiowa.rap.ucar.edu> Co-authored-by: John Halley Gotway <johnhg@kiowa.rap.ucar.edu> Co-authored-by: jprestop <jpresto@ucar.edu> Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu> Co-authored-by: Randy Bullock <bullock@seneca.rap.ucar.edu> Co-authored-by: davidfillmore <fillmore.winslow.david@gmail.com> Co-authored-by: rgbullock <bullock@ucar.edu> Co-authored-by: Seth Linden <linden@ucar.edu> Co-authored-by: hsoh-u <hsoh@ucar.edu> Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> Co-authored-by: John Halley Gotway <johnhg@seneca.rap.ucar.edu> Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> Co-authored-by: mo-mglover <78152252+mo-mglover@users.noreply.github.com> Co-authored-by: davidalbo <dave@ucar.edu> Co-authored-by: lisagoodrich <33230218+lisagoodrich@users.noreply.github.com> Co-authored-by: Dan Adriaansen <dadriaan@ucar.edu> * Doc-only update to expliciltly list the maximum character lengths for variable-length strings passed to ascii2nc. * Added pcp_combine -subtract example (#2153) * Add testing and docs status badges. * Feature #2152 poly_xy (#2154) * #14 Using literal instead of numbers and make sure no overflow * #14 Using literal instead of numbers * #14 Added tmp_buf_size * #14 Define n_kw_infos first * #14 Using literal instead of numbers * Bugfix #2148 develop misses (#2150) * dtcenter/METplus-Internal#14 formatting code * METplus-Internal #16 Added python scanning * METplus-Internal #16 changed name * METplus-Internal #16 initial release for python scanning * METplus-Internalv#19 added maic.cc & main.h * METplus-Internalv#19 renamed main to met_main and added get_tool_name * METplus-Internalv#19 Initial release * changing 'Linked issue' to 'Development issue' * change Linked issue/Development issue ci-skip-all * change Linked issue/Development issue ci-skip-all * change Linked issue/Development issue ci-skip-all * Feature 1920 repo (#2161) * List line types that can be aggregated per dtcenter/METplus#1625 * Hotfix for the develop branch to correct a bad MET_TEST_BASE path that causes the nightly build to fail. * Feature 2072 tc rmw winds (#2165) * bug fix for NAN and for using missing data as if its not missing * added parameters to control converting winds to tangential/radial components * bug fixes * Refactored to put wind conversion into it's own class TCRMW_WindConverter * changed default for converting winds to TRUE, and put parameters for wind conversion into the unit test config files * Per #2072, updating tc_rmw Makefile.in to compile new files. * Per #2072, update to the output from the lex/yacc version on seneca. * Changed Error to Warning in mlog messages, for consistency with MET conventions * Removing commented out code. * Per #2072, combine warnings spread across multiple lines into a single call to the logger. Replace references to ugrid/vgrid with u/v wind. Fix a typo and clean up some whitespace. Co-authored-by: Dave Albo <dave@seneca.rap.ucar.edu> Co-authored-by: John Halley Gotway <johnhg@ucar.edu> * change Linked issue/Development issue ci-skip-all * Bugfix #2173 develop shift_data_plane (#2175) * Bugfix #2170 develop agg_ecnt (#2177) * Feature 2022 anom corr to vcnt (#2171) * Per #2022: Added ANOM_CORR, ANOM_CORR_BCL, ANOM_CORR_BCU to vcnt_columns. SL * Per #2022: Added ANOM_CORR, ANOM_CORR_BCL, ANOM_CORR_BCU to write_vcnt_cols. SL * Per #2022: Added ANOM_CORR to VL1L2Info. SL * Per #2022: updated the VL1L2Info functions to initialize and compute ANOM_CORR. SL * Per #2022: in calc_ncep_stats modified calculation for ANOM_CORR to check for bad data in the denominator first. SL * Per #2022: for the VL1L2Info, changed all derived VCNT variables to be CIInfo objects (instead of doubles). SL * Per #2022: making changes to VL1L2Info related to making all VCTN variables CIInfo objects. In progress. SL * Per #2022: Added ANOM_CORR_UNCNTR. SL * Per issue 2022: Modified functions in the VL1L2Info class to set the NCEP stat variables correctly since they are now CIInfo class memembers. SL * Per issue 2022: Added ANOM_CORR_UNCNTR (_BCL, _BCU). SL * Per issue 2022: modified write_vcnt_cols(), added ANOM_CORR_UNCNTR (_BCL, _BCU). SL * Per issue #2022: updated write_vcnt_cols. VCNT variables are now CIInfo objects to have to .v for the value. SL * Per issue #2022: In calc_ncep_stats, put in place-holder for ANOM_CORR.v value SL * Per issue #2022: in calc_ncep_stats(), changed placeholder for ANOM_CORR. SL * Per issue #2022: moved alpha initialization to the init_from_scratch() function. SL * Per issue #2022: in calc_ncep_stats(), set ANOM_CORR to bad_data_double as a placeholder for now. SL * Per issue #2022: in vcnt_cols(), added ANOM_CORR_NCL/NCU. SL * Per issue #2022: in write_vcnt_cols(), added ANOM_CORR_NCL/NCU variables. SL * Per #2022, adding new VCNT columns to the header table files, User's Guide documentation, and lining some code up more consistently. * Per #2022, added FA_SPEED_BAR and OA_SPEEED_BAR to the VAL1L2 line type. Added code to compute the ANOM_CORR confidence interval. However, I still need to update the writing of the VCNT line type to set the alpha value and actually write the computed CI's. * Per #2022, update the writing of the VCNT output line to write the alpha value and anomaly correlation CI's. * Per #2022, update point_stat and grid_stat to allocate alpha values for VL1L2Info objects to store CI's. * Per #2022, call VL1L2Info::zero_out() instead of clear() so that we don't throw away allocated alpha values. * Per #2022, update the Point-Stat and Grid-Stat logic to include the number of alpha values when deciding how many VCNT lines to write. * Per #2022, update the VL1L2Info::operator+= logic to copy over the allocated alpha values. * Per #2022, enhance stat_analysis to support an aggregate_stat job type for VAL1L2->VCNT. This writes ANOM_CORR and ANOM_CORR_UNCNTR output columns. However, there are still things to address. The other stats should be written as NA instead of 0. Recommend changing the VL1L2Info::zero_out() function to intialize stat values to NA instead of 0. Need to update the docs to reflect this support. Also need to actually compute the parametric CI's for ANOM_CORR. Big picture, we should store the VCNT stats in a new VCNTInfo class... just like the SL1L2Info/CNTInfo setup. Storing the sums AND stats in the same VL1L2Info class is unnecessarily confusing. * Per #2022, update Stat-Analysis docs to list VAL1L2 -> VCNT as a supported conversion type. * Per #2022, when computing VL1L2Info CI's, n should be set to vacount rather than the number of input pairs. * Per #2022, add a new call to Stat-Analysis to exercise the VAL1L2 to VCNT conversion. * Per #2022, ci-run-unit fix an aggregation bug in VL1L2Info::operator+=(). * Per issue #2022, added _vcnt.txt output files to check to relevant tests based on if vcnt output file is being generated via the config file. SL Co-authored-by: Seth Linden <linden@seneca.rap.ucar.edu> Co-authored-by: John Halley Gotway <johnhg@ucar.edu> * #1815 Renamed time_as_offset to Is_offset * #1815 Added vx_data2d_dim_by_value * #1815 Added convert_value_to_offset and renamed _time_dim_offset to _cur_time_index * #1815 Added convert_value_to_offset and renamed _time_dim_offset to _cur_time_index. Support value instead of offset for time and z level * #1815 Support @ for the value insteaed of offset for slicing * #1815 Support value insteaed of offset for slicing * #1815 Deleted unused local variable * #1815 Deleted unused local variable * #1815 Added level by value with @ * #1815 Added plot_data_plane_NCCF_latlon_by_value * #1815 Added level by value with @ * Feature #2180 develop kft (#2181) * dtcenter/METplus-Internal#16 Moved from scripts/sonarqube to internal/scripts/sonarqube * #1815 Updated level * Feature 2022 stat_analysis (#2185) Proceeding with the merge without an external review based on the details listed above. * Feature #2147 HSS (#2169) * #1815 call is_eq to comapre two double values * Per #2147, this is a HOTFIX to the develop branch after PR #2169 added new CTC and CTS columns. Prevoiusly the CTC columns = NBRCTC columns and CTS = NBRCTS. But that is no longer the case. This updates write_nbrctc_cols() and write_nbrcts_cols() to actually write them rather than just calling the functions to write CTC and CTS cols, respectively. * #2187 Added add_offset_att_name & scale_factor_att_name * #2187 Apply add_offse or scale_factor if defined * ##2187 Removed handling add_offset and scale factor. They are applied at the base API * #2187 Changed a log message * Per #2193, add release notes for met-11.0.0-beta1. (#2195) Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> * Hotfix to develop since I forgot to update the version number in conf.py. * #1815 Added more attribute names * #1815 Added get_index_for_dim * #1815 Formatting * #1815 Support a differnt dimension name from thw first data variable * #1815 Use constants for attr names * Hotfix to the develop branch to fix paths in Dockefiles after restructuing the MET repo. * Another hotfix to develop to get the Docker build step working again. The failure from the configure script is caused by the version of the "alocal" executable. The contents of develop are based on seneca, which has: aclocal (GNU automake) 1.16.1 By comparison, my laptop does NOT require that bootstrap be run for: aclocal (GNU automake) 1.16.5 But the Docker image has an older version which triggers the error: aclocal (GNU automake) 1.13.4 Any machine with aclocal 1.16.x works fine. Likely any machine with a version that differs from 1.16.x does not. Running bootstrap fixes that... but also changes the contents of all the Makefile.in files making them appear as a large set of diffs in the repo. This is not an issue inside the docker image since we generally remove the source code after compiling anyway. * Updating the png version of the older flowcharts. * #1815 cleanup * #1815 cleanup * #1851 cleanup * #1851 cleanup * #1815 Added get_att_unixtime * #1851 cleanup * #1815 cleanup * #1815 cleanup * #1815 Support @ for pinterp * #1815 Do not call to_string * #1815 cleanup * Per issue #2081, updated thecontents to match MRMS Version 12.2. Updated existing entries, removed old entries, added new entries. SL (#2199) Co-authored-by: Seth Linden <linden@seneca.rap.ucar.edu> * #1815 Changed @time to time at plot_data_plane_NCCF_time * Revert "Feature 1815 level value" (#2201) * #1815 Use NcVarInfo members instead API calls * #1815 Added find_var_by_dim_name * #1815 Restored changes for this branch. The changes were reset after revert and merging with latest develop branch * #METplus-Internal_19 cleanup * #METplus-Internal_19 Added more functiofnds and corrected typo * #METplus-Internal_19 Added main.h and main.cc * #METplus-Internal_19 adjusted link order to set oom * #METplus-Internal_19 Renamed main to met_main. Moved oom handler to base API * #METplus-Internal commented out segv_handler which failed to compile at Docker * Fix typo - maintianing * Feature #2204 sum (#2205) * #2202 Added APIs to check the point data (missing data and data types) * #2202 Renamed pointdata_from_xarray to pointdata_from_python_list * #2202 Support python list * #2202 Avoid seg fault by checking point data from python embedding and provides the better log messages * #2202 Support python list * #2202 Added APIs to check the point data (missing data and data types) * #2202 Added APIs to check the point data (missing data and data types) * #2202 Added APIs to check the point data (missing data and data types) * #2202 Added MaskedArray to allowed array data type * Per #2211, enhance warning/error messages when parsing threshold types in dictionary.cc. * #2208 Check existence of ensemble member data before checking bad_data * 2215 Added station_ob to message_type & station_id to station_id for metadata_map * 2215 Add log message for missing message_type and station_id metadata variables. Set NA if a message type is empty string * 2215 Filter out XXX@RecMetaData variables from @metadata variables * Per #2215, update the user's guide to make it consistent with the default ioda2nc config file. * #METplus-Internal_19 Renamed main to met_main and added get_tool_name * Feature sonarqube v11 beta2 (#2222) Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu> * Per #2220, update the version number and release notes. * Per #2220, fix typo. * Per #2220, format the METplus-Internal release notes for MET in the same way they were handled for the METplus beta1 release. * Added *hpp * Separated template functions * Separated template functions * Changed argument for get_var_fill_value * Feature 1971 mtd error to warning (#2221) Co-authored-by: Dave Albo <dave@seneca.rap.ucar.edu> Co-authored-by: John Halley Gotway <johnhg@ucar.edu> * #2225 @Level string is not applied as an index and index string without @ is not applied as value * Feature #2220 v11.0.0-beta2_try2 (#2228) * Updating Hera and Jet files for 11.0.0 release * Adding 11.0.0 modulefiles for Hera and Jet * Updating cheyenne installation files * Reformatted slashes in paths for hera config files * Updating installation files for Orion * Removed beta2 references * Adding configuration and modulefiles for gaea installation * Changing METdatadb references to METdataio * Documentation-only commit directly to develop to correct a typo. The default dotsize if 1.0 rather than 10. * Feature 2055 nbm grib2 update (#2234) * Per issue #2055, in read_grib2_record_list() added new PDS numbers 6, 10, 15 (for NBM data). SL * Per issue #2055: in read_grib2_record_list(), in section that validates PDS number changed, for undefined PDS numbers, changed Error to Warning and continue (instead of exit). SL * Per issue #2055, created new entries for NBM grib2 data fields. SL * Per issue #2055. Added more NBM fields based on a 12z, f003 file. SL * Per issue #2055, after interogating several NBM lead-time files, added TMAX field. SL * Per issue #2055, added grib2_nbm.txt to the list. SL * Per #2055, update the grib2_nbm.txt file entries in the table_files Makefiles. * Per issue #2055, fixed syntax entry for TSTM field. SL * Per issue #2055, fixed a few more syntax errors for some of the fields. SL * Per issue #2055, in read_grib2_record_list(), added section to set level values for template number 6 (NBM data). Also added some temporary print statements. SL * Per issue #2055, in read_grib2_record_list(), added code to set levels for table number 10 (like 6). SL * Per #2055, adding support for new GRIB2_perc_val configuration option to filter records based on the forecast percentile value used by GRIB2 product definition templates 6 and 10. Seth, note that I also included the probability filtering logic tweak we discussed, since it was already present in that same file. * Per issue #2055: in find_record_matches(), in section that resolves Prob fields, changed 'break' to 'continue' for non Probs. SL * Per issue #2055, in read_grib2_record_list() cleaned up some commented out code. SL * Per issue #2055, in read_grib2_record_list() cleaned up all print statements. SL * Per issue #2055, added 5 new unit tests for NBM data. SL * Update data2d_grib2.cc * Update data2d_grib2.cc Co-authored-by: Seth Linden <linden@seneca.rap.ucar.edu> Co-authored-by: John Halley Gotway <johnhg@ucar.edu> * Feature 2207 map (#2241) * Feature #2237 tmp_dir (#2242) * Per #2196, updating files for changes to met base image (#2246) * Hotfix to the develop branch to fix typos in the names of 3 tools (grid_stat, gribtab_dat_to_flat, and gsidens2orank). Also removed commented out code from main.cc. * Feature 2250 accum (#2251) * Feature 2198 zenodo (#2252) * Per #2198, adding Seth Linden and Dave Albo to the Author List * Per #2198, updated Dave Albo to David Albo * Per #2198, added three new authors based on submitted AMS abstract * Per #2198, adding .zenodo.json file * Per #2198, adding orcids for all based on Tatiana's suggestion * Update .zenodo.json Co-authored-by: johnhg <johnhg@ucar.edu> * Update .zenodo.json Co-authored-by: johnhg <johnhg@ucar.edu> * Update .zenodo.json Co-authored-by: johnhg <johnhg@ucar.edu> Co-authored-by: johnhg <johnhg@ucar.edu> * Per #1744, adding names for rst files in index.rst * Reword to Grid-Stat opening paragraph * Feature 2196 update dockerfile (#2249) * Per #2196, update for upgrade to Python 3.8.6 * Updated based on new structure of METbaseimage * Changed MET_BASE_IMAGE references to MET_BASE_TAG; Added information to the README.md file * Changed MET_BASE_IMAGE references to MET_BASE_TAG; Updated the version from v1.0 to v1.1 * Update internal/scripts/docker/README.md Based on suggestion from John HG. Co-authored-by: johnhg <johnhg@ucar.edu> * Update internal/scripts/docker/README.md Based on suggestion from John HG. Co-authored-by: johnhg <johnhg@ucar.edu> * Update internal/scripts/docker/README.md Based on suggestion from John HG. Co-authored-by: johnhg <johnhg@ucar.edu> * Update internal/scripts/docker/README.md Based on suggestion from John HG. Co-authored-by: johnhg <johnhg@ucar.edu> * Per #2196, updating README.md based on suggestions from John HG Co-authored-by: johnhg <johnhg@ucar.edu> * Feature 1774 contrib outline (#2253) * creating chapter files * putting a space in to work through the git add and commit process. * Changed formatting of one header Co-authored-by: Lisa Goodrich <lisag@seneca.rap.ucar.edu> Co-authored-by: jprestop <jpresto@ucar.edu> * Fine tune to grid-stat opening paragraph * Feature 2206 fair crps to ecnt (#2247) * Per issue #2206, for the ECNT line type, added CRPS_EMP_FAIR. SL * Per issue #2206, for ECNT line type, added CRPS_EMP_FAIR. SL * Per issue #2206, for ecnt_columns, added CRPS_EMP_FAIR. SL * Per issue #2206, in write_ecnt_cols(), added crps_emp_fair. SL * Per issue #2206, in ECNTInfo, added crps_emp_fair. SL * Per issue #2206, started stubbing code pieced to calculate crps_emp_fair. SL * Per issue #2206, added new function mean_abs_diff(). SL * Per issue #2206, in compute_pair_vals() updated the code to calculate crps_emp and crps_emp_fair. SL * Per issue #2206, added crps_emp_fair (CRPS_EMP_FAIR) to relevant code pieces. SL * Per issue #2206, added wording for CRPS_EMP_FAIR, also added it to last column of ECNT line type. SL * Per issue #2206, added info for CRPS_EMP_FAIR and also added the math equation for this. SL * Per issue #2206, update the math equation for CRPS_EMP_FAIR. SL * Per issue #2206, added new function weighted_mean_absolute_diff(). SL * Per issue #2206, updated the crps_emp_fair calculation to subtract the weighted_mean_abs_diff. SL * Per issue #2206, in aggr_orank_lines(), updated calculation for crps_emp_fair to use weighted_mean_abs_diff. SL * Per issue #2206, renamed weighted_mean_abs_diff() to wmean_abs_diff(). SL * Per issue #2206, for the crps_emp_fair calculation, changed weighted_mean_abs_diff() to wmean_abs_diff(). SL * Per issue #2206, in aggr_orank_lines(), for the crps_emp_fair calculation, changed weighted_mean_abs_diff() to wmean_abs_diff(). SL * Per issue #2206, updated the math equation for the CRPS_EMP_FAIR calculation. SL * Per issue #2206, fixed a bug in the wmean_abs_diff() function. SL * Per issue #2206, added code for crps_emp_fair. SL * Per issue #2206, in set() function, added checks for crps_emp and crps_emp_fair, for values of 0.0. SL Co-authored-by: Seth Linden <linden@seneca.rap.ucar.edu> * automated tests: added ability to trigger build of develop-lite docker image that it used in the METplus automated tests * to fix issues with python embedding when the path changes, update development.docker file (apparently unused or rarely previously) to include changes for Python 3.8 including setting the LDFLAG to include -rpath to prevent needing LD_LIBRARY_PATH set to run the tools. Source the env file in the build MET docker script instead of setting the env vars in the configure command * Hotfix to the develop branch to revert the python version from 3.8 back to 3.6. This is done by rolling back the base image in the MET Dockerfiles to dtcenter/met-base:v1.0. Using 3.8 caused the downstream METplus python embedding examples to fail via GHA in the METplus repo. Will write up an issue to investigate more during the MET-11.0.0-beta4 development cycle. * Tweak the build_docker_image.sh script by removing references to MET_BASE_IMAGE and MET_BASE_TAG since those variables are hard-coded in the Dockerfiles themselves and this script includes no logic to handle them. * Revert "Tweak the build_docker_image.sh script by removing references to MET_BASE_IMAGE and MET_BASE_TAG since those variables are hard-coded in the Dockerfiles themselves and this script includes no logic to handle them." This reverts commit 5efc65e. * Hotfix, found 2 more spots where I needed to change v1.1 to v1.0. We should really revisit having this setting spread across so many locations. * Bugfix 2238 link error (#2255) Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu> * Feature dtcenter/METplus-Internal#15 add checksum to release (#2254) Co-authored-by: jprestop <jpresto@ucar.edu> Co-authored-by: John Halley Gotway <johnhg@ucar.edu> * feature 2216 ioda2nc station_id (#2259) Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu> Co-authored-by: John Halley Gotway <johnhg@ucar.edu> * Feature 1942 seeps (#2257) Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu> Co-authored-by: John Halley Gotway <johnhg@ucar.edu> Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> * Feature 2142 ascii2nc airnow (#2256) Co-authored-by: Dave Albo <dave@seneca.rap.ucar.edu> Co-authored-by: John Halley Gotway <johnhg@ucar.edu> * Feature 1954 semilatlon (#2262) Co-authored-by: j-opatz <59586397+j-opatz@users.noreply.github.com> * Removed future imports since they were used to transition from python 2 to 3. Rearranged logic to be more readible by checking error conditions and exiting if they are not met instead of putting all of the logic to run inside the if statement * Feature 1908 remove ens (#2265) * Feature 2266 v11.0.0-beta3 (#2268) * Minor hotfix for the develop branch. Running bootstrap for develop on seneca changes the order of vx_grid semilatlon slightly. We probably manually editted that file instead of running bootstrap to regenerate it. * Bugfix #2271 develop nbrctc (#2272) Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> * Feature 2282 docs (#2283) * #2270 Removed version number for SonrQube scanner pathdiff * Feature #2286 grib_lookup (#2288) * Feature 2227 namespace (#2290) * Per #2227 changes in this commit resolve all errors from removing 'using namespace std' from all header files in the src/basic directory * Attempt to resolve errors in GitHub actions from removing 'using namepsace std' from all src/basic/ libraries, plus removal from src/libcode/vx_gnomon/gnomon.h and src/libcode/vx_nc_util/nc_utils.h * Attempt to fix GHA errors with compilation after removal of 'using namespace std'; locally compiles fine * Per #2227, attempting to fix GHA compilation; Compiltion on seneca works fine * Per #2227, attempt to resolve GHA compilation issues; seneca compilation works fine * Per #2227, attempt to resolve GHA compilation issues; seneca compilation works fine * Per #2227, removing namespace from vx_series_data library * Per #2227, removing namespace std from vx_shapedata library * Per #2227, removing namespace std from vx_data2d_grib2 and vx_data2d_grib libraries * Per #2227, removing namespace std from vx_statistics library * Per #2227, removing namespace std from src/tools/tc_utils * Per #2227, removed namespace std from /src/tools/core header files * Per #2227, clean up and continued work * Per #2227, changes to remove namespace std and netCDF from header files * Per #2227, resolve errors in GitHub Actions build * Per #2227, resolve new errors in GitHub Actions build * Per #2227, resolve new errors in GitHub Actions build * Per #2227, resolve new errors in GitHub Actions build * Per #2227, realign variables * Per #2227, updating comment * Per #2227, updating comment * Per #2227, realign variables * Per #2227, realign variables * Per #2227, realign variables * Per #2227, updating comment * Update src/basic/vx_config/config.tab.cc Co-authored-by: johnhg <johnhg@ucar.edu> Co-authored-by: Julie Prestopnik <jpresto@seneca.rap.ucar.edu> Co-authored-by: johnhg <johnhg@ucar.edu> * Feature #2237 test_empty_config (#2291) * Feature #2297 gha (#2299) * GHA hotfix to develop Changes for #2297 included a small typo that I'm fixing directly in the develop branch. * Feature #2297 hotfix to develop A final hotfix to the develop branch to get the build_docker_trigger_metplus.yml workflow working again. I tested on feature_2297_gha branch to confirm that no double-quotes are actually needed. * Update GHA versions * Feature 2281 linker (#2296) Co-authored-by: Julie Prestopnik <jpresto@seneca.rap.ucar.edu> * Feature 2036 enhance tc pairs (#2301) Co-authored-by: Seth Linden <linden@seneca.rap.ucar.edu> Co-authored-by: John Halley Gotway <johnhg@ucar.edu> * Feature 2276 enhance ascii2nc to read ndbc buoy data (#2294) Co-authored-by: Dave Albo <dave@seneca.rap.ucar.edu> Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> * Feature 2155 time units months and years (#2300) Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu> * removing release-notes out of overview and creating a seperate chapter for this information. (#2304) * Bugfix #2309 develop tcmpr (#2310) * Bugfix #2306 ascii2nc airnow hourly (#2314) Co-authored-by: Dave Albo <dave@seneca.rap.ucar.edu> * Feature #2058 bias_ratio (#2317) * Feature 2232 refine grid diag (#2316) Co-authored-by: Seth Linden <linden@seneca.rap.ucar.edu> Co-authored-by: John Halley Gotway <johnhg@ucar.edu> Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> * Feature #392 tcdiag_line_type (#2315) * Feature #2068 ioda2nc v2.0 (#2307) Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu> Co-authored-by: John Halley Gotway <johnhg@ucar.edu> Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> * Feature #2322 v11.0.0-beta4 (#2323) * Feature #2325 ECNT_MAE (#2326) * Hotfix to the develop branch to correct the column numbers in the CNT line type table for the RMSOA, ANOM_CORR_UNCNTR, and SI statistics. * Feature #2332 spread_md (#2333) * Per #2232, add SPREAD_MD to the ECNT line type from Ensemble-Stat and computed for HiRA. * Per #2332, fix compilation error. * Per #2332, update date and release notes for met-11.0.0-beta4. * Per #2332, fix compilation error. Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu> Co-authored-by: hsoh-u <hsoh@ucar.edu> Co-authored-by: Randy Bullock <bullock@seneca.rap.ucar.edu> Co-authored-by: davidfillmore <fillmore.winslow.david@gmail.com> Co-authored-by: rgbullock <bullock@ucar.edu> Co-authored-by: John Halley Gotway <johnhg@kiowa.rap.ucar.edu> Co-authored-by: Dan Adriaansen <dadriaan@ucar.edu> Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu> Co-authored-by: jprestop <jpresto@ucar.edu> Co-authored-by: Seth Linden <linden@ucar.edu> Co-authored-by: Seth Linden <linden@seneca.rap.ucar.edu> Co-authored-by: Julie Prestopnik <jpresto@seneca.rap.ucar.edu> Co-authored-by: Seth Linden <linden@kiowa.rap.ucar.edu> Co-authored-by: j-opatz <59586397+j-opatz@users.noreply.github.com> Co-authored-by: Howard Soh <hsoh@kiowa.rap.ucar.edu> Co-authored-by: John Halley Gotway <johnhg@seneca.rap.ucar.edu> Co-authored-by: mo-mglover <78152252+mo-mglover@users.noreply.github.com> Co-authored-by: davidalbo <dave@ucar.edu> Co-authored-by: lisagoodrich <33230218+lisagoodrich@users.noreply.github.com> Co-authored-by: Molly Smith <molly.b.smith@noaa.gov> Co-authored-by: Dave Albo <dave@seneca.rap.ucar.edu> Co-authored-by: Jonathan Vigh <jvigh@ucar.edu> Co-authored-by: bikegeek <3753118+bikegeek@users.noreply.github.com> Co-authored-by: Lisa Goodrich <lisag@seneca.rap.ucar.edu>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Pull Request Testing
During development tested with a command line derived from unit test:
./wavelet_stat /d1/projects/MET/MET_test_data/unit_test/model_data/grib1/nam_st4/nam_2012040900_F012_gSt4.grib /d1/personal/linden/git/MET/test_output/pcp_combine/stage4_2012040912_F012_APCP12.nc ./WaveletStatConfig -outdir /d1/personal/linden/git/MET/test_output/wavelet_stat -v 1
After development, update the unit test and ran the unit test
git/MET/test/xml/unit_wavelet_stat.xml
Recommend testing for the reviewer(s) to perform, including the location of input datasets, and any additional instructions:
Run the unit test:
git/MET/test/xml/unit_wavelet_stat.xml
Do these changes include sufficient documentation updates, ensuring that no errors or warnings exist in the build of the documentation? [Yes or No]
Yes, the documentation was also updated:
git/MET/met/docs/Users_Guide/wavelet-stat.rst
Do these changes include sufficient testing updates? [Yes or No]
Yes
Will this PR result in changes to the test suite? [Yes or No]
If yes, describe the new output and/or changes to the existing output:
Yes/No. Added new test to unit test: unit_wavelet_stat.xml
Please complete this pull request review by [Fill in date].
Pull Request Checklist
See the METplus Workflow for details.
Select: Reviewer(s)
Select: Organization level software support Project or Repository level development cycle Project
Select: Milestone as the version that will include these changes