From f4f4fc2ae265ab838be25b5310a0dc8094c87d8b Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Mon, 18 Sep 2023 18:54:37 -0600 Subject: [PATCH] Update develop-ref after retry #2347 and dtcenter/MET#2683 (#2353) * Feature dtcenter/METplus-Internal#24 test failure (#1895) * updated version for creating release (#1897) * update release number for development towards beta5 * Adding one sentence to about daily follow up on discussions. * initial changes for new use case * updated paths, documentation * updated automated testing files * Feature #344 rearrange utils (#1900) * first additions for new use case, corrected documentation for old use case * Move Docker scripts to be consistent with other METplus components (#1907) * added picture * Correct spelling mistake * handle if input to format_thresh is already a list * Revert "handle if input to format_thresh is already a list" This reverts commit d329275d3ffd279e366789268ff5e8d666d7a84a. * handle if input to format_thresh is already a list * per #1910, added time util function to add field info to time info dictionary and updated wrapper logic to use this function to add this info instead of passing field info (var_info) to CommandBuilder.find_data to get level info, ci-run-all-diff * change logic to preserve full level string for [fcst/obs]_level so SeriesAnalysis cases build the same file names, but parse out time info for setting generic level for string sub to preserve previous behavior * try setting all input values in output dictionary, ci-run-all-diff * removed commented code * add quotes around input files to plot_point_obs command to allow support for Python Embedding as input * put all arguments on single line * remove commented code * per #1910, added unit tests to ensure that level is set properly in the forecast input template * added script to create geovista conda environment via Docker to use in the automated use case tests * Feature 1561 use case ugrid python embedding (#1927) Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> * Feature 1453 use case iodav2 python embedding (#1930) Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> * fix permissions for script that were somehow changed to be not executable * Bugfix #1939 develop - failure reading obs when zipped file also exists (#1941) * Feature #1819 alert users if updates to EnsembleStat use cases are needed (#1945) Co-authored-by: Dan Adriaansen * fixed underline that did not match title * Feature #1898 tc new options (#1932) close undefined * fixed broken doc util logic from bad PR merge * Feature #1569 EnsembleStat -ens_mean argument (#1952) * Feature #1953 seeps config in GridStat and PointStat (#1954) Co-authored-by: johnhg * Feature #1566 update Python version to 3.8.6 (#1958) * Feature #1956 5.0.0-beta4 release (#1963) * Update VERSION * added warning in release notes that location of METplus Dockerfile has changed * combined release notes for all betas for the v5.0.0 release * updated tags for each METplus component to match the upcoming release * Feature #1885 gh_projects (#1973) Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> * Switch from double to single quotes. * Update formatting of the GitHub project filters so that they can be copy/pasted directly from RTD into the GitHub project filter. * Feature #1953 PointStat output_flag.seeps and output_flag.seeps_mpr (#1976) * Feature #1711, #1783, #1814, and #1835 Documentation - Multiple files, PCPCombine constant init, Docker instructions, and MET links (#1967) Co-authored-by: Dan Adriaansen * run diff logic on push for develop and main_v* branches, fix logic to properly run all use cases * fixed name of external * added Bugfix: to default title of new bugfix issue * added space after New Use Case: * changed references to METdatadb to METdataio that were missed * Update overview.rst Removed statsmodels from the table, this package is no longer a requirement. The necessary functionality was pulled out of the statsmodels repository and incorporated into the METcalcpy repository * Update overview.rst METdataio *must* only use pandas 1.2.3. METdataio also requires pyyaml. * Update overview.rst METdatio uses pytest to test aspects of the METreformat module * Update rotate_authorship.rst Explicitly add the step to commit and push changes to the file containing the author list. * Update overview.rst Added METdataio for lxml, numpy, pandas, and pymysql * Update finalize_release_on_github_official.rst added branch protection rules step and the interface to Github was modified, you need to click on the stacked '->' icon to switch default branches * CI: replace / with _ to prevent errors creating docker tags for tests * Update overview.rst included METdataio for python 3.8.6+ requirement * Update overview.rst psutil no longer needed in METplotpy and METcalcpy * Update overview.rst METcalcpy no longer requires imutils and allow versions 0.5.3 and above * Update overview.rst cleaned up formatting for imutils * change asterisk to double asterisk to match branches that include forward slash / * update version of certifi based on dependabot auto-generated PR #1982 * Feature 1819 format doc (#1981) Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> * Update use case keywords (#1984) * Feature #1980 Use the main_vX.Y of METplus components for main METplus tests (#1985) * Feature #1986 v5.0.0 release (#1987) Closes https://github.com/dtcenter/METplus/issues/1986 * updated version for development towards beta1 of 5.1.0 * allow rest of workflow to run even if job to update data volumes fails in case server is down * updated version for action to update input data volumes and added max number of DockerHub tag pages to search to 15 * updated release notes for official METplus release to untar the sample data for next version directory * Update release guide to create the input test data directory for official MET releases. * Follow the METplus naming conventions for updating the the DTC web server data. * install library needed for geovista package, remove editable flag so geovista code will be added directly to python instead of reading it from the source code location, ci-skip-all * Feature 685 log updates (#1992) * add script to create conda environment with MetPy 1.4 for SWPC use case * Bugfix develop Fix broken documentation links (#2004) * fixed bug to set environment variables in the RegridDataPlane environment instead of the PyEmbedIngest environment * Feature #2015 modulefiles (#2024) * Adding modulefile for WCOSS2 * Updated references of 4.1.1 to 5.0.0 * Feature #2005 updates to create v5.1 conda environments that use debian10 and Python 3.10.4 (#2043) * update versions to fix bugs for METplotpy/calcpy use cases * Bugfix #2026 develop StatAnalysis looping (#2028) * Feature #2022 update python version to 3.10.4 for repo files (#2047) * Feature #2022 Use Debian 10 / Python 3.10.4 in automated tests (#2050) * add use case group name to output path so that output and error logs will contain use case category to more easily see which case was run * remove -group0 from group names for output path * add manual workflow to update truth data * ci: set gh token on checkout to enable push, ci-skip-all * Set user name/email via git config to allow push * add token to checkout * Feature #2049 Multiple interp.type.method/width (#2051) * added missing argument to script if building envs locally * get main_vX.Y truth data if running on main_vX.Y branch * Updating README.md for label creation and adding bold to common_labels.txt * Updated color for bold label * Feature #2054 consensus.write_members in TCPairs (#2057) * Create LICENSE.md (#2061) * per #2006, fix priority of obs_window config variables so that wrapper-specific version is preferred over generic OBS_WINDOW_BEGIN/END (#2062) * Feature 2065 v5.1.0 beta1 (#2066) * updated version number * added release notes * Update docs/Users_Guide/release-notes.rst Co-authored-by: John Halley Gotway --------- Co-authored-by: John Halley Gotway * updated version with -dev * adding sphinx design to 3 files so dropdown menus will work in the release notes. * fix version numbers for python requirements for building documentation * fixed tab vs spaces * Bugfix #2070 var list numeric order (#2072) * Adding files for jet and hera * added commented process list that was accidentally removed -- this is used if the pre-processing steps are needed * Bugfix #2087 develop docs_pdf (#2091) * Feature 2076 release dropdown menu (#2083) * test dropdowns * Update release-notes.rst Attempting to resolve WARNING * testing dropdowns * adding dropdown menus * testing dropdown remove from warning section * dropdown testing * dropdown menu tests * dropdown menus after the warnings section, the dropdown menu only has one space instead of the normal 2 spaces. If it has 2 spaces, it gets sucked into the warning section. * spaces with warning * no spaces for warning * Update release-notes.rst warning note has to be indented 1 space. Originally it was no spaces but then all of the dropdown menus end up in the warning message. If it's 2 spaces, as is normal, then the warning is indented, which is unnecessary. 1 space is slightly indented and keeps the dropdown menus separate. --------- Co-authored-by: jprestop * Bugfix #2096/#2098 develop - fix skip if output exists and do not error if no commands were run (#2099) * Feature 1516 use case mvmode (#2094) Co-authored-by: Tracy Hertneky Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> * Fix for Dockerfile smell DL4000 (#2112) Co-authored-by: Giovanni Rosa * Changed "PROJECT" to "CYCLE" ASSIGNMENT * Feature 2115 update use cases (#2133) * Bugfix #2082 develop regrid.convert/censor_thresh/censor_val (#2140) fix #2082 main_v5.0 regrid.convert/censor_thresh/censor_val (#2101) * Bugfix #2137 develop PointStat -obs_valid_beg/end (#2141) * Feature #2143 5.1.0-beta2 release (#2144) * update release guide steps with new info based on changes to GitHub web interface * update version for development towards rc1 release * Feature #1977 ARGO use case (#2148) Co-authored-by: j-opatz <59586397+j-opatz@users.noreply.github.com> * Feature develop cycle change (#2150) * change PROJECT ASSIGNMENT to CYCLE ASSIGNMENT * change PROJECT ASSIGNMENT to CYCLE ASSIGNMENT * change PROJECT ASSIGNMENT to CYCLE ASSIGNMENT * change PROJECT ASSIGNMENT to CYCLE ASSIGNMENT * change PROJECT ASSIGNMENT to CYCLE ASSIGNMENT * change PROJECT ASSIGNMENT to CYCLE ASSIGNMENT * Added paths to ignore --------- Co-authored-by: jprestop * Documentation: Update Truth Data instructions (#2154) * upgrade python to 3.10 for documentation builds to fix failured introduced by urllib3 (see https://github.com/urllib3/urllib3/issues/2168) * update workflow to update input test data to use branch from pull down or enter branch by hand if branch does not exist in dtcenter/METplus repo, e.g. if updating data for a PR coming from a fork * Bugfix #2161 develop PCPCombine additional field arguments in -subtract mode (#2162) * changed order of workflow event triggering info to more easily see the repository and commit sha that triggered the workflow * fix docker compose install and call -- needed due to new Alpine Linux release 3.18.0 that does not include docker-compose in the apk repository * Bugfix #2168 develop - StatAnalysis time shift (#2169) * Feature 1978 update pyreq doc (#2177) * Updating for 5.1.0 * Adding greater than or equal to signs * Feature 2106 release support (#2182) * Added a paragraph for Release Support Policy * Added a sentence about contacting the team through the METplus GitHub Discussions Forum. * Added Release Support Policy info * Per #2159, modify instructions to reflect that we only want a coordinated release for the official releases and updates to the coordinated release section for the bugfix releases. (#2183) * Feature 2147 workflow doc (#2185) * Add new images for github workflow * Added PR section * Modified the Find the GitHub Issue to be Find or Create a GitHub issue * Trying out image keyword compared with figure keyword * Trying out various widths * Added new figures * Reverting to figures from images after modifying figures * Adding the old images which are better quality * Trying out new indention * Working on formatting again * Reducing image widths * Update docs/Contributors_Guide/github_workflow.rst Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> * Update docs/Contributors_Guide/github_workflow.rst Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> * Update docs/Contributors_Guide/github_workflow.rst Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> * Update github_workflow.rst --------- Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> * added scripts to create new conda environment for PANDA-C use cases * Bugfix #2189 develop - spaces in complex thresholds (#2191) * added release notes for completed issues for rc1 * Feature 2011 tci from cesm fluxnet2015 netcdf (#2192) Co-authored-by: Mrinal Biswas Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> * Bugfix #2179 develop TCPairs fix -diag argument (#2187) * sort list of files to make command more readable and make logic more easily testable * fix unit test to expect alphabetical order of files * Feature #2197 MODE new multi-variate options (#2203) * added to release notes * added release notes for other items completed for rc1 release * Feature #1626 TCDiag Wrapper (#2201) * Feature #1610 first round of SonarQube fixes (#2200) * Feature #2195 create v5.1.0-rc1 release (#2204) * Update version for development towards official release * updated script to use metplotpy as base and install pygrib because conda was failing trying to install matplotlib in Docker for some strange reason * Feature 2188 use case panda c (#2202) * added first use case, need to finish docs and add to list * added documentation files, 3 new cases * added 5th use case, docs * final use case, corrected paths for output * Update GridStat_fcstGFS_obsERA5_lowAndTotalCloudFrac.py * removed chars, added space to last section * turn off 5/6 new air quality and comp use cases because they are causing the disk to fill up in the automated tests and only 1 is needed to test the new behavior. Reordered use case groups to be alphabetical * disabled other new air quality use case because the disk is still filling up from this case * Feature 2136 cross spectra (#2208) * per #2212, continue instead of return so other forecast leads will not be skipped * Feature #2210 MODE multi-variate variable rename (#2211) * Bugfix diff tests (#2217) * added change from NCEP produtil that was somehow not included in the changes pulled from their repo before starting SonarQube work * Feature #2228 v5.1.0-rc2 release (#2229) * update version after rc2 release to note development towards official 5.1.0 release * Feature #2188 move PANDA-C cases to clouds category and fix automated tests (#2237) Co-authored-by: j-opatz * Feature 2198 use case amdar pbl (#2206) Co-authored-by: Daniel Adriaansen Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> * Bugfix #2235 rename multivar_itensity to multivar_intensity_flag (#2236) * Feature #2230 MvMODE usecase doc updates (#2239) This updates the use-case documentation to go along with the multivariate MODE enhancements. * combine release notes from development releases for 5.1.0 release * fix indentation to prevent incorrect nesting of drop down menus * Bugfix #2241 Create directory containing -out_stat file (#2242) * Enhance diff logic to control decimal precision to prevent failures (#2243) * Bugfix #2245 use unique run ID to name logger instance (#2247) * per #2245, add METplusConfig class function that is called when object is deleted to close log handlers. This prevents OSError: [Errno 24] Too many open files from running all pytests * Feature #1626 TCDiag updates to wrapper and basic use case (#2248) Co-authored-by: Jonathan Vigh * move new use case into another group * Bugfix #2244 develop fix diff tests (#2254) * skip diff for specific png images that produce diffs occasionally but look the same visually * add 2 more keywords for use cases that occasionally flag small, unnoticeable diffs in png images * Feature #2253 update tests for diff_util (#2256) Co-authored-by: root * Feature #1974 Documentation: update coding standards section (#2260) * Feature #2253 conftest fixture to set pytest tmpdir (#2261) * Feature #2151 Documentation: Add quick search keywords METplotpy/calcpy use cases (#2259) * Feature #2138 CI enhance MET Docker tag override (#2258) * Updating development instructions. * fixed typo * fixed another typo * Feature #2207 Documentation - Updates to Contributor's Guide (#2263) Co-authored-by: Dan Adriaansen * Feature develop update install files (#2266) * Updating installation files * Removing run_commands.sh * Updated the file to add a MINICONDA_PATH * Adding files for jet * Adding orion file * Removing beta references * Corrected met version * Adding files for WCOSS2 machines * Removing rc1 from acorn file * Removing beta1 * Bugfix #1853 develop - PointStat don't require mask variables to be set (#2262) * update version of certifi to match auto-generated dependabot PR #2267 * feature #2253 more tests for diff_util.py (#2264) * update versions of METplus components for the next official release * updated release notes for 5.1.0 release * removed duplicate entries in release notes * Added EC2 instance recipee with S3 mounting ability (#2269) * Updating the Subsetting Tests by Marker section (#2272) * update release date for 5.1.0 release (#2276) * update version for development towards 6.0.0-beta1 release * added new use case that was missing * Bugfix #2279 develop - buoy station file from 2022 (#2280) * Feature 2253 series analysis tests (#2277) * Fix typo in pull_request_template.md * Make code coverage report available (#2287) * Use updated dtcenter/metplus-action-data-update tag that fixes bug that causes GHA disk to exceed capacity when too many data volumes are created. Use specific commit of coveralls python action to ensure it will not change * removed climatology keyword from use cases that aren't relevant that were accidentally added with PR #1984 * update readme with current information * Feature #2282 coord_release_docs (#2288) Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> * continue workflow if coveralls step fails -- this typically happens on a pull request from a fork because the authentication fails * Feature 2253 system util tests (#2297) * Feature #2294 LSR use case (#2295) * Feature 2253 run util tests (#2307) * Release Guide - remove beta/rc tags/releases (#2309) * Add 'component: repository maintenance' label. Already ran the script to push this change and the NOAA OAR reporting label to all the METplus repos. * Hotfix for labels, adding a new one for 'component: input data' and also fixing the get_lablels.sh to search for up to 200 existing labels. Also work on the log messages. * Fix typo in comment * Feature #2283 time looping consolidation (#2311) Co-authored-by: John Halley Gotway * New issue template: Update Truth (#2332) Co-authored-by: John Halley Gotway * feature #2253 tests for run_util, 'Usage' bugfix (#2313) * Feature #2338 Debian 12 Conda environments (#2341) Co-authored-by: John Halley Gotway * Feature 1488 usecase satwinds (#2324) Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> * Feature #2283 include times (#2345) Co-authored-by: John Halley Gotway * feature #2253 tests for config_validate, mock logger in conftest (#2320) * Feature #2299 / #2310 Remove deprecated MET config env vars and TCMPRPlotter (#2347) Co-authored-by: John Halley Gotway * remove MODEL and OBTYPE to use defaults from MET config * Feature #2348 v6.0.0-beta1 release (#2351) * update version for development towards 6.0.0-beta2 release * copy libGL and libEGL files into docker image to prevent errors with geovista python package * run use that uses geovista to test --------- Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> Co-authored-by: Julie Prestopnik Co-authored-by: j-opatz Co-authored-by: j-opatz <59586397+j-opatz@users.noreply.github.com> Co-authored-by: Hank Fisher Co-authored-by: Will Mayfield <59745143+willmayfield@users.noreply.github.com> Co-authored-by: Dan Adriaansen Co-authored-by: johnhg Co-authored-by: bikegeek <3753118+bikegeek@users.noreply.github.com> Co-authored-by: lisagoodrich <33230218+lisagoodrich@users.noreply.github.com> Co-authored-by: metplus-bot <97135045+metplus-bot@users.noreply.github.com> Co-authored-by: Lisa Goodrich Co-authored-by: Tracy Hertneky <39317287+hertneky@users.noreply.github.com> Co-authored-by: Tracy Hertneky Co-authored-by: Giovanni Rosa Co-authored-by: Giovanni Rosa Co-authored-by: mrinalbiswas Co-authored-by: Mrinal Biswas Co-authored-by: Christina Kalb Co-authored-by: jason-english <73247785+jason-english@users.noreply.github.com> Co-authored-by: Jonathan Vigh Co-authored-by: John Sharples <41682323+John-Sharples@users.noreply.github.com> Co-authored-by: root --- .../actions/run_tests/Dockerfile.run_geovista | 15 + .github/jobs/setup_and_run_use_cases.py | 2 + .github/parm/use_case_groups.json | 2 +- docs/Users_Guide/getting_started.rst | 9 +- docs/Users_Guide/release-notes.rst | 786 +----------------- metplus/VERSION | 2 +- 6 files changed, 47 insertions(+), 769 deletions(-) create mode 100644 .github/actions/run_tests/Dockerfile.run_geovista diff --git a/.github/actions/run_tests/Dockerfile.run_geovista b/.github/actions/run_tests/Dockerfile.run_geovista new file mode 100644 index 0000000000..c8d8d82310 --- /dev/null +++ b/.github/actions/run_tests/Dockerfile.run_geovista @@ -0,0 +1,15 @@ +ARG METPLUS_ENV_TAG=metplus_base.v5 +ARG METPLUS_IMG_TAG=develop + +FROM dtcenter/metplus-envs:${METPLUS_ENV_TAG} as env + +ARG METPLUS_IMG_TAG=develop +FROM dtcenter/metplus-dev:${METPLUS_IMG_TAG} + +RUN mkdir -p /usr/local/conda/envs && mkdir -p /usr/local/conda/bin +COPY --from=env /usr/local/conda/envs /usr/local/conda/envs/ +COPY --from=env /usr/local/conda/bin/conda /usr/local/conda/bin/conda + +# copy libGL and libEGL libraries to prevent dynamic lib errors +COPY --from=env /lib/x86_64-linux-gnu/libEGL* /lib/x86_64-linux-gnu/ +COPY --from=env /lib/x86_64-linux-gnu/libGL* /lib/x86_64-linux-gnu/ diff --git a/.github/jobs/setup_and_run_use_cases.py b/.github/jobs/setup_and_run_use_cases.py index 8ed341ec4a..931a79d76d 100755 --- a/.github/jobs/setup_and_run_use_cases.py +++ b/.github/jobs/setup_and_run_use_cases.py @@ -181,6 +181,8 @@ def _get_dockerfile_name(requirements): return f'{dockerfile_name}_gfdl' if 'cartopy' in str(requirements).lower(): return f'{dockerfile_name}_cartopy' + if 'geovista' in str(requirements).lower(): + return f'{dockerfile_name}_geovista' return dockerfile_name diff --git a/.github/parm/use_case_groups.json b/.github/parm/use_case_groups.json index 1752f76df0..041b000b2c 100644 --- a/.github/parm/use_case_groups.json +++ b/.github/parm/use_case_groups.json @@ -277,6 +277,6 @@ { "category": "unstructured_grids", "index_list": "0", - "run": false + "run": true } ] diff --git a/docs/Users_Guide/getting_started.rst b/docs/Users_Guide/getting_started.rst index 5d7287bd25..4d25c4f28d 100644 --- a/docs/Users_Guide/getting_started.rst +++ b/docs/Users_Guide/getting_started.rst @@ -387,9 +387,10 @@ METplus is available on DockerHub. The METplus Docker image includes all of the MET executables from the corresponding METplus Coordinated Release and the METplus wrappers are pre-configured to use them. -To pull the latest official release, run:: +To pull the latest official release, use the X.Y-latest tag where X.Y is the +version, e.g.:: - docker pull dtcenter/metplus:latest + docker pull dtcenter/metplus:5.1-latest Tags for previous releases and development releases are also available. Refer to the list of @@ -414,12 +415,12 @@ To make these data available in a METplus Docker container, first create a Docker data volume from the desired tag and give it a name with the *--name* argument:: - docker create --name met_tool_wrapper dtcenter/metplus-data:4.1-met_tool_wrapper + docker create --name met_tool_wrapper dtcenter/metplus-data:5.1-met_tool_wrapper Then mount the data volume to the container using the *--volumes-from* argument to the *docker run* command:: - docker run --rm -it --volumes-from met_tool_wrapper dtcenter/metplus:4.1.4 bash + docker run --rm -it --volumes-from met_tool_wrapper dtcenter/metplus:5.1-latest bash The input data will be available inside the container under /data/input/METplus_Data. diff --git a/docs/Users_Guide/release-notes.rst b/docs/Users_Guide/release-notes.rst index eedb3f9b26..a07a77b894 100644 --- a/docs/Users_Guide/release-notes.rst +++ b/docs/Users_Guide/release-notes.rst @@ -30,295 +30,45 @@ When applicable, release notes are followed by the `GitHub issue `__ number which describes the bugfix, enhancement, or new feature. -METplus Version 5.1.0 Release Notes (2023-07-31) ------------------------------------------------- +METplus Version 6.0.0 Beta 1 Release Notes (2023-09-15) +------------------------------------------------------- .. dropdown:: Enhancements - * Add support for multiple interp widths - (`#2049 `_) - * TCPairs - Add support for setting consensus.write_members - (`#2054 `_) - * Update use cases to use new Python directory structure in MET - (`#2115 `_) - * Add support for new multivariate MODE settings - (`#2197 `_, - `#2210 `_, - `#2230 `_, - `#2235 `_) + * **Remove support for deprecated environment variables for old wrapped MET config files** + (`#2299 `_) + * Improve time formatting logic to include certain times and use day of week to subset + (`#2283 `_) + * Remove TCMPRPlotter wrapper + (`#2310 `_) .. dropdown:: Bugfix - * StatAnalysis - allow run once for each valid time - (`#2026 `_) - * App specific OBS_WINDOW variables not taking precedence over generic - (`#2006 `_) - * Skip-if-output-exists logic incorrectly skips files - (`#2096 `_) - * PointStat -obs_valid_beg/end arguments not set properly - (`#2137 `_) - * Allow setting of convert, censor_thresh, and censor_val in regrid dictionary - (`#2082 `_) - * TCPairs setting -diag option causes failure - (`#2179 `_) - * Define the order of the forecast variables numerically rather than alphabetically - (`#2070 `_) - * Prevent error if no commands were run because they were skipped - (`#2098 `_) - * Allow spaces for complex categorical thresholds - (`#2189 `_) - * PCPCombine - Extra field options not set in -subtract mode - (`#2161 `_) - * StatAnalysis time shifting failure - (`#2168 `_) - * TCPairs: skip times logic incorrectly skips additional times - (`#2212 `_) - * TCStat fails if -out_stat directory does not exist - (`#2241 `_) - * Ensure log instances for concurrent runs are unique - (`#2245 `_) - * PointStat - Support running with no MASK provided - (`#1853 `_) + * Update buoy use case to use buoy station file from 2022 + (`#2279 `_) + * Prevent failure in LSR use case + (`#2294 `_) + .. dropdown:: New Wrappers - * TCDiag (beta) - (`#1626 `_) + NONE .. dropdown:: New Use Cases - * Multi-Variate MODE (`#1516 `_) - * Read in Argo profile data netCDF files for use in METplus with python embedding - (`#1977 `_) - * PANDA-C: MPAS to SATCORPS - (`#2188 `_) - * PANDA-C: MPAS to MERRA2 - (`#2188 `_) - * PANDA-C: MPAS to ERA5 - (`#2188 `_) - * PANDA-C: GFS to SATCORPS - (`#2188 `_) - * PANDA-C: GFS to MERRA2 - (`#2188 `_) - * PANDA-C: GFS to ERA5 - (`#2188 `_) - * PointStat AMDAR PBLH with python embedding - (`#2198 `_) - * Space-time cross-spectra for S2S forecasts - (`#2136 `_) + * Scatterometer wind data + (`#1488 `_) .. dropdown:: Documentation - * Enhance the Release Notes by adding dropdown menus - (`#2076 `_) - * Update the METplus Components Python Requirements - (`#1978 `_, - `#2016 `_) - * Add documentation on support for releases to the Release Guide - (`#2106 `_) - * Add use case quick search keywords for METplotpy and METcalcpy - (`#2151 `_) + NONE .. dropdown:: Internal - * Improve use case testing - (`#685 `_) - * Update conda environments to use 3.10 for automated use case tests - (`#2005 `_) - * Add modulefiles to the repository - (`#2015 `_) - * **Upgrade to using Python 3.10.4** - (`#2022 `_) - * Add 'LICENSE.md' to the METplus repo - (`#2058 `_) - * Update Contributor's Guide to use GH Action to update truth data - (`#2068 `_) - * Enhance GitHub Workflow documentation - (`#2147 `_) - * Update the development release guide instructions to remove references to a Coordinated release - (`#2159 `_) - * Refactored code to resolve many SonarQube items - (`#1610 `_) - * Improve Contributor's Guide - (`#2138 `_, - `#2207 `_) - * Bugfix: Fix difference test logic - (`#2244 `_) - * Remove base environment from Docker Conda images - (`#2249 `_) - * PR Templates (all METplus component repos): Improve language about linking relevant issue(s) - (`#2257 `_) - - -METplus Version 5.0.0 Release Notes (2022-12-09) ------------------------------------------------- - - .. warning:: **MAJOR CHANGES**: - - * The LOOP_ORDER config variable was removed. The variable set in a user's - config file will be ignored in favor of executing the logic that - corresponds to *LOOP_ORDER = processes*, where all times are processed for - the first item in the PROCESS_LIST, then all times are processed for the - second item in the PROCESS_LIST, etc. This may change the order that - commands are executed in a use case, but it should not change the results. - * The METplus Dockerfile was moved to internal/scripts/docker. - It was previously found in scripts/docker. - * Use cases that include **EnsembleStat** wrapper will require config file - updates. See :ref:`upgrade-instructions`. - * The default value of :term:`SCRUB_STAGING_DIR` is now *True*. - This means some intermediate files that are auto-generated by METplus such - as file lists and uncompressed files will automatically be removed unless - this option is set by the user. - These files are typically only used to debug unexpected issues. - * The default value of :term:`METPLUS_CONF` now includes the - :term:`LOG_TIMESTAMP` so each METplus run will generate a unique final - config file, e.g. metplus_final.conf.20220921121733. - - - .. dropdown:: Enhancements - - * **Enhance MODE wrapper to support multi-variate MODE** - (`#1585 `_) - * **Allow FCST_IS_PROB variable setting specific to tool - (FCST__IS_PROB)** - (`#1586 `_) - * **Enhance climatology field settings to be consistent with fcst/obs field** - (`#1599 `_) - * Update Hovmoeller Use case to use updated Hovmoeller plotting - (`#1650 `_) - * **Update the EnsembleStat wrapper and use case examples to remove - ensemble post processing logic** - (`#1816 `_) - * Enhance logic to consistently create directories - (`#1657 `_) - * Create checksum for released code - (`#262 `_) - * Add the user ID to the log output at beginning and end of each - METplus wrappers run - (`dtcenter/METplus-Internal#20 `_) - * Update logic to name final conf and intermediate files with a unique - identifier - (`dtcenter/METplus-Internal#32 `_) - * Change default logging time information - (`dtcenter/METplus-Internal#34 `_) - * **Remove LOOP_ORDER config variable** - (`#1687 `_) - * **Add unique identifier for each METplus run to configuration** - (`#1829 `_) - * StatAnalysis - Support setting multiple jobs - (`#1842 `_) - * StatAnalysis - Set MET verbosity - (`#1772 `_) - * StatAnalysis - Support using both init/valid variables in - string substitution - (`#1861 `_) - * StatAnalysis - Allow filename template tags in jobs - (`#1862 `_) - * StatAnalysis - Support looping over groups of list items - (`#1870 `_) - * StatAnalysis - Allow processing of time ranges other than daily - (`#1871 `_) - * StatAnalysis - Add support for using a custom loop list - (`#1893 `_) - * Remove MakePlots wrapper - (`#1843 `_) - * Add support in EnsembleStat wrapper for setting -ens_mean - command line argument - (`#1569 `_) - * Enhance METplus to have better signal handling for shutdown events - (`dtcenter/METplus-Internal#27 `_) - * TCPairs and TCStat - add support for new config options and - command line arguments - (`#1898 `_) - * Enhance the GridStat and PointStat wrappers to handle the - addition of SEEPS - (`#1953 `_) - * SeriesAnalysis - add support for setting mask dictionary - (`#1926 `_) - * Update Python requirement to 3.8.6 - (`#1566 `_) - * Enhance StatAnalysis wrapper to support now and today - (`#1669 `_) - * **Clean up use case configuration files** - (`#1402 `_) - * Add support for creating multiple input datasets - (`#1694 `_) - - .. dropdown:: Bugfixes - - * PCPCombine - custom loop list does not work for subtract method - (`#1884 `_) - * Set level properly in filename template for EnsembleStat forecast input - (`#1910 `_) - * Prevent duplicate observation files using a file window if - compressed equivalent files exist in same directory - (`#1939 `_) - * Allow NA value for _CLIMO_[MEAN/STDEV]_HOUR_INTERVAL - (`#1787 `_) - * Reconcile setting of METPLOTPY_BASE for use cases - (`#1713 `_) - * Add support for the {custom} loop string in the MODEL config variable - (`#1382 `_) - * Fix PCPCombine extra options removal of semi-colon - (`#1534 `_) - * Fix reset of arguments for some wrappers - (i.e. GenEnsProd) after each run - (`#1555 `_) - * Enhance METDbLoad Wrapper to find MODE .txt files - (`#1608 `_) - * Add missing brackets around list variable values for StatAnalysis wrapper - (`#1641 `_) - * Allow NA value for _CLIMO_[MEAN/STDEV]_DAY_INTERVAL - (`#1653 `_) - - .. dropdown:: New Wrappers - - * PlotPointObs - (`#1489 `_) - - .. dropdown:: New Use Cases - - * PANDA-C use cases - (`#1686 `_) - * MJO-ENSO diagnostics - (`#1330 `_) - * Probability of Exceedence for 85th percentile temperatures - (`#1808 `_) - * FV3 Physics Tendency plotting via METplotpy - (`#1852 `_) - * StatAnalysis Python Embedding using IODA v2.0 - (`#1453 `_) - * StatAnalysis Python Embedding to read native grid (u-grid) - (`#1561 `_) - - .. dropdown:: Documentation - - * Update documentation to include instructions - to disable UserScript wrapper - (`dtcenter/METplus-Internal#33 `_) - - .. dropdown:: Internal - - * Organize utility scripts used by multiple wrappers - (`#344 `_) - * Fix GitHub Actions warnings - update the version of actions - and replace set-output - (`#1863 `_) - * Update diff logic to handle CSV files that have rounding differences - (`#1865 `_) - * Add unit tests for expected failure - (`dtcenter/METplus-Internal#24 `_) - * Add instructions in Release Guide for "Recreate an Existing Release" - (`#1746 `_) - * Add modulefiles used for installations on various machines - (`#1749 `_) - * Document GitHub Discussions procedure for the Contributor's Guide - (`#1159 `_) - * Create a METplus "Release Guide" describing how to build - releases for the METplus components - (`#673 `_) - * Update documentation about viewing RTD URLs on branches - (`#1512 `_) + * Add coordinated release checklist to the METplus Release Guide + (`#2282 `_) + * Recreate Docker/Conda environments after METbaseimage OS upgrade + (`#2338 `_) .. _upgrade-instructions: @@ -326,496 +76,6 @@ METplus Version 5.0.0 Release Notes (2022-12-09) METplus Wrappers Upgrade Instructions ===================================== -EnsembleStat/GenEnsProd (v5.0.0) --------------------------------- - -.. note:: - - If :ref:`ensemble_stat_wrapper` is not found in the :term:`PROCESS_LIST` - for any use cases, then this section is not relevant. - If upgrading from v5.0.X to v5.1.0 or higher, - then this section is not relevant. - -The METplus v5.0.0 coordinated release includes changes that remove ensemble -product generation from EnsembleStat. GenEnsProd is now required to generate -ensemble products. There are 3 situations listed below that require slightly -different modifications: - -* :ref:`upgrade-ensemble-case1` -* :ref:`upgrade-ensemble-case2` -* :ref:`upgrade-ensemble-case3` - -.. _upgrade-ensemble-case1: - -Case 1: EnsembleStat only generating ensemble products -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -If the use case had been calling EnsembleStat **WITHOUT** the **-grid_obs** or -**-point_obs** command line options, we can assume it was only doing ensemble -post-processing. -That call to EnsembleStat should be replaced with a call to -GenEnsProd instead. - -Rename Variables -"""""""""""""""" - -.. role:: raw-html(raw) - :format: html - -.. list-table:: - :widths: 50 50 - :header-rows: 1 - - * - Old Name - - New Name - * - FCST_ENSEMBLE_STAT_INPUT_DIR - - GEN_ENS_PROD_INPUT_DIR - * - FCST_ENSEMBLE_STAT_INPUT_TEMPLATE - - GEN_ENS_PROD_INPUT_TEMPLATE - * - ENSEMBLE_STAT_OUTPUT_DIR - - GEN_ENS_PROD_OUTPUT_DIR - * - ENSEMBLE_STAT_OUTPUT_TEMPLATE - - GEN_ENS_PROD_OUTPUT_TEMPLATE :raw-html:`
` - **and add filename, see** :ref:`below` - * - ENSEMBLE_STAT_N_MEMBERS - - GEN_ENS_PROD_N_MEMBERS - * - ENSEMBLE_STAT_ENS_THRESH - - GEN_ENS_PROD_ENS_THRESH - * - ENSEMBLE_STAT_ENS_VLD_THRESH - - GEN_ENS_PROD_VLD_THRESH - * - ENSEMBLE_STAT_ENSEMBLE_FLAG_LATLON - - GEN_ENS_PROD_ENSEMBLE_FLAG_LATLON - * - ENSEMBLE_STAT_ENSEMBLE_FLAG_MEAN - - GEN_ENS_PROD_ENSEMBLE_FLAG_MEAN - * - ENSEMBLE_STAT_ENSEMBLE_FLAG_STDEV - - GEN_ENS_PROD_ENSEMBLE_FLAG_STDEV - * - ENSEMBLE_STAT_ENSEMBLE_FLAG_MINUS - - GEN_ENS_PROD_ENSEMBLE_FLAG_MINUS - * - ENSEMBLE_STAT_ENSEMBLE_FLAG_PLUS - - GEN_ENS_PROD_ENSEMBLE_FLAG_PLUS - * - ENSEMBLE_STAT_ENSEMBLE_FLAG_MIN - - GEN_ENS_PROD_ENSEMBLE_FLAG_MIN - * - ENSEMBLE_STAT_ENSEMBLE_FLAG_MAX - - GEN_ENS_PROD_ENSEMBLE_FLAG_MAX - * - ENSEMBLE_STAT_ENSEMBLE_FLAG_RANGE - - GEN_ENS_PROD_ENSEMBLE_FLAG_RANGE - * - ENSEMBLE_STAT_ENSEMBLE_FLAG_VLD_COUNT - - GEN_ENS_PROD_ENSEMBLE_FLAG_VLD_COUNT - * - ENSEMBLE_STAT_ENSEMBLE_FLAG_FREQUENCY - - GEN_ENS_PROD_ENSEMBLE_FLAG_FREQUENCY - * - ENSEMBLE_STAT_ENSEMBLE_FLAG_NEP - - GEN_ENS_PROD_ENSEMBLE_FLAG_NEP - * - ENSEMBLE_STAT_ENSEMBLE_FLAG_NMEP - - GEN_ENS_PROD_ENSEMBLE_FLAG_NMEP - * - ENSEMBLE_STAT_REGRID_TO_GRID - - GEN_ENS_PROD_REGRID_TO_GRID - * - ENSEMBLE_STAT_REGRID_METHOD - - GEN_ENS_PROD_REGRID_METHOD - * - ENSEMBLE_STAT_REGRID_WIDTH - - GEN_ENS_PROD_REGRID_WIDTH - * - ENSEMBLE_STAT_REGRID_VLD_THRESH - - GEN_ENS_PROD_REGRID_VLD_THRESH - * - ENSEMBLE_STAT_REGRID_SHAPE - - GEN_ENS_PROD_REGRID_SHAPE - * - ENSEMBLE_STAT_NBRHD_PROB_WIDTH - - GEN_ENS_PROD_NBRHD_PROB_WIDTH - * - ENSEMBLE_STAT_NBRHD_PROB_SHAPE - - GEN_ENS_PROD_NBRHD_PROB_SHAPE - * - ENSEMBLE_STAT_NBRHD_PROB_VLD_THRESH - - GEN_ENS_PROD_NBRHD_PROB_VLD_THRESH - * - ENSEMBLE_STAT_NMEP_SMOOTH_VLD_THRESH - - GEN_ENS_PROD_NMEP_SMOOTH_VLD_THRESH - * - ENSEMBLE_STAT_NMEP_SMOOTH_SHAPE - - GEN_ENS_PROD_NMEP_SMOOTH_SHAPE - * - ENSEMBLE_STAT_NMEP_SMOOTH_METHOD - - GEN_ENS_PROD_NMEP_SMOOTH_METHOD - * - ENSEMBLE_STAT_NMEP_SMOOTH_WIDTH - - GEN_ENS_PROD_NMEP_SMOOTH_WIDTH - * - ENSEMBLE_STAT_NMEP_SMOOTH_GAUSSIAN_DX - - GEN_ENS_PROD_NMEP_SMOOTH_GAUSSIAN_DX - * - ENSEMBLE_STAT_NMEP_SMOOTH_GAUSSIAN_RADIUS - - GEN_ENS_PROD_NMEP_SMOOTH_GAUSSIAN_RADIUS - -.. _upgrade-ensemble-case1-filename: - -Set GenEnsProd output template to include filename -"""""""""""""""""""""""""""""""""""""""""""""""""" - -* **If the EnsembleStat output template was set**, then copy the value and add a - template for the NetCDF output filename at the end following a forward slash - ‘/’ character. - - For example, if - - .. code-block:: ini - - ENSEMBLE_STAT_OUTPUT_TEMPLATE = {valid?fmt=%Y%m%d%H} - - then set - - .. code-block:: ini - - GEN_ENS_PROD_OUTPUT_TEMPLATE = {valid?fmt=%Y%m%d%H}/gen_ens_prod_{valid?fmt=%Y%m%d_%H%M%S}V_ens.nc - - or something similar. - -* **If the EnsembleStat output template was not set,** then set GenEnsProd’s - template to the desired NetCDF output filename. - - Example: - - .. code-block:: ini - - GEN_ENS_PROD_OUTPUT_TEMPLATE = gen_ens_prod_{valid?fmt=%Y%m%d_%H%M%S}V_ens.nc - -**Ensure that any downstream wrappers in the PROCESS_LIST are configured -to read the correct GenEnsProd output file instead of the _ens.nc file -that was previously generated by EnsembleStat.** - -Handle ENS_VAR variables -""""""""""""""""""""""""""" - -**If ENS_VAR_\* variables are already set,** this section can be skipped. - -**If ENS_VAR_\* variables are not set,** add ENS_VAR variables. - -* If FCST/OBS verification is **NOT** being performed in the use case using another - wrapper, then rename the FCST_VAR variables to ENS_VAR. - - For example: - - .. list-table:: - :widths: 50 50 - :header-rows: 1 - - * - Old Name - - New Name - * - FCST_VAR1_NAME - - ENS_VAR1_NAME - * - FCST_VAR1_LEVELS - - ENS_VAR1_LEVELS - * - FCST_VAR2_NAME - - ENS_VAR2_NAME - * - FCST_VAR2_LEVELS - - ENS_VAR2_LEVELS - - -* If FCST/OBS verification is being performed by another tool, then add - ENS_VAR variables and set them equal to the corresponding - FCST_VAR values. - - For example: - - .. code-block:: ini - - ENS_VAR1_NAME = {FCST_VAR1_NAME} - ENS_VAR1_LEVELS = {FCST_VAR1_LEVELS} - ENS_VAR2_NAME = {FCST_VAR2_NAME} - ENS_VAR2_LEVELS = {FCST_VAR2_LEVELS} - -Remove Variables -"""""""""""""""" - -**Remove any remaining ENSEMBLE_STAT_\* variables that are no longer used.** - -Some examples: - -.. list-table:: - :widths: 50 - - * - ENSEMBLE_STAT_ENSEMBLE_FLAG_RANK - * - ENSEMBLE_STAT_ENSEMBLE_FLAG_WEIGHT - * - ENSEMBLE_STAT_MESSAGE_TYPE - * - ENSEMBLE_STAT_OUTPUT_FLAG_ECNT - * - ENSEMBLE_STAT_OUTPUT_FLAG_RPS - * - ENSEMBLE_STAT_OUTPUT_FLAG_RHIST - * - ENSEMBLE_STAT_OUTPUT_FLAG_PHIST - * - ENSEMBLE_STAT_OUTPUT_FLAG_ORANK - * - ENSEMBLE_STAT_OUTPUT_FLAG_SSVAR - * - ENSEMBLE_STAT_OUTPUT_FLAG_RELP - * - ENSEMBLE_STAT_OUTPUT_FLAG_PCT - * - ENSEMBLE_STAT_OUTPUT_FLAG_PSTD - * - ENSEMBLE_STAT_OUTPUT_FLAG_PJC - * - ENSEMBLE_STAT_OUTPUT_FLAG_PRC - * - ENSEMBLE_STAT_OUTPUT_FLAG_ECLV - * - ENSEMBLE_STAT_DUPLICATE_FLAG - * - ENSEMBLE_STAT_SKIP_CONST - * - ENSEMBLE_STAT_OBS_ERROR_FLAG - * - ENSEMBLE_STAT_ENS_SSVAR_BIN_SIZE - * - ENSEMBLE_STAT_ENS_PHIST_BIN_SIZE - * - ENSEMBLE_STAT_CI_ALPHA - * - ENSEMBLE_STAT_MASK_GRID - * - ENSEMBLE_STAT_MASK_POLY - * - ENSEMBLE_STAT_INTERP_FIELD - * - ENSEMBLE_STAT_INTERP_VLD_THRESH - * - ENSEMBLE_STAT_INTERP_SHAPE - * - ENSEMBLE_STAT_INTERP_METHOD - * - ENSEMBLE_STAT_INTERP_WIDTH - * - ENSEMBLE_STAT_OBS_QUALITY_INC/EXC - * - ENSEMBLE_STAT_GRID_WEIGHT_FLAG - -.. _upgrade-ensemble-case2: - -Case 2: EnsembleStat performing ensemble verification but not generating ensemble products -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -The use case will no longer generate a **_ens.nc** file and may create other -files (**_orank.nc** and **txt**) that contain requested output. - -Rename Variables -"""""""""""""""" - -.. list-table:: - :widths: 50 50 - :header-rows: 1 - - * - Old Name - - New Name - * - ENSEMBLE_STAT_ENSEMBLE_FLAG_MEAN - - ENSEMBLE_STAT_NC_ORANK_FLAG_MEAN - * - ENSEMBLE_STAT_ENSEMBLE_FLAG_RANK - - ENSEMBLE_STAT_NC_ORANK_FLAG_RANK - * - ENSEMBLE_STAT_ENSEMBLE_FLAG_WEIGHT - - ENSEMBLE_STAT_NC_ORANK_FLAG_WEIGHT - * - ENSEMBLE_STAT_ENSEMBLE_FLAG_VLD_COUNT - - ENSEMBLE_STAT_NC_ORANK_FLAG_VLD_COUNT - -Remove Variables -"""""""""""""""" - -.. list-table:: - :widths: 50 - - * - All ENS_VAR_* variables - * - All ENSEMBLE_STAT_ENSEMBLE_FLAG_* variables - * - ENSEMBLE_STAT_NBRHD_PROB_WIDTH - * - ENSEMBLE_STAT_NBRHD_PROB_SHAPE - * - ENSEMBLE_STAT_NBRHD_PROB_VLD_THRESH - * - ENSEMBLE_STAT_NMEP_SMOOTH_VLD_THRESH - * - ENSEMBLE_STAT_NMEP_SMOOTH_SHAPE - * - ENSEMBLE_STAT_NMEP_SMOOTH_METHOD - * - ENSEMBLE_STAT_NMEP_SMOOTH_WIDTH - * - ENSEMBLE_STAT_NMEP_SMOOTH_GAUSSIAN_DX - * - ENSEMBLE_STAT_NMEP_SMOOTH_GAUSSIAN_RADIUS - -.. _upgrade-ensemble-case3: - -Case 3: EnsembleStat generating ensemble products and performing ensemble verification -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Add GenEnsProd to PROCESS_LIST -"""""""""""""""""""""""""""""" - -GenEnsProd will need to be added to the PROCESS_LIST in addition to -EnsembleStat to generate the ensemble verification output. - - .. code-block:: ini - - PROCESS_LIST = ..., EnsembleStat, GenEnsProd, ... - -Set input variables -""""""""""""""""""" - -Set the input dir and template variables for **GenEnsProd** to match -the values set for FCST input to EnsembleStat. -Also set the output dir to match EnsembleStat output dir. - - .. code-block:: ini - - GEN_ENS_PROD_INPUT_DIR = {FCST_ENSEMBLE_STAT_INPUT_DIR} - GEN_ENS_PROD_INPUT_TEMPLATE = {FCST_ENSEMBLE_STAT_INPUT_TEMPLATE} - GEN_ENS_PROD_OUTPUT_DIR = {ENSEMBLE_STAT_OUTPUT_DIR} - -Set GenEnsProd output template to include filename -"""""""""""""""""""""""""""""""""""""""""""""""""" - -* **If the EnsembleStat output template is set**, then copy the value and add a - template for the NetCDF output filename at the end following a forward slash - ‘/’ character. - - For example, if - - .. code-block:: ini - - ENSEMBLE_STAT_OUTPUT_TEMPLATE = {valid?fmt=%Y%m%d%H} - - then set - - .. code-block:: ini - - GEN_ENS_PROD_OUTPUT_TEMPLATE = {valid?fmt=%Y%m%d%H}/gen_ens_prod_{valid?fmt=%Y%m%d_%H%M%S}V_ens.nc - - or something similar. - -* **If the EnsembleStat output template is not set,** then set GenEnsProd’s - template to the desired NetCDF output filename. Here is an example: - - .. code-block:: ini - - GEN_ENS_PROD_OUTPUT_TEMPLATE = gen_ens_prod_{valid?fmt=%Y%m%d_%H%M%S}V_ens.nc - -**Ensure that any downstream wrappers in the PROCESS_LIST are configured -to read the correct GenEnsProd output file instead of the _ens.nc file -that was previously generated by EnsembleStat.** - -Handle ENS_VAR variables -"""""""""""""""""""""""" - -**If ENS_VAR_\* variables are already set,** this section can be skipped. - -**If ENS_VAR_\* variables are not set,** add ENS_VAR variables. - -* If FCST_ENSEMBLE_STAT_VAR_\* variables are set, - set the ENS_VAR_\* values to the same values. - - For example: - - .. code-block:: ini - - ENS_VAR1_NAME = {FCST_ENSEMBLE_STAT_VAR1_NAME} - ENS_VAR1_LEVELS = {FCST_ENSEMBLE_STAT_VAR1_LEVELS} - ENS_VAR2_NAME = {FCST_ENSEMBLE_STAT_VAR2_NAME} - ENS_VAR2_LEVELS = {FCST_ENSEMBLE_STAT_VAR2_LEVELS} - -* If FCST_ENSEMBLE_STAT_VAR_\* variables are **not** set, - set the ENS_VAR_\* values to the values set for the FCST_VAR_\*. - - For example: - - .. code-block:: ini - - ENS_VAR1_NAME = {FCST_VAR1_NAME} - ENS_VAR1_LEVELS = {FCST_VAR1_LEVELS} - ENS_VAR2_NAME = {FCST_VAR2_NAME} - ENS_VAR2_LEVELS = {FCST_VAR2_LEVELS} - -Set GenEnsProd Variables -"""""""""""""""""""""""" - -**If any of the following ENSEMBLE_STAT_\* variables are set in the -configuration file, then rename them to the corresponding -GEN_ENS_PROD_\* variable. -These are no longer valid settings for EnsembleStat.** - -.. list-table:: - :widths: 50 50 - :header-rows: 1 - - * - Old Name - - New Name - * - ENSEMBLE_STAT_NBRHD_PROB_WIDTH - - GEN_ENS_PROD_NBRHD_PROB_WIDTH - * - ENSEMBLE_STAT_NBRHD_PROB_SHAPE - - GEN_ENS_PROD_NBRHD_PROB_SHAPE - * - ENSEMBLE_STAT_NBRHD_PROB_VLD_THRESH - - GEN_ENS_PROD_NBRHD_PROB_VLD_THRESH - * - ENSEMBLE_STAT_NMEP_SMOOTH_VLD_THRESH - - GEN_ENS_PROD_NMEP_SMOOTH_VLD_THRESH - * - ENSEMBLE_STAT_NMEP_SMOOTH_SHAPE - - GEN_ENS_PROD_NMEP_SMOOTH_SHAPE - * - ENSEMBLE_STAT_NMEP_SMOOTH_METHOD - - GEN_ENS_PROD_NMEP_SMOOTH_METHOD - * - ENSEMBLE_STAT_NMEP_SMOOTH_WIDTH - - GEN_ENS_PROD_NMEP_SMOOTH_WIDTH - * - ENSEMBLE_STAT_NMEP_SMOOTH_GAUSSIAN_DX - - GEN_ENS_PROD_NMEP_SMOOTH_GAUSSIAN_DX - * - ENSEMBLE_STAT_NMEP_SMOOTH_GAUSSIAN_RADIUS - - GEN_ENS_PROD_NMEP_SMOOTH_GAUSSIAN_RADIUS - -**If any of the following ENSEMBLE_STAT_\* variables are set in the -configuration file, then set the corresponding GEN_ENS_PROD_\* -variables to the same value or reference the ENSEMBLE_STAT_\* version.** - -.. list-table:: - :widths: 50 - - * - ENSEMBLE_STAT_N_MEMBERS - * - ENSEMBLE_STAT_ENS_THRESH - * - ENSEMBLE_STAT_REGRID_TO_GRID - * - ENSEMBLE_STAT_REGRID_METHOD - * - ENSEMBLE_STAT_REGRID_WIDTH - * - ENSEMBLE_STAT_REGRID_VLD_THRESH - * - ENSEMBLE_STAT_REGRID_SHAPE - * - FCST_ENSEMBLE_STAT_INPUT_GRID_DATATYPE - -Example: - - .. code-block:: ini - - GEN_ENS_PROD_N_MEMBERS = {ENSEMBLE_STAT_N_MEMBERS} - GEN_ENS_PROD_ENS_THRESH = {ENSEMBLE_STAT_ENS_THRESH} - GEN_ENS_PROD_REGRID_TO_GRID = {ENSEMBLE_STAT_REGRID_TO_GRID} - GEN_ENS_PROD_REGRID_METHOD = {ENSEMBLE_STAT_REGRID_METHOD} - GEN_ENS_PROD_REGRID_WIDTH = {ENSEMBLE_STAT_REGRID_WIDTH} - GEN_ENS_PROD_REGRID_VLD_THRESH = {ENSEMBLE_STAT_REGRID_VLD_THRESH} - GEN_ENS_PROD_REGRID_SHAPE = {ENSEMBLE_STAT_REGRID_SHAPE} - GEN_ENS_PROD_INPUT_DATATYPE = {FCST_ENSEMBLE_STAT_INPUT_GRID_DATATYPE} - -**If any of the following ENSEMBLE_STAT_ENSEMBLE_FLAG_\* variables are set -in the configuration file, then rename them to the corresponding -ENSEMBLE_STAT_NC_ORANK_FLAG_\* variables AND add the corresponding -GEN_ENS_PROD_ENSEMBLE_FLAG_\* variables with the same value.** - -.. list-table:: - :widths: 50 - - * - ENSEMBLE_STAT_ENSEMBLE_FLAG_LATLON - * - ENSEMBLE_STAT_ENSEMBLE_FLAG_MEAN - * - ENSEMBLE_STAT_ENSEMBLE_FLAG_VLD_COUNT - -For example, if - - .. code-block:: ini - - ENSEMBLE_STAT_ENSEMBLE_FLAG_LATLON = TRUE - -then remove it and set - - .. code-block:: ini - - ENSEMBLE_STAT_NC_ORANK_FLAG_LATLON = TRUE - GEN_ENS_PROD_ENSEMBLE_FLAG_LATLON = TRUE - -Another example, if - - .. code-block:: ini - - ENSEMBLE_STAT_ENSEMBLE_FLAG_MEAN = FALSE - -then remove it and set - - .. code-block:: ini - - ENSEMBLE_STAT_NC_ORANK_FLAG_MEAN = FALSE - GEN_ENS_PROD_ENSEMBLE_FLAG_MEAN = FALSE - -Rename Variables -"""""""""""""""" - -.. list-table:: - :widths: 50 50 - :header-rows: 1 - - * - Old Name - - New Name - * - ENSEMBLE_STAT_ENSEMBLE_FLAG_STDEV - - GEN_ENS_PROD_ENSEMBLE_FLAG_STDEV - * - ENSEMBLE_STAT_ENSEMBLE_FLAG_MINUS - - GEN_ENS_PROD_ENSEMBLE_FLAG_MINUS - * - ENSEMBLE_STAT_ENSEMBLE_FLAG_PLUS - - GEN_ENS_PROD_ENSEMBLE_FLAG_PLUS - * - ENSEMBLE_STAT_ENSEMBLE_FLAG_MIN - - GEN_ENS_PROD_ENSEMBLE_FLAG_MIN - * - ENSEMBLE_STAT_ENSEMBLE_FLAG_MAX - - GEN_ENS_PROD_ENSEMBLE_FLAG_MAX - * - ENSEMBLE_STAT_ENSEMBLE_FLAG_RANGE - - GEN_ENS_PROD_ENSEMBLE_FLAG_RANGE - * - ENSEMBLE_STAT_ENSEMBLE_FLAG_FREQUENCY - - GEN_ENS_PROD_ENSEMBLE_FLAG_FREQUENCY - * - ENSEMBLE_STAT_ENSEMBLE_FLAG_NEP - - GEN_ENS_PROD_ENSEMBLE_FLAG_NEP - * - ENSEMBLE_STAT_ENSEMBLE_FLAG_NMEP - - GEN_ENS_PROD_ENSEMBLE_FLAG_NMEP +No upgrade instructions are needed for this release. -For further assistance, please navigate to the -`METplus Discussions `_ page. +**TODO: Add info about switching to using provided wrapped MET config file instead of user-defined.** diff --git a/metplus/VERSION b/metplus/VERSION index 8fb565199c..9d9136f3d9 100644 --- a/metplus/VERSION +++ b/metplus/VERSION @@ -1 +1 @@ -6.0.0-beta1-dev +6.0.0-beta2-dev