diff --git a/.github/actions/run_tests/Dockerfile.run_geovista b/.github/actions/run_tests/Dockerfile.run_geovista new file mode 100644 index 0000000000..c8d8d82310 --- /dev/null +++ b/.github/actions/run_tests/Dockerfile.run_geovista @@ -0,0 +1,15 @@ +ARG METPLUS_ENV_TAG=metplus_base.v5 +ARG METPLUS_IMG_TAG=develop + +FROM dtcenter/metplus-envs:${METPLUS_ENV_TAG} as env + +ARG METPLUS_IMG_TAG=develop +FROM dtcenter/metplus-dev:${METPLUS_IMG_TAG} + +RUN mkdir -p /usr/local/conda/envs && mkdir -p /usr/local/conda/bin +COPY --from=env /usr/local/conda/envs /usr/local/conda/envs/ +COPY --from=env /usr/local/conda/bin/conda /usr/local/conda/bin/conda + +# copy libGL and libEGL libraries to prevent dynamic lib errors +COPY --from=env /lib/x86_64-linux-gnu/libEGL* /lib/x86_64-linux-gnu/ +COPY --from=env /lib/x86_64-linux-gnu/libGL* /lib/x86_64-linux-gnu/ diff --git a/.github/jobs/setup_and_run_use_cases.py b/.github/jobs/setup_and_run_use_cases.py index 8ed341ec4a..931a79d76d 100755 --- a/.github/jobs/setup_and_run_use_cases.py +++ b/.github/jobs/setup_and_run_use_cases.py @@ -181,6 +181,8 @@ def _get_dockerfile_name(requirements): return f'{dockerfile_name}_gfdl' if 'cartopy' in str(requirements).lower(): return f'{dockerfile_name}_cartopy' + if 'geovista' in str(requirements).lower(): + return f'{dockerfile_name}_geovista' return dockerfile_name diff --git a/.github/parm/use_case_groups.json b/.github/parm/use_case_groups.json index 1752f76df0..041b000b2c 100644 --- a/.github/parm/use_case_groups.json +++ b/.github/parm/use_case_groups.json @@ -277,6 +277,6 @@ { "category": "unstructured_grids", "index_list": "0", - "run": false + "run": true } ] diff --git a/docs/Users_Guide/getting_started.rst b/docs/Users_Guide/getting_started.rst index 5d7287bd25..4d25c4f28d 100644 --- a/docs/Users_Guide/getting_started.rst +++ b/docs/Users_Guide/getting_started.rst @@ -387,9 +387,10 @@ METplus is available on DockerHub. The METplus Docker image includes all of the MET executables from the corresponding METplus Coordinated Release and the METplus wrappers are pre-configured to use them. -To pull the latest official release, run:: +To pull the latest official release, use the X.Y-latest tag where X.Y is the +version, e.g.:: - docker pull dtcenter/metplus:latest + docker pull dtcenter/metplus:5.1-latest Tags for previous releases and development releases are also available. Refer to the list of @@ -414,12 +415,12 @@ To make these data available in a METplus Docker container, first create a Docker data volume from the desired tag and give it a name with the *--name* argument:: - docker create --name met_tool_wrapper dtcenter/metplus-data:4.1-met_tool_wrapper + docker create --name met_tool_wrapper dtcenter/metplus-data:5.1-met_tool_wrapper Then mount the data volume to the container using the *--volumes-from* argument to the *docker run* command:: - docker run --rm -it --volumes-from met_tool_wrapper dtcenter/metplus:4.1.4 bash + docker run --rm -it --volumes-from met_tool_wrapper dtcenter/metplus:5.1-latest bash The input data will be available inside the container under /data/input/METplus_Data. diff --git a/docs/Users_Guide/release-notes.rst b/docs/Users_Guide/release-notes.rst index eedb3f9b26..a07a77b894 100644 --- a/docs/Users_Guide/release-notes.rst +++ b/docs/Users_Guide/release-notes.rst @@ -30,295 +30,45 @@ When applicable, release notes are followed by the `GitHub issue `__ number which describes the bugfix, enhancement, or new feature. -METplus Version 5.1.0 Release Notes (2023-07-31) ------------------------------------------------- +METplus Version 6.0.0 Beta 1 Release Notes (2023-09-15) +------------------------------------------------------- .. dropdown:: Enhancements - * Add support for multiple interp widths - (`#2049 `_) - * TCPairs - Add support for setting consensus.write_members - (`#2054 `_) - * Update use cases to use new Python directory structure in MET - (`#2115 `_) - * Add support for new multivariate MODE settings - (`#2197 `_, - `#2210 `_, - `#2230 `_, - `#2235 `_) + * **Remove support for deprecated environment variables for old wrapped MET config files** + (`#2299 `_) + * Improve time formatting logic to include certain times and use day of week to subset + (`#2283 `_) + * Remove TCMPRPlotter wrapper + (`#2310 `_) .. dropdown:: Bugfix - * StatAnalysis - allow run once for each valid time - (`#2026 `_) - * App specific OBS_WINDOW variables not taking precedence over generic - (`#2006 `_) - * Skip-if-output-exists logic incorrectly skips files - (`#2096 `_) - * PointStat -obs_valid_beg/end arguments not set properly - (`#2137 `_) - * Allow setting of convert, censor_thresh, and censor_val in regrid dictionary - (`#2082 `_) - * TCPairs setting -diag option causes failure - (`#2179 `_) - * Define the order of the forecast variables numerically rather than alphabetically - (`#2070 `_) - * Prevent error if no commands were run because they were skipped - (`#2098 `_) - * Allow spaces for complex categorical thresholds - (`#2189 `_) - * PCPCombine - Extra field options not set in -subtract mode - (`#2161 `_) - * StatAnalysis time shifting failure - (`#2168 `_) - * TCPairs: skip times logic incorrectly skips additional times - (`#2212 `_) - * TCStat fails if -out_stat directory does not exist - (`#2241 `_) - * Ensure log instances for concurrent runs are unique - (`#2245 `_) - * PointStat - Support running with no MASK provided - (`#1853 `_) + * Update buoy use case to use buoy station file from 2022 + (`#2279 `_) + * Prevent failure in LSR use case + (`#2294 `_) + .. dropdown:: New Wrappers - * TCDiag (beta) - (`#1626 `_) + NONE .. dropdown:: New Use Cases - * Multi-Variate MODE (`#1516 `_) - * Read in Argo profile data netCDF files for use in METplus with python embedding - (`#1977 `_) - * PANDA-C: MPAS to SATCORPS - (`#2188 `_) - * PANDA-C: MPAS to MERRA2 - (`#2188 `_) - * PANDA-C: MPAS to ERA5 - (`#2188 `_) - * PANDA-C: GFS to SATCORPS - (`#2188 `_) - * PANDA-C: GFS to MERRA2 - (`#2188 `_) - * PANDA-C: GFS to ERA5 - (`#2188 `_) - * PointStat AMDAR PBLH with python embedding - (`#2198 `_) - * Space-time cross-spectra for S2S forecasts - (`#2136 `_) + * Scatterometer wind data + (`#1488 `_) .. dropdown:: Documentation - * Enhance the Release Notes by adding dropdown menus - (`#2076 `_) - * Update the METplus Components Python Requirements - (`#1978 `_, - `#2016 `_) - * Add documentation on support for releases to the Release Guide - (`#2106 `_) - * Add use case quick search keywords for METplotpy and METcalcpy - (`#2151 `_) + NONE .. dropdown:: Internal - * Improve use case testing - (`#685 `_) - * Update conda environments to use 3.10 for automated use case tests - (`#2005 `_) - * Add modulefiles to the repository - (`#2015 `_) - * **Upgrade to using Python 3.10.4** - (`#2022 `_) - * Add 'LICENSE.md' to the METplus repo - (`#2058 `_) - * Update Contributor's Guide to use GH Action to update truth data - (`#2068 `_) - * Enhance GitHub Workflow documentation - (`#2147 `_) - * Update the development release guide instructions to remove references to a Coordinated release - (`#2159 `_) - * Refactored code to resolve many SonarQube items - (`#1610 `_) - * Improve Contributor's Guide - (`#2138 `_, - `#2207 `_) - * Bugfix: Fix difference test logic - (`#2244 `_) - * Remove base environment from Docker Conda images - (`#2249 `_) - * PR Templates (all METplus component repos): Improve language about linking relevant issue(s) - (`#2257 `_) - - -METplus Version 5.0.0 Release Notes (2022-12-09) ------------------------------------------------- - - .. warning:: **MAJOR CHANGES**: - - * The LOOP_ORDER config variable was removed. The variable set in a user's - config file will be ignored in favor of executing the logic that - corresponds to *LOOP_ORDER = processes*, where all times are processed for - the first item in the PROCESS_LIST, then all times are processed for the - second item in the PROCESS_LIST, etc. This may change the order that - commands are executed in a use case, but it should not change the results. - * The METplus Dockerfile was moved to internal/scripts/docker. - It was previously found in scripts/docker. - * Use cases that include **EnsembleStat** wrapper will require config file - updates. See :ref:`upgrade-instructions`. - * The default value of :term:`SCRUB_STAGING_DIR` is now *True*. - This means some intermediate files that are auto-generated by METplus such - as file lists and uncompressed files will automatically be removed unless - this option is set by the user. - These files are typically only used to debug unexpected issues. - * The default value of :term:`METPLUS_CONF` now includes the - :term:`LOG_TIMESTAMP` so each METplus run will generate a unique final - config file, e.g. metplus_final.conf.20220921121733. - - - .. dropdown:: Enhancements - - * **Enhance MODE wrapper to support multi-variate MODE** - (`#1585 `_) - * **Allow FCST_IS_PROB variable setting specific to tool - (FCST__IS_PROB)** - (`#1586 `_) - * **Enhance climatology field settings to be consistent with fcst/obs field** - (`#1599 `_) - * Update Hovmoeller Use case to use updated Hovmoeller plotting - (`#1650 `_) - * **Update the EnsembleStat wrapper and use case examples to remove - ensemble post processing logic** - (`#1816 `_) - * Enhance logic to consistently create directories - (`#1657 `_) - * Create checksum for released code - (`#262 `_) - * Add the user ID to the log output at beginning and end of each - METplus wrappers run - (`dtcenter/METplus-Internal#20 `_) - * Update logic to name final conf and intermediate files with a unique - identifier - (`dtcenter/METplus-Internal#32 `_) - * Change default logging time information - (`dtcenter/METplus-Internal#34 `_) - * **Remove LOOP_ORDER config variable** - (`#1687 `_) - * **Add unique identifier for each METplus run to configuration** - (`#1829 `_) - * StatAnalysis - Support setting multiple jobs - (`#1842 `_) - * StatAnalysis - Set MET verbosity - (`#1772 `_) - * StatAnalysis - Support using both init/valid variables in - string substitution - (`#1861 `_) - * StatAnalysis - Allow filename template tags in jobs - (`#1862 `_) - * StatAnalysis - Support looping over groups of list items - (`#1870 `_) - * StatAnalysis - Allow processing of time ranges other than daily - (`#1871 `_) - * StatAnalysis - Add support for using a custom loop list - (`#1893 `_) - * Remove MakePlots wrapper - (`#1843 `_) - * Add support in EnsembleStat wrapper for setting -ens_mean - command line argument - (`#1569 `_) - * Enhance METplus to have better signal handling for shutdown events - (`dtcenter/METplus-Internal#27 `_) - * TCPairs and TCStat - add support for new config options and - command line arguments - (`#1898 `_) - * Enhance the GridStat and PointStat wrappers to handle the - addition of SEEPS - (`#1953 `_) - * SeriesAnalysis - add support for setting mask dictionary - (`#1926 `_) - * Update Python requirement to 3.8.6 - (`#1566 `_) - * Enhance StatAnalysis wrapper to support now and today - (`#1669 `_) - * **Clean up use case configuration files** - (`#1402 `_) - * Add support for creating multiple input datasets - (`#1694 `_) - - .. dropdown:: Bugfixes - - * PCPCombine - custom loop list does not work for subtract method - (`#1884 `_) - * Set level properly in filename template for EnsembleStat forecast input - (`#1910 `_) - * Prevent duplicate observation files using a file window if - compressed equivalent files exist in same directory - (`#1939 `_) - * Allow NA value for _CLIMO_[MEAN/STDEV]_HOUR_INTERVAL - (`#1787 `_) - * Reconcile setting of METPLOTPY_BASE for use cases - (`#1713 `_) - * Add support for the {custom} loop string in the MODEL config variable - (`#1382 `_) - * Fix PCPCombine extra options removal of semi-colon - (`#1534 `_) - * Fix reset of arguments for some wrappers - (i.e. GenEnsProd) after each run - (`#1555 `_) - * Enhance METDbLoad Wrapper to find MODE .txt files - (`#1608 `_) - * Add missing brackets around list variable values for StatAnalysis wrapper - (`#1641 `_) - * Allow NA value for _CLIMO_[MEAN/STDEV]_DAY_INTERVAL - (`#1653 `_) - - .. dropdown:: New Wrappers - - * PlotPointObs - (`#1489 `_) - - .. dropdown:: New Use Cases - - * PANDA-C use cases - (`#1686 `_) - * MJO-ENSO diagnostics - (`#1330 `_) - * Probability of Exceedence for 85th percentile temperatures - (`#1808 `_) - * FV3 Physics Tendency plotting via METplotpy - (`#1852 `_) - * StatAnalysis Python Embedding using IODA v2.0 - (`#1453 `_) - * StatAnalysis Python Embedding to read native grid (u-grid) - (`#1561 `_) - - .. dropdown:: Documentation - - * Update documentation to include instructions - to disable UserScript wrapper - (`dtcenter/METplus-Internal#33 `_) - - .. dropdown:: Internal - - * Organize utility scripts used by multiple wrappers - (`#344 `_) - * Fix GitHub Actions warnings - update the version of actions - and replace set-output - (`#1863 `_) - * Update diff logic to handle CSV files that have rounding differences - (`#1865 `_) - * Add unit tests for expected failure - (`dtcenter/METplus-Internal#24 `_) - * Add instructions in Release Guide for "Recreate an Existing Release" - (`#1746 `_) - * Add modulefiles used for installations on various machines - (`#1749 `_) - * Document GitHub Discussions procedure for the Contributor's Guide - (`#1159 `_) - * Create a METplus "Release Guide" describing how to build - releases for the METplus components - (`#673 `_) - * Update documentation about viewing RTD URLs on branches - (`#1512 `_) + * Add coordinated release checklist to the METplus Release Guide + (`#2282 `_) + * Recreate Docker/Conda environments after METbaseimage OS upgrade + (`#2338 `_) .. _upgrade-instructions: @@ -326,496 +76,6 @@ METplus Version 5.0.0 Release Notes (2022-12-09) METplus Wrappers Upgrade Instructions ===================================== -EnsembleStat/GenEnsProd (v5.0.0) --------------------------------- - -.. note:: - - If :ref:`ensemble_stat_wrapper` is not found in the :term:`PROCESS_LIST` - for any use cases, then this section is not relevant. - If upgrading from v5.0.X to v5.1.0 or higher, - then this section is not relevant. - -The METplus v5.0.0 coordinated release includes changes that remove ensemble -product generation from EnsembleStat. GenEnsProd is now required to generate -ensemble products. There are 3 situations listed below that require slightly -different modifications: - -* :ref:`upgrade-ensemble-case1` -* :ref:`upgrade-ensemble-case2` -* :ref:`upgrade-ensemble-case3` - -.. _upgrade-ensemble-case1: - -Case 1: EnsembleStat only generating ensemble products -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -If the use case had been calling EnsembleStat **WITHOUT** the **-grid_obs** or -**-point_obs** command line options, we can assume it was only doing ensemble -post-processing. -That call to EnsembleStat should be replaced with a call to -GenEnsProd instead. - -Rename Variables -"""""""""""""""" - -.. role:: raw-html(raw) - :format: html - -.. list-table:: - :widths: 50 50 - :header-rows: 1 - - * - Old Name - - New Name - * - FCST_ENSEMBLE_STAT_INPUT_DIR - - GEN_ENS_PROD_INPUT_DIR - * - FCST_ENSEMBLE_STAT_INPUT_TEMPLATE - - GEN_ENS_PROD_INPUT_TEMPLATE - * - ENSEMBLE_STAT_OUTPUT_DIR - - GEN_ENS_PROD_OUTPUT_DIR - * - ENSEMBLE_STAT_OUTPUT_TEMPLATE - - GEN_ENS_PROD_OUTPUT_TEMPLATE :raw-html:`
` - **and add filename, see** :ref:`below` - * - ENSEMBLE_STAT_N_MEMBERS - - GEN_ENS_PROD_N_MEMBERS - * - ENSEMBLE_STAT_ENS_THRESH - - GEN_ENS_PROD_ENS_THRESH - * - ENSEMBLE_STAT_ENS_VLD_THRESH - - GEN_ENS_PROD_VLD_THRESH - * - ENSEMBLE_STAT_ENSEMBLE_FLAG_LATLON - - GEN_ENS_PROD_ENSEMBLE_FLAG_LATLON - * - ENSEMBLE_STAT_ENSEMBLE_FLAG_MEAN - - GEN_ENS_PROD_ENSEMBLE_FLAG_MEAN - * - ENSEMBLE_STAT_ENSEMBLE_FLAG_STDEV - - GEN_ENS_PROD_ENSEMBLE_FLAG_STDEV - * - ENSEMBLE_STAT_ENSEMBLE_FLAG_MINUS - - GEN_ENS_PROD_ENSEMBLE_FLAG_MINUS - * - ENSEMBLE_STAT_ENSEMBLE_FLAG_PLUS - - GEN_ENS_PROD_ENSEMBLE_FLAG_PLUS - * - ENSEMBLE_STAT_ENSEMBLE_FLAG_MIN - - GEN_ENS_PROD_ENSEMBLE_FLAG_MIN - * - ENSEMBLE_STAT_ENSEMBLE_FLAG_MAX - - GEN_ENS_PROD_ENSEMBLE_FLAG_MAX - * - ENSEMBLE_STAT_ENSEMBLE_FLAG_RANGE - - GEN_ENS_PROD_ENSEMBLE_FLAG_RANGE - * - ENSEMBLE_STAT_ENSEMBLE_FLAG_VLD_COUNT - - GEN_ENS_PROD_ENSEMBLE_FLAG_VLD_COUNT - * - ENSEMBLE_STAT_ENSEMBLE_FLAG_FREQUENCY - - GEN_ENS_PROD_ENSEMBLE_FLAG_FREQUENCY - * - ENSEMBLE_STAT_ENSEMBLE_FLAG_NEP - - GEN_ENS_PROD_ENSEMBLE_FLAG_NEP - * - ENSEMBLE_STAT_ENSEMBLE_FLAG_NMEP - - GEN_ENS_PROD_ENSEMBLE_FLAG_NMEP - * - ENSEMBLE_STAT_REGRID_TO_GRID - - GEN_ENS_PROD_REGRID_TO_GRID - * - ENSEMBLE_STAT_REGRID_METHOD - - GEN_ENS_PROD_REGRID_METHOD - * - ENSEMBLE_STAT_REGRID_WIDTH - - GEN_ENS_PROD_REGRID_WIDTH - * - ENSEMBLE_STAT_REGRID_VLD_THRESH - - GEN_ENS_PROD_REGRID_VLD_THRESH - * - ENSEMBLE_STAT_REGRID_SHAPE - - GEN_ENS_PROD_REGRID_SHAPE - * - ENSEMBLE_STAT_NBRHD_PROB_WIDTH - - GEN_ENS_PROD_NBRHD_PROB_WIDTH - * - ENSEMBLE_STAT_NBRHD_PROB_SHAPE - - GEN_ENS_PROD_NBRHD_PROB_SHAPE - * - ENSEMBLE_STAT_NBRHD_PROB_VLD_THRESH - - GEN_ENS_PROD_NBRHD_PROB_VLD_THRESH - * - ENSEMBLE_STAT_NMEP_SMOOTH_VLD_THRESH - - GEN_ENS_PROD_NMEP_SMOOTH_VLD_THRESH - * - ENSEMBLE_STAT_NMEP_SMOOTH_SHAPE - - GEN_ENS_PROD_NMEP_SMOOTH_SHAPE - * - ENSEMBLE_STAT_NMEP_SMOOTH_METHOD - - GEN_ENS_PROD_NMEP_SMOOTH_METHOD - * - ENSEMBLE_STAT_NMEP_SMOOTH_WIDTH - - GEN_ENS_PROD_NMEP_SMOOTH_WIDTH - * - ENSEMBLE_STAT_NMEP_SMOOTH_GAUSSIAN_DX - - GEN_ENS_PROD_NMEP_SMOOTH_GAUSSIAN_DX - * - ENSEMBLE_STAT_NMEP_SMOOTH_GAUSSIAN_RADIUS - - GEN_ENS_PROD_NMEP_SMOOTH_GAUSSIAN_RADIUS - -.. _upgrade-ensemble-case1-filename: - -Set GenEnsProd output template to include filename -"""""""""""""""""""""""""""""""""""""""""""""""""" - -* **If the EnsembleStat output template was set**, then copy the value and add a - template for the NetCDF output filename at the end following a forward slash - ‘/’ character. - - For example, if - - .. code-block:: ini - - ENSEMBLE_STAT_OUTPUT_TEMPLATE = {valid?fmt=%Y%m%d%H} - - then set - - .. code-block:: ini - - GEN_ENS_PROD_OUTPUT_TEMPLATE = {valid?fmt=%Y%m%d%H}/gen_ens_prod_{valid?fmt=%Y%m%d_%H%M%S}V_ens.nc - - or something similar. - -* **If the EnsembleStat output template was not set,** then set GenEnsProd’s - template to the desired NetCDF output filename. - - Example: - - .. code-block:: ini - - GEN_ENS_PROD_OUTPUT_TEMPLATE = gen_ens_prod_{valid?fmt=%Y%m%d_%H%M%S}V_ens.nc - -**Ensure that any downstream wrappers in the PROCESS_LIST are configured -to read the correct GenEnsProd output file instead of the _ens.nc file -that was previously generated by EnsembleStat.** - -Handle ENS_VAR variables -""""""""""""""""""""""""""" - -**If ENS_VAR_\* variables are already set,** this section can be skipped. - -**If ENS_VAR_\* variables are not set,** add ENS_VAR variables. - -* If FCST/OBS verification is **NOT** being performed in the use case using another - wrapper, then rename the FCST_VAR variables to ENS_VAR. - - For example: - - .. list-table:: - :widths: 50 50 - :header-rows: 1 - - * - Old Name - - New Name - * - FCST_VAR1_NAME - - ENS_VAR1_NAME - * - FCST_VAR1_LEVELS - - ENS_VAR1_LEVELS - * - FCST_VAR2_NAME - - ENS_VAR2_NAME - * - FCST_VAR2_LEVELS - - ENS_VAR2_LEVELS - - -* If FCST/OBS verification is being performed by another tool, then add - ENS_VAR variables and set them equal to the corresponding - FCST_VAR values. - - For example: - - .. code-block:: ini - - ENS_VAR1_NAME = {FCST_VAR1_NAME} - ENS_VAR1_LEVELS = {FCST_VAR1_LEVELS} - ENS_VAR2_NAME = {FCST_VAR2_NAME} - ENS_VAR2_LEVELS = {FCST_VAR2_LEVELS} - -Remove Variables -"""""""""""""""" - -**Remove any remaining ENSEMBLE_STAT_\* variables that are no longer used.** - -Some examples: - -.. list-table:: - :widths: 50 - - * - ENSEMBLE_STAT_ENSEMBLE_FLAG_RANK - * - ENSEMBLE_STAT_ENSEMBLE_FLAG_WEIGHT - * - ENSEMBLE_STAT_MESSAGE_TYPE - * - ENSEMBLE_STAT_OUTPUT_FLAG_ECNT - * - ENSEMBLE_STAT_OUTPUT_FLAG_RPS - * - ENSEMBLE_STAT_OUTPUT_FLAG_RHIST - * - ENSEMBLE_STAT_OUTPUT_FLAG_PHIST - * - ENSEMBLE_STAT_OUTPUT_FLAG_ORANK - * - ENSEMBLE_STAT_OUTPUT_FLAG_SSVAR - * - ENSEMBLE_STAT_OUTPUT_FLAG_RELP - * - ENSEMBLE_STAT_OUTPUT_FLAG_PCT - * - ENSEMBLE_STAT_OUTPUT_FLAG_PSTD - * - ENSEMBLE_STAT_OUTPUT_FLAG_PJC - * - ENSEMBLE_STAT_OUTPUT_FLAG_PRC - * - ENSEMBLE_STAT_OUTPUT_FLAG_ECLV - * - ENSEMBLE_STAT_DUPLICATE_FLAG - * - ENSEMBLE_STAT_SKIP_CONST - * - ENSEMBLE_STAT_OBS_ERROR_FLAG - * - ENSEMBLE_STAT_ENS_SSVAR_BIN_SIZE - * - ENSEMBLE_STAT_ENS_PHIST_BIN_SIZE - * - ENSEMBLE_STAT_CI_ALPHA - * - ENSEMBLE_STAT_MASK_GRID - * - ENSEMBLE_STAT_MASK_POLY - * - ENSEMBLE_STAT_INTERP_FIELD - * - ENSEMBLE_STAT_INTERP_VLD_THRESH - * - ENSEMBLE_STAT_INTERP_SHAPE - * - ENSEMBLE_STAT_INTERP_METHOD - * - ENSEMBLE_STAT_INTERP_WIDTH - * - ENSEMBLE_STAT_OBS_QUALITY_INC/EXC - * - ENSEMBLE_STAT_GRID_WEIGHT_FLAG - -.. _upgrade-ensemble-case2: - -Case 2: EnsembleStat performing ensemble verification but not generating ensemble products -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -The use case will no longer generate a **_ens.nc** file and may create other -files (**_orank.nc** and **txt**) that contain requested output. - -Rename Variables -"""""""""""""""" - -.. list-table:: - :widths: 50 50 - :header-rows: 1 - - * - Old Name - - New Name - * - ENSEMBLE_STAT_ENSEMBLE_FLAG_MEAN - - ENSEMBLE_STAT_NC_ORANK_FLAG_MEAN - * - ENSEMBLE_STAT_ENSEMBLE_FLAG_RANK - - ENSEMBLE_STAT_NC_ORANK_FLAG_RANK - * - ENSEMBLE_STAT_ENSEMBLE_FLAG_WEIGHT - - ENSEMBLE_STAT_NC_ORANK_FLAG_WEIGHT - * - ENSEMBLE_STAT_ENSEMBLE_FLAG_VLD_COUNT - - ENSEMBLE_STAT_NC_ORANK_FLAG_VLD_COUNT - -Remove Variables -"""""""""""""""" - -.. list-table:: - :widths: 50 - - * - All ENS_VAR_* variables - * - All ENSEMBLE_STAT_ENSEMBLE_FLAG_* variables - * - ENSEMBLE_STAT_NBRHD_PROB_WIDTH - * - ENSEMBLE_STAT_NBRHD_PROB_SHAPE - * - ENSEMBLE_STAT_NBRHD_PROB_VLD_THRESH - * - ENSEMBLE_STAT_NMEP_SMOOTH_VLD_THRESH - * - ENSEMBLE_STAT_NMEP_SMOOTH_SHAPE - * - ENSEMBLE_STAT_NMEP_SMOOTH_METHOD - * - ENSEMBLE_STAT_NMEP_SMOOTH_WIDTH - * - ENSEMBLE_STAT_NMEP_SMOOTH_GAUSSIAN_DX - * - ENSEMBLE_STAT_NMEP_SMOOTH_GAUSSIAN_RADIUS - -.. _upgrade-ensemble-case3: - -Case 3: EnsembleStat generating ensemble products and performing ensemble verification -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Add GenEnsProd to PROCESS_LIST -"""""""""""""""""""""""""""""" - -GenEnsProd will need to be added to the PROCESS_LIST in addition to -EnsembleStat to generate the ensemble verification output. - - .. code-block:: ini - - PROCESS_LIST = ..., EnsembleStat, GenEnsProd, ... - -Set input variables -""""""""""""""""""" - -Set the input dir and template variables for **GenEnsProd** to match -the values set for FCST input to EnsembleStat. -Also set the output dir to match EnsembleStat output dir. - - .. code-block:: ini - - GEN_ENS_PROD_INPUT_DIR = {FCST_ENSEMBLE_STAT_INPUT_DIR} - GEN_ENS_PROD_INPUT_TEMPLATE = {FCST_ENSEMBLE_STAT_INPUT_TEMPLATE} - GEN_ENS_PROD_OUTPUT_DIR = {ENSEMBLE_STAT_OUTPUT_DIR} - -Set GenEnsProd output template to include filename -"""""""""""""""""""""""""""""""""""""""""""""""""" - -* **If the EnsembleStat output template is set**, then copy the value and add a - template for the NetCDF output filename at the end following a forward slash - ‘/’ character. - - For example, if - - .. code-block:: ini - - ENSEMBLE_STAT_OUTPUT_TEMPLATE = {valid?fmt=%Y%m%d%H} - - then set - - .. code-block:: ini - - GEN_ENS_PROD_OUTPUT_TEMPLATE = {valid?fmt=%Y%m%d%H}/gen_ens_prod_{valid?fmt=%Y%m%d_%H%M%S}V_ens.nc - - or something similar. - -* **If the EnsembleStat output template is not set,** then set GenEnsProd’s - template to the desired NetCDF output filename. Here is an example: - - .. code-block:: ini - - GEN_ENS_PROD_OUTPUT_TEMPLATE = gen_ens_prod_{valid?fmt=%Y%m%d_%H%M%S}V_ens.nc - -**Ensure that any downstream wrappers in the PROCESS_LIST are configured -to read the correct GenEnsProd output file instead of the _ens.nc file -that was previously generated by EnsembleStat.** - -Handle ENS_VAR variables -"""""""""""""""""""""""" - -**If ENS_VAR_\* variables are already set,** this section can be skipped. - -**If ENS_VAR_\* variables are not set,** add ENS_VAR variables. - -* If FCST_ENSEMBLE_STAT_VAR_\* variables are set, - set the ENS_VAR_\* values to the same values. - - For example: - - .. code-block:: ini - - ENS_VAR1_NAME = {FCST_ENSEMBLE_STAT_VAR1_NAME} - ENS_VAR1_LEVELS = {FCST_ENSEMBLE_STAT_VAR1_LEVELS} - ENS_VAR2_NAME = {FCST_ENSEMBLE_STAT_VAR2_NAME} - ENS_VAR2_LEVELS = {FCST_ENSEMBLE_STAT_VAR2_LEVELS} - -* If FCST_ENSEMBLE_STAT_VAR_\* variables are **not** set, - set the ENS_VAR_\* values to the values set for the FCST_VAR_\*. - - For example: - - .. code-block:: ini - - ENS_VAR1_NAME = {FCST_VAR1_NAME} - ENS_VAR1_LEVELS = {FCST_VAR1_LEVELS} - ENS_VAR2_NAME = {FCST_VAR2_NAME} - ENS_VAR2_LEVELS = {FCST_VAR2_LEVELS} - -Set GenEnsProd Variables -"""""""""""""""""""""""" - -**If any of the following ENSEMBLE_STAT_\* variables are set in the -configuration file, then rename them to the corresponding -GEN_ENS_PROD_\* variable. -These are no longer valid settings for EnsembleStat.** - -.. list-table:: - :widths: 50 50 - :header-rows: 1 - - * - Old Name - - New Name - * - ENSEMBLE_STAT_NBRHD_PROB_WIDTH - - GEN_ENS_PROD_NBRHD_PROB_WIDTH - * - ENSEMBLE_STAT_NBRHD_PROB_SHAPE - - GEN_ENS_PROD_NBRHD_PROB_SHAPE - * - ENSEMBLE_STAT_NBRHD_PROB_VLD_THRESH - - GEN_ENS_PROD_NBRHD_PROB_VLD_THRESH - * - ENSEMBLE_STAT_NMEP_SMOOTH_VLD_THRESH - - GEN_ENS_PROD_NMEP_SMOOTH_VLD_THRESH - * - ENSEMBLE_STAT_NMEP_SMOOTH_SHAPE - - GEN_ENS_PROD_NMEP_SMOOTH_SHAPE - * - ENSEMBLE_STAT_NMEP_SMOOTH_METHOD - - GEN_ENS_PROD_NMEP_SMOOTH_METHOD - * - ENSEMBLE_STAT_NMEP_SMOOTH_WIDTH - - GEN_ENS_PROD_NMEP_SMOOTH_WIDTH - * - ENSEMBLE_STAT_NMEP_SMOOTH_GAUSSIAN_DX - - GEN_ENS_PROD_NMEP_SMOOTH_GAUSSIAN_DX - * - ENSEMBLE_STAT_NMEP_SMOOTH_GAUSSIAN_RADIUS - - GEN_ENS_PROD_NMEP_SMOOTH_GAUSSIAN_RADIUS - -**If any of the following ENSEMBLE_STAT_\* variables are set in the -configuration file, then set the corresponding GEN_ENS_PROD_\* -variables to the same value or reference the ENSEMBLE_STAT_\* version.** - -.. list-table:: - :widths: 50 - - * - ENSEMBLE_STAT_N_MEMBERS - * - ENSEMBLE_STAT_ENS_THRESH - * - ENSEMBLE_STAT_REGRID_TO_GRID - * - ENSEMBLE_STAT_REGRID_METHOD - * - ENSEMBLE_STAT_REGRID_WIDTH - * - ENSEMBLE_STAT_REGRID_VLD_THRESH - * - ENSEMBLE_STAT_REGRID_SHAPE - * - FCST_ENSEMBLE_STAT_INPUT_GRID_DATATYPE - -Example: - - .. code-block:: ini - - GEN_ENS_PROD_N_MEMBERS = {ENSEMBLE_STAT_N_MEMBERS} - GEN_ENS_PROD_ENS_THRESH = {ENSEMBLE_STAT_ENS_THRESH} - GEN_ENS_PROD_REGRID_TO_GRID = {ENSEMBLE_STAT_REGRID_TO_GRID} - GEN_ENS_PROD_REGRID_METHOD = {ENSEMBLE_STAT_REGRID_METHOD} - GEN_ENS_PROD_REGRID_WIDTH = {ENSEMBLE_STAT_REGRID_WIDTH} - GEN_ENS_PROD_REGRID_VLD_THRESH = {ENSEMBLE_STAT_REGRID_VLD_THRESH} - GEN_ENS_PROD_REGRID_SHAPE = {ENSEMBLE_STAT_REGRID_SHAPE} - GEN_ENS_PROD_INPUT_DATATYPE = {FCST_ENSEMBLE_STAT_INPUT_GRID_DATATYPE} - -**If any of the following ENSEMBLE_STAT_ENSEMBLE_FLAG_\* variables are set -in the configuration file, then rename them to the corresponding -ENSEMBLE_STAT_NC_ORANK_FLAG_\* variables AND add the corresponding -GEN_ENS_PROD_ENSEMBLE_FLAG_\* variables with the same value.** - -.. list-table:: - :widths: 50 - - * - ENSEMBLE_STAT_ENSEMBLE_FLAG_LATLON - * - ENSEMBLE_STAT_ENSEMBLE_FLAG_MEAN - * - ENSEMBLE_STAT_ENSEMBLE_FLAG_VLD_COUNT - -For example, if - - .. code-block:: ini - - ENSEMBLE_STAT_ENSEMBLE_FLAG_LATLON = TRUE - -then remove it and set - - .. code-block:: ini - - ENSEMBLE_STAT_NC_ORANK_FLAG_LATLON = TRUE - GEN_ENS_PROD_ENSEMBLE_FLAG_LATLON = TRUE - -Another example, if - - .. code-block:: ini - - ENSEMBLE_STAT_ENSEMBLE_FLAG_MEAN = FALSE - -then remove it and set - - .. code-block:: ini - - ENSEMBLE_STAT_NC_ORANK_FLAG_MEAN = FALSE - GEN_ENS_PROD_ENSEMBLE_FLAG_MEAN = FALSE - -Rename Variables -"""""""""""""""" - -.. list-table:: - :widths: 50 50 - :header-rows: 1 - - * - Old Name - - New Name - * - ENSEMBLE_STAT_ENSEMBLE_FLAG_STDEV - - GEN_ENS_PROD_ENSEMBLE_FLAG_STDEV - * - ENSEMBLE_STAT_ENSEMBLE_FLAG_MINUS - - GEN_ENS_PROD_ENSEMBLE_FLAG_MINUS - * - ENSEMBLE_STAT_ENSEMBLE_FLAG_PLUS - - GEN_ENS_PROD_ENSEMBLE_FLAG_PLUS - * - ENSEMBLE_STAT_ENSEMBLE_FLAG_MIN - - GEN_ENS_PROD_ENSEMBLE_FLAG_MIN - * - ENSEMBLE_STAT_ENSEMBLE_FLAG_MAX - - GEN_ENS_PROD_ENSEMBLE_FLAG_MAX - * - ENSEMBLE_STAT_ENSEMBLE_FLAG_RANGE - - GEN_ENS_PROD_ENSEMBLE_FLAG_RANGE - * - ENSEMBLE_STAT_ENSEMBLE_FLAG_FREQUENCY - - GEN_ENS_PROD_ENSEMBLE_FLAG_FREQUENCY - * - ENSEMBLE_STAT_ENSEMBLE_FLAG_NEP - - GEN_ENS_PROD_ENSEMBLE_FLAG_NEP - * - ENSEMBLE_STAT_ENSEMBLE_FLAG_NMEP - - GEN_ENS_PROD_ENSEMBLE_FLAG_NMEP +No upgrade instructions are needed for this release. -For further assistance, please navigate to the -`METplus Discussions `_ page. +**TODO: Add info about switching to using provided wrapped MET config file instead of user-defined.** diff --git a/metplus/VERSION b/metplus/VERSION index 8fb565199c..9d9136f3d9 100644 --- a/metplus/VERSION +++ b/metplus/VERSION @@ -1 +1 @@ -6.0.0-beta1-dev +6.0.0-beta2-dev