-
Notifications
You must be signed in to change notification settings - Fork 37
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Feature 1516 use case mvmode #2094
Merged
Merged
Changes from 8 commits
Commits
Show all changes
11 commits
Select commit
Hold shift + click to select a range
3b3df5e
Add multivariate MODE use-case
hertneky 14c4149
Merge remote-tracking branch 'upstream/develop' into feature_1516_use…
0cd460a
documentation formatting
hertneky f5e6601
line block fix
hertneky 3829811
bullet formatting
hertneky ef1b27d
turn on test
hertneky a14805f
turn off test
hertneky efbcb60
Merge remote-tracking branch 'upstream/develop' into feature_1516_use…
hertneky 97d462b
Merge branch 'develop' into feature_1516_useCase_mvmode
georgemccabe e68e84c
clean up formatting in use case doc and removed commented config opti…
georgemccabe cd9305c
add link to section
georgemccabe File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
206 changes: 206 additions & 0 deletions
206
docs/use_cases/model_applications/short_range/MODEMultivar_fcstHRRR_obsMRMS_HRRRanl.py
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,206 @@ | ||
""" | ||
MODE: Multivariate | ||
========================================================================= | ||
|
||
model_applications/ | ||
short_range/ | ||
MODEMultivar_fcstHRRR_obsMRMS_HRRRanl.conf | ||
|
||
""" | ||
############################################################################## | ||
# Scientific Objective | ||
# -------------------- | ||
# | ||
# This use case demonstrates how to run Multivariate MODE to identify complex | ||
# objects from two or more fields, defined by a logical expression. This use | ||
# case identifies blizzard-like objects defined by: 1) the presence of snow | ||
# precipitation type, 2) 10-m winds > 20 mph, and 3) visibility < 1/2 mile. | ||
# The use of multivariate MODE is well-suited to assess the structure and | ||
# placement of complex high-impact events such as blizzard conditions and heavy | ||
# snow bands. Output from this use-case consists of the MODE forecast and observation | ||
# super objects and the MODE ASCII, NetCDF, and PostScript files. | ||
# | ||
|
||
############################################################################## | ||
# Datasets | ||
# -------- | ||
# | ||
# | **Forecast dataset:** 1-hour HRRR in grib2 | ||
# | **Observation dataset:** MRMS and HRRR analysis in grib2 | ||
# | ||
# The forecast and observation fields are only a subset of the full domain in | ||
# order for a faster run-time of Multivariate MODE. An example command using | ||
# wgrib2 to create the HRRR subdomain is:: | ||
# | ||
# wgrib2 infile.grib2 -new_grid_winds earth -new_grid lambert:262.5:38.5:38.5:38.5 -83.0:400:3000 37.0:400:3000 outfile.grib2 | ||
# | ||
|
||
############################################################################## | ||
# METplus Components | ||
# ------------------ | ||
# | ||
# This use case runs MODE using multiple variables to output the super objects | ||
# based on a user-defined logical expression. Currently, the initial multivariate | ||
# MODE run only outputs the super objects and additional steps are required to | ||
# produce the statistical output. GenVxMask is run on a field(s) of interest | ||
# using the super objects to mask the field(s). Finally, MODE is run a second | ||
# time on the super-object-masked field(s) to output attribute statistics for | ||
# the field(s). | ||
# | ||
# **Note:** The second MODE run can also be run directly on the super objects if | ||
# field-specific statistics, such as intensity, is not desired. | ||
# | ||
|
||
############################################################################## | ||
# METplus Workflow | ||
# ---------------- | ||
# | ||
# The following tools are used for each run time: | ||
# | ||
# MODE(mv), GenVxMask(fcst_super), GenVxMask(obs_super), MODE (super) | ||
# | ||
# Where the first instance of MODE runs over multiple variables to identify | ||
# super objects for the forecast and observation, GenVxMask masks the raw input | ||
# field(s) using the super objects, and the second instance of MODE is run | ||
# traditionally to compare the masked forecast and observed super objects and | ||
# and provide statistics. | ||
# | ||
# This example runs a single forecast hour. | ||
# | ||
# | **Initialization:** 2021020100 | ||
# | **Forecast lead:** 21 | ||
# | ||
|
||
############################################################################## | ||
# METplus Configuration | ||
# --------------------- | ||
# | ||
# METplus first loads all of the configuration files found in parm/metplus_config, | ||
# then it loads any configuration files passed to METplus via the command line | ||
# with the -c option, i.e. -c parm/use_cases/model_applications/short_range/MODEMultivar_fcstHRRR_obsMRMS_HRRRanl.conf | ||
# | ||
# .. highlight:: bash | ||
# .. literalinclude:: ../../../../parm/use_cases/model_applications/short_range/MODEMultivar_fcstHRRR_obsMRMS_HRRRanl.conf | ||
|
||
############################################################################## | ||
# MET Configuration | ||
# --------------------- | ||
# | ||
# METplus sets environment variables based on user settings in the METplus configuration file. | ||
# See :ref:`How METplus controls MET config file settings<metplus-control-met>` for more details. | ||
# | ||
# **YOU SHOULD NOT SET ANY OF THESE ENVIRONMENT VARIABLES YOURSELF! THEY WILL BE OVERWRITTEN BY METPLUS WHEN IT CALLS THE MET TOOLS!** | ||
# | ||
# If there is a setting in the MET configuration file that is currently not supported by METplus you'd like to control, please refer to: | ||
# :ref:`Overriding Unsupported MET config file settings<met-config-overrides>` | ||
# | ||
# .. note:: See the :ref:`MODE MET Configuration<mode-met-conf>` section of the User's Guide for more information on the environment variables used in the file below: | ||
# | ||
# .. highlight:: bash | ||
# .. literalinclude:: ../../../../parm/met_config/MODEConfig_wrapped | ||
|
||
############################################################################## | ||
# Running METplus | ||
# --------------- | ||
# | ||
# This use case can be run two ways: | ||
# | ||
# 1) Passing in MODEMultivar_fcstHRRR_obsMRMS_HRRRanl.conf then a user-specific system configuration file:: | ||
# | ||
# run_metplus.py -c /path/to/METplus/parm/use_cases/model_applications/short_range/MODEMultivar_fcstHRRR_obsMRMS_HRRRanl.conf -c /path/to/user_system.conf | ||
# | ||
# 2) Modifying the configurations in parm/metplus_config, then passing in MODEMultivar_fcstHRRR_obsMRMS_HRRRanl.conf:: | ||
# | ||
# run_metplus.py -c /path/to/METplus/parm/use_cases/model_applications/short_range/MODEMultivar_fcstHRRR_obsMRMS_HRRRanl.conf | ||
# | ||
# The former method is recommended. Whether you add them to a user-specific configuration file or modify the metplus_config files, the following variables must be set correctly: | ||
# | ||
# * **INPUT_BASE** - Path to directory where sample data tarballs are unpacked (See Datasets section to obtain tarballs). This is not required to run METplus, but it is required to run the examples in parm/use_cases | ||
# * **OUTPUT_BASE** - Path where METplus output will be written. This must be in a location where you have write permissions | ||
# * **MET_INSTALL_DIR** - Path to location where MET is installed locally | ||
# | ||
# Example User Configuration File:: | ||
# | ||
# [dir] | ||
# INPUT_BASE = /path/to/sample/input/data | ||
# OUTPUT_BASE = /path/to/output/dir | ||
# MET_INSTALL_DIR = /path/to/met-X.Y | ||
# | ||
# **NOTE:** All of these items must be found under the [dir] section. | ||
# | ||
|
||
############################################################################## | ||
# Expected Output | ||
# --------------- | ||
# | ||
# A successful run will output the following both to the screen and to the logfile:: | ||
# | ||
# INFO: METplus has successfully finished running. | ||
# | ||
# Refer to the value set for **OUTPUT_BASE** to find where the output data was generated. | ||
# Output for this use case will be found in OUTPUT_BASE for the various MET tools | ||
# and will contain the following files: | ||
# | ||
# **mode/2021020100/f21** | ||
# | ||
# Multivariate output - first instance | ||
# | ||
# Precipitation type = snow | ||
# | ||
# * 00/mode_210000L_20210201_210000V_000000A_cts.txt | ||
# * 00/mode_210000L_20210201_210000V_000000A_obj.nc | ||
# * 00/mode_210000L_20210201_210000V_000000A_obj.txt | ||
# * 00/mode_210000L_20210201_210000V_000000A.ps | ||
# | ||
# Visibility | ||
# | ||
# * 01/mode_210000L_20210201_210000V_000000A_cts.txt | ||
# * 01/mode_210000L_20210201_210000V_000000A_obj.nc | ||
# * 01/mode_210000L_20210201_210000V_000000A_obj.txt | ||
# * 01/mode_210000L_20210201_210000V_000000A.ps | ||
# | ||
# 10-m Winds | ||
# | ||
# * 02/mode_210000L_20210201_210000V_000000A_cts.txt | ||
# * 02/mode_210000L_20210201_210000V_000000A_obj.nc | ||
# * 02/mode_210000L_20210201_210000V_000000A_obj.txt | ||
# * 02/mode_210000L_20210201_210000V_000000A.ps | ||
# | ||
# Super Objects | ||
# | ||
# * f_super.nc | ||
# * o_super.nc | ||
# | ||
# MODE 10-m wind super object output - second instance | ||
# | ||
# * mode_HRRR_vs_ANALYSIS_WIND_super_Z10_210000L_20210201_210000V_000000A_cts.txt | ||
# * mode_HRRR_vs_ANALYSIS_WIND_super_Z10_210000L_20210201_210000V_000000A_obj.nc | ||
# * mode_HRRR_vs_ANALYSIS_WIND_super_Z10_210000L_20210201_210000V_000000A_obj.txt | ||
# * mode_HRRR_vs_ANALYSIS_WIND_super_Z10_210000L_20210201_210000V_000000A.ps | ||
# | ||
# **gen_vx_mask/2021020100** | ||
# | ||
# * fcst_wind_super_2021020100_f21.nc | ||
# * obs_wind_super_2021020121.nc | ||
|
||
############################################################################## | ||
# Keywords | ||
# -------- | ||
# | ||
# .. note:: | ||
# | ||
# * MODEToolUseCase | ||
# * GenVxMaskToolUseCase | ||
# * ShortRangeAppUseCase | ||
# * GRIB2FileUseCase | ||
# * RegriddingInToolUseCase | ||
# * NOAAWPCOrgUseCase | ||
# * NCAROrgUseCase | ||
# * DiagnosticsUseCase | ||
# | ||
# | ||
# Navigate to the :ref:`quick-search` page to discover other similar use cases. | ||
# | ||
# | ||
# sphinx_gallery_thumbnail_path = '_static/short_range-MODEMultivar_fcstHRRR_obsMRMS_HRRRanl.png' | ||
# |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
154 changes: 154 additions & 0 deletions
154
parm/use_cases/model_applications/short_range/MODEMultivar_fcstHRRR_obsMRMS_HRRRanl.conf
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,154 @@ | ||
[config] | ||
|
||
# Documentation for this use-case can be found at: | ||
# https://metplus.readthedocs.io/en/latest/generated/model_applications/short_range/MODEMultivar_fcstHRRR_obsMRMS_HRRRanl.html | ||
|
||
# Processes to run | ||
PROCESS_LIST = MODE(mv),GenVxMask(fcst_super),GenVxMask(obs_super),MODE(super) | ||
|
||
# Time Info | ||
LOOP_ORDER = times | ||
LOOP_BY = INIT | ||
|
||
INIT_TIME_FMT = %Y%m%d%H | ||
INIT_BEG = 2021020100 | ||
INIT_END = 2021020100 | ||
|
||
LEAD_SEQ = 21 | ||
|
||
MODEL = HRRR | ||
OBTYPE = ANALYSIS | ||
|
||
################################## | ||
# Multivariate MODE Configurations | ||
################################## | ||
# Run MODE to output super objects | ||
[mv] | ||
MODE_MULTIVAR_LOGIC = #1 && #2 && #3 | ||
|
||
FCST_MODE_INPUT_DIR = {INPUT_BASE}/model_applications/short_range/MODEMultivar_fcstHRRR_obsMRMS_HRRRanl | ||
FCST_MODE_INPUT_TEMPLATE = hrrr.t{init?fmt=%H}z.wrfprsf{lead?fmt=%H}.sub.grib2,hrrr.t{init?fmt=%H}z.wrfprsf{lead?fmt=%H}.sub.grib2,hrrr.t{init?fmt=%H}z.wrfprsf{lead?fmt=%H}.sub.grib2 | ||
|
||
OBS_MODE_INPUT_DIR = {INPUT_BASE}/model_applications/short_range/MODEMultivar_fcstHRRR_obsMRMS_HRRRanl | ||
OBS_MODE_INPUT_TEMPLATE = PrecipFlag_00.00_{valid?fmt=%Y%m%d}-{valid?fmt=%2H}0000.sub.grib2,hrrr.t{valid?fmt=%H}z.wrfprsf00.sub.grib2,hrrr.t{valid?fmt=%H}z.wrfprsf00.sub.grib2 | ||
|
||
MODE_OUTPUT_DIR = {OUTPUT_BASE}/mode | ||
MODE_OUTPUT_TEMPLATE = {init?fmt=%Y%m%d%H}/f{lead?fmt=%2H} | ||
|
||
FCST_VAR1_NAME = CSNOW | ||
FCST_VAR1_LEVELS = L0 | ||
FCST_VAR1_OPTIONS = conv_radius = 0; conv_thresh = ==1 | ||
|
||
OBS_VAR1_NAME = PrecipFlag | ||
OBS_VAR1_LEVELS = L0 | ||
OBS_VAR1_OPTIONS = conv_radius = 0; conv_thresh = ==3 | ||
|
||
FCST_VAR2_NAME = VIS | ||
FCST_VAR2_LEVELS = L0 | ||
FCST_VAR2_OPTIONS = conv_radius = 5; conv_thresh = <=804.672; merge_thresh = <=1207.008; merge_flag = THRESH | ||
|
||
OBS_VAR2_NAME = VIS | ||
OBS_VAR2_LEVELS = L0 | ||
OBS_VAR2_OPTIONS = conv_radius = 5; conv_thresh = <=804.672; merge_thresh = <=1207.008; merge_flag = THRESH | ||
|
||
FCST_VAR3_NAME = WIND | ||
FCST_VAR3_LEVELS = Z10 | ||
FCST_VAR3_OPTIONS = conv_radius = 5; conv_thresh = >=8.9408; merge_thresh = >=6.7056; merge_flag = THRESH | ||
|
||
OBS_VAR3_NAME = WIND | ||
OBS_VAR3_LEVELS = Z10 | ||
OBS_VAR3_OPTIONS = conv_radius = 5; conv_thresh = >=8.9408; merge_thresh = >=6.7056; merge_flag = THRESH | ||
|
||
MODE_FCST_FILTER_ATTR_NAME = AREA | ||
MODE_FCST_FILTER_ATTR_THRESH = >=25 | ||
MODE_OBS_FILTER_ATTR_NAME = AREA | ||
MODE_OBS_FILTER_ATTR_THRESH = >=25 | ||
|
||
MODE_MATCH_FLAG = MERGE_BOTH | ||
|
||
MODE_REGRID_TO_GRID = FCST | ||
MODE_REGRID_METHOD = NEAREST | ||
MODE_REGRID_WIDTH = 1 | ||
MODE_REGRID_VLD_THRESH = 0.5 | ||
|
||
########################## | ||
# GenVxMask configurations | ||
########################## | ||
# Mask fcst field with the fcst super object field | ||
[fcst_super] | ||
GEN_VX_MASK_INPUT_DIR = {INPUT_BASE}/model_applications/short_range/MODEMultivar_fcstHRRR_obsMRMS_HRRRanl | ||
GEN_VX_MASK_INPUT_TEMPLATE = hrrr.t{init?fmt=%H}z.wrfprsf{lead?fmt=%H}.sub.grib2 | ||
GEN_VX_MASK_INPUT_MASK_DIR = {OUTPUT_BASE}/mode | ||
GEN_VX_MASK_INPUT_MASK_TEMPLATE = {init?fmt=%Y%m%d%H}/f{lead?fmt=%2H}/f_super.nc | ||
GEN_VX_MASK_OPTIONS = -type data -input_field 'name="WIND";level="Z10";' -mask_field 'name="super";level="L0";' -thresh 'eq0' -value -9999 -name 'WIND_super' | ||
GEN_VX_MASK_OUTPUT_DIR = {OUTPUT_BASE}/gen_vx_mask | ||
GEN_VX_MASK_OUTPUT_TEMPLATE = {init?fmt=%Y%m%d%H}/fcst_wind_super_{init?fmt=%Y%m%d%H}_f{lead?fmt=%2H}.nc | ||
|
||
# Mask obs field with the obs super objects | ||
[obs_super] | ||
GEN_VX_MASK_INPUT_DIR = {INPUT_BASE}/model_applications/short_range/MODEMultivar_fcstHRRR_obsMRMS_HRRRanl | ||
GEN_VX_MASK_INPUT_TEMPLATE = hrrr.t{valid?fmt=%H}z.wrfprsf00.sub.grib2 | ||
GEN_VX_MASK_INPUT_MASK_DIR = {OUTPUT_BASE}/mode | ||
GEN_VX_MASK_INPUT_MASK_TEMPLATE = {init?fmt=%Y%m%d%H}/f{lead?fmt=%2H}/o_super.nc | ||
GEN_VX_MASK_OPTIONS = -type data -input_field 'name="WIND";level="Z10";' -mask_field 'name="super";level="L0";' -thresh 'eq0' -value -9999 -name 'WIND_super' | ||
GEN_VX_MASK_OUTPUT_DIR = {OUTPUT_BASE}/gen_vx_mask | ||
GEN_VX_MASK_OUTPUT_TEMPLATE = {init?fmt=%Y%m%d%H}/obs_wind_super_{valid?fmt=%Y%m%d%H}.nc | ||
|
||
####################### | ||
# MODE on precip supers | ||
####################### | ||
[super] | ||
FCST_VAR1_NAME = WIND_super | ||
FCST_VAR1_LEVELS = Z10 | ||
|
||
OBS_VAR1_NAME = WIND_super | ||
OBS_VAR1_LEVELS = Z10 | ||
|
||
MODE_CONV_RADIUS = 0 | ||
MODE_CONV_THRESH = ne-9999 | ||
MODE_MERGE_FLAG = NONE | ||
|
||
MODE_OUTPUT_PREFIX = {MODEL}_vs_{OBTYPE}_{CURRENT_OBS_NAME}_{CURRENT_OBS_LEVEL} | ||
|
||
MODE_GRID_RES = 3 | ||
|
||
#MODE_INTEREST_FUNCTION_CENTROID_DIST = | ||
#MODE_INTEREST_FUNCTION_BOUNDARY_DIST = | ||
#MODE_INTEREST_FUNCTION_CONVEX_HULL_DIST = | ||
|
||
#MODE_TOTAL_INTEREST_THRESH = | ||
|
||
#MODE_MATCH_FLAG = | ||
|
||
#MODE_WEIGHT_CENTROID_DIST = | ||
#MODE_WEIGHT_BOUNDARY_DIST = | ||
#MODE_WEIGHT_CONVEX_HULL_DIST = | ||
#MODE_WEIGHT_ANGLE_DIFF = | ||
#MODE_WEIGHT_ASPECT_DIFF = | ||
#MODE_WEIGHT_AREA_RATIO = | ||
#MODE_WEIGHT_INT_AREA_RATIO = | ||
#MODE_WEIGHT_CURVATURE_RATIO = | ||
#MODE_WEIGHT_COMPLEXITY_RATIO = | ||
#MODE_WEIGHT_INTEN_PERC_RATIO = | ||
#MODE_WEIGHT_INTEN_PERC_VALUE = | ||
|
||
MODE_NC_PAIRS_FLAG_LATLON = TRUE | ||
MODE_NC_PAIRS_FLAG_RAW = TRUE | ||
MODE_NC_PAIRS_FLAG_OBJECT_RAW = TRUE | ||
MODE_NC_PAIRS_FLAG_OBJECT_ID = TRUE | ||
MODE_NC_PAIRS_FLAG_CLUSTER_ID = TRUE | ||
MODE_NC_PAIRS_FLAG_POLYLINES = TRUE | ||
|
||
MODE_QUILT = True | ||
|
||
MODE_PS_PLOT_FLAG = TRUE | ||
MODE_CT_STATS_FLAG = TRUE | ||
|
||
FCST_MODE_INPUT_DIR = {OUTPUT_BASE}/gen_vx_mask | ||
FCST_MODE_INPUT_TEMPLATE = {init?fmt=%Y%m%d%H}/fcst_wind_super_{init?fmt=%Y%m%d%H}_f{lead?fmt=%2H}.nc | ||
|
||
OBS_MODE_INPUT_DIR = {OUTPUT_BASE}/gen_vx_mask | ||
OBS_MODE_INPUT_TEMPLATE = {init?fmt=%Y%m%d%H}/obs_wind_super_{valid?fmt=%Y%m%d%H}.nc | ||
|
||
MODE_OUTPUT_DIR = {OUTPUT_BASE}/mode | ||
MODE_OUTPUT_TEMPLATE = {init?fmt=%Y%m%d%H}/f{lead?fmt=%2H} |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The documentation for this is slightly outdated. A user no longer needs the "-c" between each configuration file. These should be removed here, as well as the second method of running the use case (just below this line).