Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[develop] Reorganize verification tasks (rename tasks and files; split tasks into two) #618

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
7b2ae3b
Specify location of staged forecast output for vx in the machine file…
gsketefian Feb 9, 2023
3b8c582
Merge branch 'develop' into feature/vx_stage_fcst_by_platform
gsketefian Feb 13, 2023
c6bcff0
Undo unnecessary changes in a previous commit.
gsketefian Feb 13, 2023
1cf3266
Add experiment variable TEST_VX_FCST_INPUT_BASEDIR that specifies loc…
gsketefian Feb 14, 2023
ea0a460
Change condition for setting default value of VX_FCST_INPUT_BASEDIR s…
gsketefian Feb 14, 2023
4653165
Merge branch 'develop' into feature/vx_stage_fcst_by_platform
gsketefian Feb 14, 2023
771092d
Ensure that there is a vx task entry defined for each vx task and tha…
gsketefian Feb 8, 2023
dfd1122
Change vx task names to better match final ones we want.
gsketefian Feb 3, 2023
1ba6e28
Modify the way the metplus log files are named to be in line with wha…
gsketefian Feb 16, 2023
8202526
Merge branch 'develop' into feature/rename_vx_tasks
gsketefian Feb 17, 2023
da84e39
Spacing edits.
gsketefian Feb 17, 2023
4aa513d
Merge branch 'develop' into feature/rename_vx_tasks
gsketefian Feb 17, 2023
f734eb1
Change names of METplus log files so that tool names are camel case.
gsketefian Feb 17, 2023
5c00438
Rename ex-scripts for vx tasks so that they're all lower case (as req…
gsketefian Feb 23, 2023
3306a03
Remove extra slashes before ${SLASH_ENSMEM_SUBDIR}.
gsketefian Feb 24, 2023
d9e425d
Bring in changes from another branch for splitting of SFC and UPA tas…
gsketefian Feb 23, 2023
6522c5b
Rename METplus configuration files for surface and upper air.
gsketefian Feb 24, 2023
01eeeda
Change names of METplus conf and log files to follow a more consisten…
gsketefian Feb 25, 2023
df84243
Merge branch 'develop' into feature/rename_vx_tasks
gsketefian Feb 27, 2023
6c05450
In the ex-scripts for deterministic PointStat verification and Ensmeb…
gsketefian Feb 28, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ on gridded fields by initialization time for all forecast hours.
#
#-----------------------------------------------------------------------
#
$SCRIPTSdir/exregional_run_vx_ensgrid.sh || \
$SCRIPTSdir/exregional_run_met_ensemblestat_vx_grid.sh || \
print_err_msg_exit "\
Call to ex-script corresponding to J-job \"${scrfunc_fn}\" failed."
#
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ by initialitation time for all forecast hours.
#
#-----------------------------------------------------------------------
#
$SCRIPTSdir/exregional_run_vx_enspoint.sh || \
$SCRIPTSdir/exregional_run_met_ensemblestat_vx_point.sh || \
print_err_msg_exit "\
Call to ex-script corresponding to J-job \"${scrfunc_fn}\" failed."
#
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ by initialization time for all forecast hours.
#
#-----------------------------------------------------------------------
#
$SCRIPTSdir/exregional_run_vx_gridstat.sh || \
$SCRIPTSdir/exregional_run_met_gridstat_vx.sh || \
print_err_msg_exit "\
Call to ex-script corresponding to J-job \"${scrfunc_fn}\" failed."
#
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ on gridded fields by initialization time for all forecast hours.
#
#-----------------------------------------------------------------------
#
$SCRIPTSdir/exregional_run_vx_ensgrid_mean.sh || \
$SCRIPTSdir/exregional_run_met_gridstat_vx_ensmean.sh || \
print_err_msg_exit "\
Call to ex-script corresponding to J-job \"${scrfunc_fn}\" failed."
#
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ on gridded fields by initialization time for all forecast hours.
#
#-----------------------------------------------------------------------
#
$SCRIPTSdir/exregional_run_vx_ensgrid_prob.sh || \
$SCRIPTSdir/exregional_run_met_gridstat_vx_ensprob.sh || \
print_err_msg_exit "\
Call to ex-script corresponding to J-job \"${scrfunc_fn}\" failed."
#
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ by initialitation time for all forecast hours.
#
#-----------------------------------------------------------------------
#
$SCRIPTSdir/exregional_run_vx_pointstat.sh || \
$SCRIPTSdir/exregional_run_met_pointstat_vx.sh || \
print_err_msg_exit "\
Call to ex-script corresponding to J-job \"${scrfunc_fn}\" failed."
#
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ by initialitation time for all forecast hours.
#
#-----------------------------------------------------------------------
#
$SCRIPTSdir/exregional_run_vx_enspoint_mean.sh || \
$SCRIPTSdir/exregional_run_met_pointstat_vx_ensmean.sh || \
print_err_msg_exit "\
Call to ex-script corresponding to J-job \"${scrfunc_fn}\" failed."
#
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ by initialitation time for all forecast hours.
#
#-----------------------------------------------------------------------
#
$SCRIPTSdir/exregional_run_vx_enspoint_prob.sh || \
$SCRIPTSdir/exregional_run_met_pointstat_vx_ensprob.sh || \
print_err_msg_exit "\
Call to ex-script corresponding to J-job \"${scrfunc_fn}\" failed."
#
Expand Down
930 changes: 548 additions & 382 deletions parm/FV3LAM_wflow.xml

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
Expand Up @@ -347,7 +347,7 @@ ENSEMBLE_STAT_CLIMO_STDEV_INPUT_TEMPLATE =
ENSEMBLE_STAT_OUTPUT_TEMPLATE = {init?fmt=%Y%m%d%H}/metprd/ensemble_stat

# Specify the name of the metplus.log file
LOG_METPLUS = {LOG_DIR}/metplus.log.{ENV[LOG_SUFFIX]}_sfc
LOG_METPLUS = {LOG_DIR}/metplus.log.{ENV[LOG_SUFFIX]}_SFC_{ENV[CDATE]}

# Specify where the location and name of the final metplus_final.conf
METPLUS_CONF={OUTPUT_BASE}/{ENV[CDATE]}/metprd/ensemble_stat/metplus_final.conus_surface.conf
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -417,7 +417,7 @@ ENSEMBLE_STAT_CLIMO_STDEV_INPUT_TEMPLATE =
ENSEMBLE_STAT_OUTPUT_TEMPLATE = {init?fmt=%Y%m%d%H}/metprd/ensemble_stat

# Specify the name of the metplus.log file
LOG_METPLUS = {LOG_DIR}/metplus.log.{ENV[LOG_SUFFIX]}_upa
LOG_METPLUS = {LOG_DIR}/metplus.log.{ENV[LOG_SUFFIX]}_UPA_{ENV[CDATE]}

# Specify where the location and name of the final metplus_final.conf
METPLUS_CONF={OUTPUT_BASE}/{ENV[CDATE]}/metprd/ensemble_stat/metplus_final.upper_air.conf
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -292,7 +292,7 @@ POINT_STAT_CLIMO_STDEV_INPUT_TEMPLATE =
POINT_STAT_OUTPUT_TEMPLATE = metprd/point_stat

# Specify the name of the metplus.log file
LOG_METPLUS = {LOG_DIR}/metplus.log.{ENV[LOG_SUFFIX]}_sfc
LOG_METPLUS = {LOG_DIR}/metplus.log.{ENV[LOG_SUFFIX]}_SFC{ENV[USCORE_ENSMEM_NAME_OR_NULL]}_{ENV[CDATE]}

# Specify where the location and name of the final metplus_final.conf
METPLUS_CONF={OUTPUT_BASE}/metprd/point_stat/metplus_final.conus_surface{ENV[DOT_ENSMEM]}.conf
Original file line number Diff line number Diff line change
Expand Up @@ -298,7 +298,7 @@ POINT_STAT_CLIMO_STDEV_INPUT_TEMPLATE =
POINT_STAT_OUTPUT_TEMPLATE = metprd/point_stat

# Specify the name of the metplus.log file
LOG_METPLUS = {LOG_DIR}/metplus.log.{ENV[LOG_SUFFIX]}_upa
LOG_METPLUS = {LOG_DIR}/metplus.log.{ENV[LOG_SUFFIX]}_UPA{ENV[USCORE_ENSMEM_NAME_OR_NULL]}_{ENV[CDATE]}

# Specify where the location and name of the final metplus_final.conf
METPLUS_CONF={OUTPUT_BASE}/metprd/point_stat/metplus_final.upper_air{ENV[DOT_ENSMEM]}.conf
Original file line number Diff line number Diff line change
Expand Up @@ -263,7 +263,7 @@ POINT_STAT_CLIMO_STDEV_INPUT_TEMPLATE =
POINT_STAT_OUTPUT_TEMPLATE = {init?fmt=%Y%m%d%H}/metprd/ensemble_stat_mean

# Specify the name of the metplus.log file
LOG_METPLUS = {LOG_DIR}/metplus.log.{ENV[LOG_SUFFIX]}_sfc
LOG_METPLUS = {LOG_DIR}/metplus.log.{ENV[LOG_SUFFIX]}_ensmean_SFC_{ENV[CDATE]}

# Specify where the location and name of the final metplus_final.conf
METPLUS_CONF={OUTPUT_BASE}/{ENV[CDATE]}/metprd/ensemble_stat_mean/metplus_final.conus_surface.conf
Original file line number Diff line number Diff line change
Expand Up @@ -339,7 +339,7 @@ POINT_STAT_CLIMO_STDEV_INPUT_TEMPLATE =
POINT_STAT_OUTPUT_TEMPLATE = {init?fmt=%Y%m%d%H}/metprd/ensemble_stat_mean

# Specify the name of the metplus.log file
LOG_METPLUS = {LOG_DIR}/metplus.log.{ENV[LOG_SUFFIX]}_upa
LOG_METPLUS = {LOG_DIR}/metplus.log.{ENV[LOG_SUFFIX]}_ensmean_UPA_{ENV[CDATE]}

# Specify where the location and name of the final metplus_final.conf
METPLUS_CONF={OUTPUT_BASE}/{ENV[CDATE]}/metprd/ensemble_stat_mean/metplus_final.upper_air.conf
Original file line number Diff line number Diff line change
Expand Up @@ -439,7 +439,7 @@ POINT_STAT_CLIMO_STDEV_INPUT_TEMPLATE =
POINT_STAT_OUTPUT_TEMPLATE = {init?fmt=%Y%m%d%H}/metprd/ensemble_stat_prob

# Specify the name of the metplus.log file
LOG_METPLUS = {LOG_DIR}/metplus.log.{ENV[LOG_SUFFIX]}_sfc
LOG_METPLUS = {LOG_DIR}/metplus.log.{ENV[LOG_SUFFIX]}_ensprob_SFC_{ENV[CDATE]}

# Specify where the location and name of the final metplus_final.conf
METPLUS_CONF={OUTPUT_BASE}/{ENV[CDATE]}/metprd/ensemble_stat_prob/metplus_final.conus_surface.conf
Original file line number Diff line number Diff line change
Expand Up @@ -563,7 +563,7 @@ POINT_STAT_CLIMO_STDEV_INPUT_TEMPLATE =
POINT_STAT_OUTPUT_TEMPLATE = {init?fmt=%Y%m%d%H}/metprd/ensemble_stat_prob

# Specify the name of the metplus.log file
LOG_METPLUS = {LOG_DIR}/metplus.log.{ENV[LOG_SUFFIX]}_upa
LOG_METPLUS = {LOG_DIR}/metplus.log.{ENV[LOG_SUFFIX]}_ensprob_UPA_{ENV[CDATE]}

# Specify where the location and name of the final metplus_final.conf
METPLUS_CONF={OUTPUT_BASE}/{ENV[CDATE]}/metprd/ensemble_stat_prob/metplus_final.upper_air.conf
Original file line number Diff line number Diff line change
Expand Up @@ -117,9 +117,9 @@ export DOT_ENSMEM=${dot_ensmem}
#

if [ ${VAR} == "APCP" ]; then
LOG_SUFFIX=ensgrid_${CDATE}_${VAR}_${ACCUM}h
LOG_SUFFIX=EnsembleStat_${VAR}${ACCUM}h_${CDATE}
else
LOG_SUFFIX=ensgrid_${CDATE}_${VAR}
LOG_SUFFIX=EnsembleStat_${VAR}_${CDATE}
fi

#
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -111,7 +111,7 @@ export DOT_ENSMEM=${dot_ensmem}
#
#-----------------------------------------------------------------------
#
LOG_SUFFIX=enspoint_${CDATE}
LOG_SUFFIX="EnsembleStat"

#
#-----------------------------------------------------------------------
Expand Down Expand Up @@ -145,11 +145,7 @@ export NUM_ENS_MEMBERS

${METPLUS_PATH}/ush/run_metplus.py \
-c ${METPLUS_CONF}/common.conf \
-c ${METPLUS_CONF}/EnsembleStat_conus_sfc.conf

${METPLUS_PATH}/ush/run_metplus.py \
-c ${METPLUS_CONF}/common.conf \
-c ${METPLUS_CONF}/EnsembleStat_upper_air.conf
-c ${METPLUS_CONF}/EnsembleStat_${VAR}.conf

#
#-----------------------------------------------------------------------
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -88,8 +88,8 @@ else
export INPUT_BASE=${VX_FCST_INPUT_BASEDIR}/${CDATE}/postprd
export OUTPUT_BASE=${EXPTDIR}/${CDATE}
else
export INPUT_BASE=${VX_FCST_INPUT_BASEDIR}/${CDATE}/${SLASH_ENSMEM_SUBDIR}/postprd
export OUTPUT_BASE=${EXPTDIR}/${CDATE}/${SLASH_ENSMEM_SUBDIR}
export INPUT_BASE=${VX_FCST_INPUT_BASEDIR}/${CDATE}${SLASH_ENSMEM_SUBDIR}/postprd
export OUTPUT_BASE=${EXPTDIR}/${CDATE}${SLASH_ENSMEM_SUBDIR}
fi
export MEM_BASE=$EXPTDIR/$CDATE
export LOG_DIR=${EXPTDIR}/log
Expand All @@ -108,20 +108,15 @@ export DOT_ENSMEM=${dot_ensmem}
#
#-----------------------------------------------------------------------
#
if [[ ${DO_ENSEMBLE} == "FALSE" ]]; then
if [ ${VAR} == "APCP" ]; then
LOG_SUFFIX=gridstat_${CDATE}_${VAR}_${ACCUM}h
else
LOG_SUFFIX=gridstat_${CDATE}_${VAR}
fi
elif [[ ${DO_ENSEMBLE} == "TRUE" ]]; then
if [ ${VAR} == "APCP" ]; then
LOG_SUFFIX=GridStat_${VAR}${ACCUM}h${USCORE_ENSMEM_NAME_OR_NULL}_${CDATE}
else
LOG_SUFFIX=GridStat_${VAR}${USCORE_ENSMEM_NAME_OR_NULL}_${CDATE}
fi

if [[ ${DO_ENSEMBLE} == "TRUE" ]]; then
ENSMEM=`echo ${SLASH_ENSMEM_SUBDIR} | cut -d"/" -f2`
VX_FCST_MODEL_NAME=${VX_FCST_MODEL_NAME}_${ENSMEM}
if [ ${VAR} == "APCP" ]; then
LOG_SUFFIX=gridstat_${CDATE}_${ENSMEM}_${VAR}_${ACCUM}h
else
LOG_SUFFIX=gridstat_${CDATE}_${ENSMEM}_${VAR}
fi
fi

#
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -114,9 +114,9 @@ export DOT_ENSMEM=${dot_ensmem}
#

if [ ${VAR} == "APCP" ]; then
LOG_SUFFIX=ensgrid_mean_${CDATE}_${VAR}_${ACCUM}h
LOG_SUFFIX=GridStat_ensmean_${VAR}${ACCUM}h_${CDATE}
else
LOG_SUFFIX=ensgrid_mean_${CDATE}_${VAR}
LOG_SUFFIX=GridStat_ensmean_${VAR}_${CDATE}
fi

#
Expand Down Expand Up @@ -159,11 +159,11 @@ if [ ${VAR} == "APCP" ]; then
export acc="${ACCUM}h"
${METPLUS_PATH}/ush/run_metplus.py \
-c ${METPLUS_CONF}/common.conf \
-c ${METPLUS_CONF}/GridStat_${VAR}${acc}_mean.conf
-c ${METPLUS_CONF}/GridStat_ensmean_${VAR}${acc}.conf
else
${METPLUS_PATH}/ush/run_metplus.py \
-c ${METPLUS_CONF}/common.conf \
-c ${METPLUS_CONF}/GridStat_${VAR}_mean.conf
-c ${METPLUS_CONF}/GridStat_ensmean_${VAR}.conf
fi

#
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -114,9 +114,9 @@ export DOT_ENSMEM=${dot_ensmem}
#

if [ ${VAR} == "APCP" ]; then
LOG_SUFFIX=ensgrid_prob_${CDATE}_${VAR}_${ACCUM}h
LOG_SUFFIX=GridStat_ensprob_${VAR}${ACCUM}h_${CDATE}
else
LOG_SUFFIX=ensgrid_prob_${CDATE}_${VAR}
LOG_SUFFIX=GridStat_ensprob_${VAR}_${CDATE}
fi

#
Expand Down Expand Up @@ -159,11 +159,11 @@ if [ ${VAR} == "APCP" ]; then
export acc="${ACCUM}h"
${METPLUS_PATH}/ush/run_metplus.py \
-c ${METPLUS_CONF}/common.conf \
-c ${METPLUS_CONF}/GridStat_${VAR}${acc}_prob.conf
-c ${METPLUS_CONF}/GridStat_ensprob_${VAR}${acc}.conf
else
${METPLUS_PATH}/ush/run_metplus.py \
-c ${METPLUS_CONF}/common.conf \
-c ${METPLUS_CONF}/GridStat_${VAR}_prob.conf
-c ${METPLUS_CONF}/GridStat_ensprob_${VAR}.conf
fi

#
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -88,8 +88,8 @@ else
export INPUT_BASE=${VX_FCST_INPUT_BASEDIR}/${CDATE}/postprd
export OUTPUT_BASE=${EXPTDIR}/${CDATE}
else
export INPUT_BASE=${VX_FCST_INPUT_BASEDIR}/${CDATE}/${SLASH_ENSMEM_SUBDIR}/postprd
export OUTPUT_BASE=${EXPTDIR}/${CDATE}/${SLASH_ENSMEM_SUBDIR}
export INPUT_BASE=${VX_FCST_INPUT_BASEDIR}/${CDATE}${SLASH_ENSMEM_SUBDIR}/postprd
export OUTPUT_BASE=${EXPTDIR}/${CDATE}${SLASH_ENSMEM_SUBDIR}
fi
export MEM_BASE=$EXPTDIR/$CDATE
export LOG_DIR=${EXPTDIR}/log
Expand All @@ -108,12 +108,10 @@ export DOT_ENSMEM=${dot_ensmem}
#
#-----------------------------------------------------------------------
#
if [[ ${DO_ENSEMBLE} == "FALSE" ]]; then
LOG_SUFFIX=pointstat_${CDATE}
elif [[ ${DO_ENSEMBLE} == "TRUE" ]]; then
LOG_SUFFIX="PointStat"
if [[ ${DO_ENSEMBLE} == "TRUE" ]]; then
ENSMEM=`echo ${SLASH_ENSMEM_SUBDIR} | cut -d"/" -f2`
VX_FCST_MODEL_NAME=${VX_FCST_MODEL_NAME}_${ENSMEM}
LOG_SUFFIX=pointstat_${CDATE}_${ENSMEM}
fi

#
Expand Down Expand Up @@ -147,11 +145,7 @@ export POST_OUTPUT_DOMAIN_NAME

${METPLUS_PATH}/ush/run_metplus.py \
-c ${METPLUS_CONF}/common.conf \
-c ${METPLUS_CONF}/PointStat_conus_sfc.conf

${METPLUS_PATH}/ush/run_metplus.py \
-c ${METPLUS_CONF}/common.conf \
-c ${METPLUS_CONF}/PointStat_upper_air.conf
-c ${METPLUS_CONF}/PointStat_${VAR}.conf

#
#-----------------------------------------------------------------------
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -112,7 +112,7 @@ export DOT_ENSMEM=${dot_ensmem}
#
#-----------------------------------------------------------------------
#
LOG_SUFFIX=enspoint_mean_${CDATE}
LOG_SUFFIX="PointStat"

#
#-----------------------------------------------------------------------
Expand Down Expand Up @@ -145,11 +145,7 @@ export POST_OUTPUT_DOMAIN_NAME

${METPLUS_PATH}/ush/run_metplus.py \
-c ${METPLUS_CONF}/common.conf \
-c ${METPLUS_CONF}/PointStat_conus_sfc_mean.conf

${METPLUS_PATH}/ush/run_metplus.py \
-c ${METPLUS_CONF}/common.conf \
-c ${METPLUS_CONF}/PointStat_upper_air_mean.conf
-c ${METPLUS_CONF}/PointStat_ensmean_${VAR}.conf
#
#-----------------------------------------------------------------------
#
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -112,8 +112,7 @@ export DOT_ENSMEM=${dot_ensmem}
#
#-----------------------------------------------------------------------
#
LOG_SUFFIX=enspoint_prob_${CDATE}

LOG_SUFFIX="PointStat"
#
#-----------------------------------------------------------------------
#
Expand Down Expand Up @@ -145,11 +144,7 @@ export POST_OUTPUT_DOMAIN_NAME

${METPLUS_PATH}/ush/run_metplus.py \
-c ${METPLUS_CONF}/common.conf \
-c ${METPLUS_CONF}/PointStat_conus_sfc_prob.conf

${METPLUS_PATH}/ush/run_metplus.py \
-c ${METPLUS_CONF}/common.conf \
-c ${METPLUS_CONF}/PointStat_upper_air_prob.conf
-c ${METPLUS_CONF}/PointStat_ensprob_${VAR}.conf
#
#-----------------------------------------------------------------------
#
Expand Down
Loading