CRPSS data are missing from output ECNT linetype records #1725
-
|
I am testing EnsembleStat verification for 30-member GEFS with single climo (just climo mean as reference) in METplus The running is successful. But checking the ensemble_stat output stat file I found the CRPSS data (#28 or #4 after ECNT) is missing (labeled as NA). [config] [user_env_vars] [runtime] |
Beta Was this translation helpful? Give feedback.
Replies: 11 comments 5 replies
-
|
More info for this case: |
Beta Was this translation helpful? Give feedback.
-
|
@BinbinZhou-NOAA I see that you're getting differing results when running Ensemble-Stat directly on the command line versus running it via METplus. I agree with your diagnosis. The likely culprit is in how the configuration is being set via METplus. I want to ask about these 2 settings from what you provided above: It would seem that the climo mean input field is EMPTY... but then you're trying to define it via the OVERRIDES option. I do recall having a conversation with @georgemccabe about the handling of OVERRIDES. He'd be the best person to advise here. But he's out of the office this week. If you have time before the METplus telecon on Monday, please try moving the climo_mean setting out of ENSEMBLE_STAT_MET_CONFIG_OVERRIDES. Instead, specify it using the ENSEMBLE_STAT_CLIMO_MEAN_FIELD setting. Does that produce any different results? |
Beta Was this translation helpful? Give feedback.
-
|
John, Our dev machine was just down, so have to wait for its back. Thanks! Binbin |
Beta Was this translation helpful? Give feedback.
-
|
I remember I have tested your suggestion before. That is, removed the ENSEMBLE_STAT_MET_CONFIG_OVERRIDES statement My setting ENSEMBLE_STAT_CLIMO_MEAN_FIELD is correct? |
Beta Was this translation helpful? Give feedback.
-
|
I re-ran the case in which ENSEMBLE_STAT_MET_CONFIG_OVERRIDES is removed and set it in ENSEMBLE_STAT_CLIMO_MEAN_FIELD, |
Beta Was this translation helpful? Give feedback.
-
|
OK, let's touch base with @georgemccabe at the NOAA METplus telecon on Monday to help debug this behavior. Thanks. |
Beta Was this translation helpful? Give feedback.
-
|
@BinbinZhou-NOAA could you send us the log file from this run with |
Beta Was this translation helpful? Give feedback.
-
|
The MET config file used when calling ensemble_stat outside of METplus would also be useful to compare. |
Beta Was this translation helpful? Give feedback.
-
|
I just sent both METplus and MET running samples to HERA in directory: MET sample: evs_gefs_MET_stats_crpss In evs_gefs_METplus_stats_crpss, in evs_gefs_MET_stats_crpss |
Beta Was this translation helpful? Give feedback.
-
|
I just sent both METplus and MET running samples to HERA in directory:
/scratch2/NCEPDEV/fv3-cam/noscrub/Binbin.Zhou
2 directories for
METPlus sample: evs_gefs_METplus_stats_crpss
MET sample: evs_gefs_MET_stats_crpss
In evs_gefs_METplus_stats_crpss,
EnsembleStat_fcstGEFS_obsModelAnalysis_climoERAI.conf is original conf file
run_gefs_valid_at_t00z_fhr1_g2g.sh defines all required environment
variables
sub-directory /grid2grid/run_gefs_valid_at_t00z_fhr1 is running output_base
directory, in which
/logs contains log files
/data contains all input files (30 member files, analysis data, c;limo data
files, etc)
in evs_gefs_MET_stats_crpss
verf_g2g_ens_stat_config_bin1 is MET config file, and run.sh is execution
script file
all data in /data sub-directory
…On Mon, Aug 8, 2022 at 11:30 AM George McCabe ***@***.***> wrote:
The MET config file used when calling ensemble_stat outside of METplus
would also be useful to compare.
—
Reply to this email directly, view it on GitHub
<#1725 (reply in thread)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AQBMPFOPKM4KJG3E44RBDJLVYERZDANCNFSM55MA7ZQA>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
--
Binbin Zhou
IMSG at NOAA/NWS/NCEP/EMC
5830 University Research Ct.
College Park, MD 20740
***@***.***
301-683-3683
|
Beta Was this translation helpful? Give feedback.
-
|
Hi @BinbinZhou-NOAA, thank you for making these files available. (1) I noticed in your MET config file that you have the following set:
This is typically used to get the field information from the fcst dictionary and use it for the climo_mean field info. There is a METplus setting to accomplish this called ENSEMBLE_STAT_CLIMO_MEAN_USE_FCST . Set this variable to True instead of setting climo_mean.field explicitly.
(2) In your METplus log file, I'm noticing that the settings for climo_mean are using some values that differ from the MET config file settings:
I see that the climo_mean.regrid dictionary is set to values that differ from the defaults via METplus. These are unset in the MET config, so the default values are used. Looking at your METplus config file (evs_gefs_METplus_stats_crpss/EnsembleStat_fcstGEFS_obsModelAnalysis_climoERAI.conf), I notice that these settings do not match what is being set in the METPLUS_CLIMO_MEAN_DICT environment variable: Is it possible that you changed these settings in the conf file since you generated the log file? If so, could you rerun METplus with |
Beta Was this translation helpful? Give feedback.
-
|
Based on your settings in the MET config file, I think you will want to set the following variables and remove the other ENSEMBLE_STAT_CLIMO_MEAN/STDEV_* variables: |
Beta Was this translation helpful? Give feedback.
-
|
Do you mean remove all of below?
ENSEMBLE_STAT_CLIMO_MEAN_REGRID_METHOD = BILIN
ENSEMBLE_STAT_CLIMO_MEAN_REGRID_WIDTH = 2
ENSEMBLE_STAT_CLIMO_MEAN_REGRID_VLD_THRESH = 0.5
ENSEMBLE_STAT_CLIMO_MEAN_REGRID_SHAPE = SQUARE
ENSEMBLE_STAT_CLIMO_MEAN_FIELD =
ENSEMBLE_STAT_CLIMO_MEAN_TIME_INTERP_METHOD = DW_MEAN
ENSEMBLE_STAT_CLIMO_MEAN_MATCH_MONTH = FALSE
ENSEMBLE_STAT_CLIMO_MEAN_DAY_INTERVAL = 31
ENSEMBLE_STAT_CLIMO_MEAN_HOUR_INTERVAL = 6
ENSEMBLE_STAT_CLIMO_MEAN_INPUT_DIR = {climpath}
ENSEMBLE_STAT_CLIMO_STDEV_INPUT_DIR = {climpath}
ENSEMBLE_STAT_CLIMO_MEAN_INPUT_TEMPLATE = mean_{valid?fmt=%m%d}
ENSEMBLE_STAT_CLIMO_STDEV_INPUT_TEMPLATE = stdev_{valid?fmt=%m%d}
and just add
ENSEMBLE_STAT_CLIMO_MEAN_FILE_NAME = {climpath}/mean_{valid?fmt=%m%d}
ENSEMBLE_STAT_CLIMO_MEAN_USE_FCST = True
ENSEMBLE_STAT_CLIMO_STDEV_FILE_NAME = {climpath}/stdev_{valid?fmt=%m%d}
ENSEMBLE_STAT_CLIMO_STDEV_USE_FCST = True
…On Mon, Aug 8, 2022 at 1:31 PM George McCabe ***@***.***> wrote:
Based on your settings in the MET config file, I think you will want to
set the following variables and remove the other
ENSEMBLE_STAT_CLIMO_MEAN/STDEV_* variables:
ENSEMBLE_STAT_CLIMO_MEAN_FILE_NAME = {climpath}/mean_{valid?fmt=%m%d}
ENSEMBLE_STAT_CLIMO_MEAN_USE_FCST = True
ENSEMBLE_STAT_CLIMO_STDEV_FILE_NAME = {climpath}/stdev_{valid?fmt=%m%d}
ENSEMBLE_STAT_CLIMO_STDEV_USE_FCST = True
—
Reply to this email directly, view it on GitHub
<#1725 (reply in thread)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AQBMPFOQL7BOOX6R4QZERCTVYE76XANCNFSM55MA7ZQA>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
--
Binbin Zhou
IMSG at NOAA/NWS/NCEP/EMC
5830 University Research Ct.
College Park, MD 20740
***@***.***
301-683-3683
|
Beta Was this translation helpful? Give feedback.
-
|
Yes, I believe so. |
Beta Was this translation helpful? Give feedback.
-
|
I re-ran the case by taking your suggestion. It is successful and the CRPSS score is generated ! |
Beta Was this translation helpful? Give feedback.
-
|
Great to hear! I will lock this discussion, but feel free to open a new topic if you have any other issues. |
Beta Was this translation helpful? Give feedback.
Hi @BinbinZhou-NOAA, thank you for making these files available.
(1) I noticed in your MET config file that you have the following set:
climo_mean = fcst;This is typically used to get the field information from the fcst dictionary and use it for the climo_mean field info. There is a METplus setting to accomplish this called ENSEMBLE_STAT_CLIMO_MEAN_USE_FCST . Set this variable to True instead of setting climo_mean.field explicitly.
ENSEMBLE_STAT_CLIMO_MEAN_USE_FCST = True(2) In your METplus log file, I'm noticing that the settings for climo_mean are using some values that differ from the MET config file settings: