Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature 1943 gridstat seeps #2344

Merged
merged 47 commits into from
Nov 16, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
760f641
#1943 Added seeps to output_flag
Nov 9, 2022
b5d4b46
#1943 Added conf_key_seeps_flag
Nov 9, 2022
782cde9
#1943 Changed argument type PairDataPoint to SeepsAggScore
Nov 9, 2022
09d6b42
#1943 Added compute_seeps_density_vector and compute_aggregated_seeps…
Nov 9, 2022
67b3486
#1943 Added SeepsClimoGrid
Nov 9, 2022
440e4a1
#1943 Support SEEPS
Nov 9, 2022
1a63e83
#1943 Enabled sxx columns
Nov 10, 2022
89fef19
#1943 Enabled sxx columns
Nov 10, 2022
4f2a5fb
#1943 Enabled sxx columns
Nov 10, 2022
7149c7d
Per #2339, rename SeepsAggScore::init() to SeepsAggScore::clear().
JohnHalleyGotway Nov 10, 2022
a346450
Per #2339, first step... enhance Stat-Analysis to parse the SEEPS and…
JohnHalleyGotway Nov 10, 2022
3cd4716
Per #2339, update stat_analysis to make -job filter -line_type SEEPS …
JohnHalleyGotway Nov 10, 2022
65de0a7
#1943 Changed float to double
Nov 11, 2022
ddb8154
#1943 Added SEEPS
Nov 11, 2022
c3504b8
#1943 Added seeps_qc
Nov 11, 2022
a9c75b7
#1943 Added seeps_qc
Nov 11, 2022
0c97d5f
#1943 support seeps_qc
Nov 11, 2022
c3beffd
#1943 Support seeps_qc
Nov 11, 2022
3e86076
#1943 DSave SEEPS_MPR score, obs_cat, and fcst_cat to netcdf output
Nov 11, 2022
a9e1adc
#1943 Support seeps_qc
Nov 11, 2022
2a59fe7
#1943 Add SEEPS MPR score, obs category amnd forecasr category to Net…
Nov 11, 2022
7e50df7
#1943 added SEEPS_QC
Nov 11, 2022
2d5474a
#1943 Added grid_stat_UK_SEEPS
Nov 12, 2022
335b973
Feature #2339 stat_analysis_seeps (#2343)
JohnHalleyGotway Nov 12, 2022
a3e2872
#1943 Corrected env_name for seeps climo
Nov 12, 2022
3816fda
#1943 Moved density_radius_rad from src/libcode/vx_statistics/compute…
Nov 12, 2022
7831623
#1943 Fix uninitialied variables
Nov 12, 2022
133291b
#1943 activated weighted SEEPS score
Nov 12, 2022
857cd80
Empty config issues.
JohnHalleyGotway Nov 13, 2022
2e32fdb
Empty config issue again.
JohnHalleyGotway Nov 13, 2022
6c341e5
#1943 Added seeps_qc
Nov 14, 2022
df44a59
Merge branch 'feature_1943_gridstat_seeps' of github.com:dtcenter/MET…
Nov 14, 2022
6dd56de
#1943 Added seeps_qc
Nov 14, 2022
025a2d5
#1943 Removed seeps_qs
Nov 15, 2022
357a420
#1943 Added seeps_p1_thresh
Nov 15, 2022
d970a8e
#1943 Added seeps_p1_thresh and removed seeps_qc
Nov 15, 2022
b2db2be
#1943 Added seeps_p1_thresh and removed seeps_qc
Nov 15, 2022
2dbdc0d
#1943 Added seeps_p1_thresh and removed seeps_qc
Nov 15, 2022
d426ac6
#1943 use seeps_p1_thresh for SEEPS QC for p1 value
Nov 15, 2022
45c9b03
#1943 use seeps_p1_thresh for SEEPS QC for p1 value
Nov 15, 2022
78edfc0
#1943 Use seeps_p1_thresh for SEEPS QC for p1 value
Nov 15, 2022
d2c3e97
#1943 Parse seeps_p1_thresh
Nov 15, 2022
755d17f
#1943 Parse seeps_p1_thresh
Nov 15, 2022
a5e0f71
#1943 Added SEEPS_P1_THRESH
Nov 15, 2022
c92819d
#1943 Replaced SEEPS_QC to SEEPS_P1_THRESH
Nov 15, 2022
8be50d5
#1943 Initial release
Nov 15, 2022
9f97c0d
#1943 Turn off weighted SEEPS score
Nov 16, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions data/config/GridStatConfig_default
Original file line number Diff line number Diff line change
Expand Up @@ -234,6 +234,7 @@ output_flag = {
nbrcnt = NONE;
grad = NONE;
dmap = NONE;
seeps = NONE;
}

//
Expand All @@ -246,6 +247,7 @@ nc_pairs_flag = {
diff = TRUE;
climo = TRUE;
climo_cdp = FALSE;
seeps = FALSE;
weight = FALSE;
nbrhd = FALSE;
fourier = FALSE;
Expand All @@ -254,6 +256,11 @@ nc_pairs_flag = {
apply_mask = TRUE;
}

////////////////////////////////////////////////////////////////////////////////
// Threshold for SEEPS p1 (Probability of being dry)

seeps_p1_thresh = NA;

////////////////////////////////////////////////////////////////////////////////

grid_weight_flag = NONE;
Expand Down
4 changes: 4 additions & 0 deletions data/config/PointStatConfig_default
Original file line number Diff line number Diff line change
Expand Up @@ -290,6 +290,10 @@ output_flag = {
seeps_mpr = NONE;
}

////////////////////////////////////////////////////////////////////////////////
// Threshold for SEEPS p1 (Probability of being dry)

seeps_p1_thresh = NA;
////////////////////////////////////////////////////////////////////////////////

tmp_dir = "/tmp";
Expand Down
8 changes: 8 additions & 0 deletions docs/Users_Guide/config_options.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1456,6 +1456,14 @@ In this way, the number of bins impacts the resolution of the climatological
probabilities. These derived probability values are used to compute the
climatological Brier Score and Brier Skill Score.


The "seeps_p1_thresh" option controls the threshold of p1 (probability of being dry) values.

.. code-block:: none

seeps_p1_thresh = >=0.1&&<=0.85;


mask_missing_flag
^^^^^^^^^^^^^^^^^

Expand Down
6 changes: 6 additions & 0 deletions docs/Users_Guide/grid-stat.rst
Original file line number Diff line number Diff line change
Expand Up @@ -362,6 +362,7 @@ _____________________
nbrcnt = BOTH;
grad = BOTH;
dmap = BOTH;
seeps = NONE;
}


Expand Down Expand Up @@ -410,9 +411,14 @@ The **output_flag** array controls the type of output that the Grid-Stat tool ge

21. **DMAP** for Distance Map Statistics

22. **SEEPS** for SEEPS (Stable Equitable Error in Probability Space) score. It's described in :numref:`table_PS_format_info_SEEPS`. The SEEPS score of matched pair data is saved into the NetCDF.


Note that the first two line types are easily derived from one another. The user is free to choose which measure is most desired. The output line types are described in more detail in :numref:`grid_stat-output`.

The SEEPS climo file is not distributed with MET tools because of the file size. It should be configured by using the environment variable, MET_SEEPS_GRID_CLIMO_NAME.


_____________________

.. code-block:: none
Expand Down
2 changes: 2 additions & 0 deletions docs/Users_Guide/point-stat.rst
Original file line number Diff line number Diff line change
Expand Up @@ -487,6 +487,8 @@ Note that writing out matched pair data (MPR lines) for a large number of cases

If all line types corresponding to a particular verification method are set to NONE, the computation of those statistics will be skipped in the code and thus make the Point-Stat tool run more efficiently. For example, if FHO, CTC, and CTS are all set to NONE, the Point-Stat tool will skip the categorical verification step.

The default SEEPS climo file exists at MET_BASE/share/met/climo/seeps/PPT24_seepsweights.nc. It can be overridden by using the environment variable, MET_SEEPS_POINT_CLIMO_NAME.

.. _point_stat-output:

point_stat output
Expand Down
4 changes: 2 additions & 2 deletions docs/Users_Guide/stat-analysis.rst
Original file line number Diff line number Diff line change
Expand Up @@ -40,12 +40,12 @@ The **-derive** job command option can be used to perform the derivation of stat
Aggregated values from multiple STAT lines
------------------------------------------

The Stat-Analysis "aggregate" job aggregates values from multiple STAT lines of the same type. The user may specify the specific line type of interest and any other relevant search criteria. The Stat-Analysis tool then creates sums of each of the values in all lines matching the search criteria. The aggregated data are output as the same line type as the user specified. The STAT line types which may be aggregated in this way are the contingency table (FHO, CTC, PCT, MCTC, NBRCTC), partial sums (SL1L2, SAL1L2, VL1L2, and VAL1L2), and other (ISC, ECNT, RPS, RHIST, PHIST, RELP, NBRCNT, SSVAR, and GRAD) line types.
The Stat-Analysis "aggregate" job aggregates values from multiple STAT lines of the same type. The user may specify the specific line type of interest and any other relevant search criteria. The Stat-Analysis tool then creates sums of each of the values in all lines matching the search criteria. The aggregated data are output as the same line type as the user specified. The STAT line types which may be aggregated in this way are the contingency table (FHO, CTC, PCT, MCTC, NBRCTC), partial sums (SL1L2, SAL1L2, VL1L2, and VAL1L2), and other (ISC, ECNT, RPS, RHIST, PHIST, RELP, NBRCNT, SSVAR, GRAD, and SEEPS) line types.

Aggregate STAT lines and produce aggregated statistics
------------------------------------------------------

The Stat-Analysis "aggregate-stat" job aggregates multiple STAT lines of the same type together and produces relevant statistics from the aggregated line. This may be done in the same manner listed above in :numref:`StA_Aggregated-values-from`. However, rather than writing out the aggregated STAT line itself, the relevant statistics generated from that aggregated line are provided in the output. Specifically, if a contingency table line type (FHO, CTC, PCT, MCTC, or NBRCTC) has been aggregated, contingency table statistics (CTS, ECLV, PSTD, MCTS, or NBRCTS) line types can be computed. If a partial sums line type (SL1L2 or SAL1L2) has been aggregated, the continuous statistics (CNT) line type can be computed. If a vector partial sums line type (VL1L2 or VAL1L2) has been aggregated, the vector continuous statistics (VCNT) line type can be computed. For ensembles, the ORANK line type can be accumulated into ECNT, RPS, RHIST, PHIST, RELP, or SSVAR output. If the matched pair line type (MPR) has been aggregated, may output line types (FHO, CTC, CTS, CNT, MCTC, MCTS, SL1L2, SAL1L2, VL1L2, VCNT, WDIR, PCT, PSTD, PJC, PRC, or ECLV) can be computed. Multiple output line types may be specified for each "aggregate-stat" job, as long as each output is derivable from the input.
The Stat-Analysis "aggregate-stat" job aggregates multiple STAT lines of the same type together and produces relevant statistics from the aggregated line. This may be done in the same manner listed above in :numref:`StA_Aggregated-values-from`. However, rather than writing out the aggregated STAT line itself, the relevant statistics generated from that aggregated line are provided in the output. Specifically, if a contingency table line type (FHO, CTC, PCT, MCTC, or NBRCTC) has been aggregated, contingency table statistics (CTS, ECLV, PSTD, MCTS, or NBRCTS) line types can be computed. If a partial sums line type (SL1L2 or SAL1L2) has been aggregated, the continuous statistics (CNT) line type can be computed. If a vector partial sums line type (VL1L2 or VAL1L2) has been aggregated, the vector continuous statistics (VCNT) line type can be computed. For ensembles, the ORANK line type can be accumulated into ECNT, RPS, RHIST, PHIST, RELP, or SSVAR output. If a SEEPS matched pair line type (SEEPS_MPR) has been aggregated, the aggregated SEEPS line type (SEEPS) can be computed. If the matched pair line type (MPR) has been aggregated, may output line types (FHO, CTC, CTS, CNT, MCTC, MCTS, SL1L2, SAL1L2, VL1L2, VCNT, WDIR, PCT, PSTD, PJC, PRC, or ECLV) can be computed. Multiple output line types may be specified for each "aggregate-stat" job, as long as each output is derivable from the input.

When aggregating the matched pair line type (MPR), additional required job command options are determined by the requested output line type(s). For example, the "-out_thresh" (or "-out_fcst_thresh" and "-out_obs_thresh" options) are required to compute contingnecy table counts (FHO, CTC) or statistics (CTS). Those same job command options can also specify filtering thresholds when computing continuous partial sums (SL1L2, SAL1L2) or statistics (CNT). Output is written for each threshold specified.

Expand Down
6 changes: 6 additions & 0 deletions internal/test_unit/config/GridStatConfig_APCP_regrid
Original file line number Diff line number Diff line change
Expand Up @@ -180,6 +180,7 @@ output_flag = {
nbrcnt = NONE;
grad = NONE;
dmap = NONE;
seeps = NONE;
}

//
Expand All @@ -199,6 +200,11 @@ nc_pairs_flag = {
apply_mask = TRUE;
}

////////////////////////////////////////////////////////////////////////////////
// Threshold for SEEPS p1 (Probability of being dry)

seeps_p1_thresh = NA;

////////////////////////////////////////////////////////////////////////////////

grid_weight_flag = NONE;
Expand Down
6 changes: 6 additions & 0 deletions internal/test_unit/config/GridStatConfig_GRIB_lvl_typ_val
Original file line number Diff line number Diff line change
Expand Up @@ -277,6 +277,7 @@ output_flag = {
nbrcnt = NONE;
grad = NONE;
dmap = NONE;
seeps = NONE;
};

//
Expand All @@ -296,6 +297,11 @@ nc_pairs_flag = {
apply_mask = FALSE;
}

////////////////////////////////////////////////////////////////////////////////
// Threshold for SEEPS p1 (Probability of being dry)

seeps_p1_thresh = NA;

////////////////////////////////////////////////////////////////////////////////

grid_weight_flag = NONE;
Expand Down
6 changes: 6 additions & 0 deletions internal/test_unit/config/GridStatConfig_GRIB_set_attr
Original file line number Diff line number Diff line change
Expand Up @@ -209,6 +209,7 @@ output_flag = {
nbrcnt = NONE;
grad = NONE;
dmap = NONE;
seeps = NONE;
};

//
Expand All @@ -228,6 +229,11 @@ nc_pairs_flag = {
apply_mask = FALSE;
}

////////////////////////////////////////////////////////////////////////////////
// Threshold for SEEPS p1 (Probability of being dry)

seeps_p1_thresh = NA;

////////////////////////////////////////////////////////////////////////////////

grid_weight_flag = NONE;
Expand Down
6 changes: 6 additions & 0 deletions internal/test_unit/config/GridStatConfig_GTG_latlon
Original file line number Diff line number Diff line change
Expand Up @@ -188,6 +188,7 @@ output_flag = {
nbrcnt = NONE;
grad = NONE;
dmap = NONE;
seeps = NONE;
}

//
Expand All @@ -207,6 +208,11 @@ nc_pairs_flag = {
apply_mask = TRUE;
}

////////////////////////////////////////////////////////////////////////////////
// Threshold for SEEPS p1 (Probability of being dry)

seeps_p1_thresh = NA;

////////////////////////////////////////////////////////////////////////////////

grid_weight_flag = NONE;
Expand Down
6 changes: 6 additions & 0 deletions internal/test_unit/config/GridStatConfig_GTG_lc
Original file line number Diff line number Diff line change
Expand Up @@ -188,6 +188,7 @@ output_flag = {
nbrcnt = NONE;
grad = NONE;
dmap = NONE;
seeps = NONE;
}

//
Expand All @@ -207,6 +208,11 @@ nc_pairs_flag = {
apply_mask = TRUE;
}

////////////////////////////////////////////////////////////////////////////////
// Threshold for SEEPS p1 (Probability of being dry)

seeps_p1_thresh = NA;

////////////////////////////////////////////////////////////////////////////////

grid_weight_flag = NONE;
Expand Down
Loading