Per #2374, incorporate SEEPS documentation.

docs/Users_Guide/appendixC.rst

@@ -382,6 +382,29 @@ For cost / loss ratio above the base rate, the ECLV is defined as:
 
 .. math:: \text{ECLV } = \frac{(cl \ast (h + f)) + m - b}{b \ast (cl - 1)}.
 
+Stable Equitable Error in Probability Space (SEEPS)
+---------------------------------------------------
+
+Included in SEEPS output :numref:`table_PS_format_info_SEEPS` and SEEPS_MPR output :numref:`table_PS_format_info_SEEPS_MPR`
+
+The SEEPS scoring matrix (equation 15 from :ref:`Rodwell et al, 2010 <Rodwell-2010>`) is:
+
+.. math:: \{S^{S}_{vf}\} = \frac{1}{2}
+          \begin{Bmatrix}
+             0 & \frac{1}{1-p_1} & \frac{1}{p_3} + \frac{1}{1-p_1}\\
+             \frac{1}{p_1} & 0 & \frac{1}{p_3}\\
+             \frac{1}{p_1} + \frac{1}{1-p_3} & \frac{1}{1-p_3} & 0
+          \end{Bmatrix}
+
+In addition, Rodwell et al (2011) note that SEEPS can be written as the mean of two 2-category scores that individually assess the dry/light and light/heavy thresholds (:ref:`Rodwell et al., 2011 <Rodwell-2011>`). Each of these scores is like 1 – HK, but written as:
+
+.. math:: \frac{n_{01}}{\text{Expected n}_{.1}} + \frac{n_{10}}{\text{Expected n}_{.0}}
+
+
+where the word expected refers to the mean value deduced from the climatology, rather than the sample mean.
+
+SEEPS scores are expected to lie between 0 and 1, with a perfect forecast having a value of 0. Individual values can be much larger than 1. Results can be presented as a skill score by using the value of 1 – SEEPS.
+
 MET verification measures for continuous variables
 ==================================================
 

docs/Users_Guide/grid-stat.rst

@@ -71,6 +71,15 @@ There are several ways to present the results of the neighborhood approaches, su
 
 The user must specify several parameters in the grid_stat configuration file to utilize the neighborhood approach, such as the interpolation method, size of the smoothing window, and required fraction of valid data points within the smoothing window. For FSS-specific results, the user must specify the size of the neighborhood window, the required fraction of valid data points within the window, and the fractional coverage threshold from which the contingency tables are defined. These parameters are described further in the practical information section below.
 
+.. _grid-stat_seeps:
+
+SEEPS scores
+------------
+
+The Stable Equitable Error in Probability Space (SEEPS) was devised for monitoring global deterministic forecasts of precipitation against the WMO gauge network (:ref:`Rodwell et al., 2010 <Rodwell-2010>`; :ref:`Haiden et al., 2012 <Haiden-2012>`) and is a multi-category score which uses a climatology to account for local variations in behavior. Please see Point-Stat documentation :numref:`PS_seeps` for more details.
+
+The capability to calculate the SEEPS has also been added to Grid-Stat. This follows the method described in :ref:`North et al, 2022 <North-2022>`, which uses the TRMM 3B42 v7 gridded satellite product for the climatological values and interpolates the forecast and observed products onto this grid for evaluation. A 24-hour TRMM climatology (valid at 00 UTC) constructed from data over the time period 1998-2015 is supplied with the release. Expansion of the capability to other fields will occur as well vetted examples and funding allow.
+
 Fourier Decomposition
 ---------------------
 

docs/Users_Guide/point-stat.rst

@@ -136,6 +136,17 @@ The HiRA framework provides a unique method for evaluating models in the neighbo
 
 Often, the neighborhood size is chosen so that multiple models to be compared have approximately the same horizontal resolution. Then, standard metrics for probabilistic forecasts, such as Brier Score, can be used to compare those forecasts. HiRA was developed using surface observation stations so the neighborhood lies completely within the horizontal plane. With any type of upper air observation, the vertical neighborhood must also be defined.
 
+.. _PS_seeps:
+
+SEEPS scores
+------------
+
+The Stable Equitable Error in Probability Space (SEEPS) was devised for monitoring global deterministic forecasts of precipitation against the WMO gauge network (:ref:`Rodwell et al., 2010 <Rodwell-2010>`; :ref:`Haiden et al., 2012 <Haiden-2012>`) and is a multi-category score which uses a climatology to account for local variations in behavior. Since the score uses probability space to define categories using the climatology, it can be aggregated over heterogeneous climate regions. Even though it was developed for use with precipitation forecasts, in principle it could be applied to any forecast parameter for which a sufficiently long time period of observations exists to create a suitable climatology. The computation of SEEPS for precipitation is only supported for now.
+
+For use with precipitation, three categories are used, named ‘dry’, ‘light’ and ‘heavy’. The ‘dry’ category is defined (using the WMO observing guidelines) with any accumulation (rounded to the nearest 0.1 millimeter) that is less than or equal to 0.2 mm.  The remaining precipitation is divided into ‘light’ and ‘heavy’ categories whose thresholds are with respect to a climatology and thus location specific. The light precipitation is defined to occur twice as often as heavy precipitation.
+
+When calculating a single SEEPS value over observing stations for a particular region, the scores should have a density weighting applied which accounts for uneven station distribution in the region of interest (see Section 9.1 in :ref:`Rodwell et al., 2010 <Rodwell-2010>`). This density weighting has not yet been implemented in MET. Global precipitation climatologies calculated from the WMO SYNOP records from 1980-2009 are supplied with the release. At the moment, a 24-hour climatology is available (valid at 00 UTC or 12 UTC), but in future a 6-hour climatology will become available.
+
 .. _PS_Statistical-measures:
 
 Statistical measures

docs/Users_Guide/refs.rst

@@ -132,7 +132,6 @@ References
 |   Weather Forecast., 32 (1), 187 - 198, doi: 10.1175/WAF-D-16-0134.1.
 |
 
-
 .. _Gilleland_PartI-2020:
 
 | Gilleland, E., 2020. Bootstrap methods for statistical inference.
@@ -164,6 +163,14 @@ References
 |   http://www.stat.washington.edu/www/research/reports/
 | 
 
+.. _Haiden-2012:
+
+| Haiden, T., M.J. Rodwell, D.S. Richardson, A. Okagaki, T. Robinson, T. Hewson, 2012.
+|   *Intercomparison of Global Model Precipitation Forecast Skill in 2010/11*
+|   *Using the SEEPS Score*. Monthly Weather Review, 140, 2720-2733.
+|   https://doi.org/10.1175/MWR-D-11-00301.1
+|
+
 .. _Hamill-2001:
 
 | Hamill, T. M., 2001: Interpretation of rank histograms for verifying ensemble
@@ -228,6 +235,15 @@ References
 |   verification. *Monthly Weather Review*, 115, 1330-1338.
 | 
 
+
+.. _North-2022:
+
+| North, R.C.,  M.P. Mittermaier, S.F. Milton, 2022. *Using SEEPS with a*
+|   *TRMM-derived Climatology to Assess Global NWP Precipitation Forecast Skill*.
+|   Monthly Weather Review, 150, 135-155.
+|   https://doi.org/10.1175/MWR-D-20-0347.1
+|
+
 .. _Ou-2016:
 
 | Ou, M. H., Charles, M., & Collins, D. C. 2016: Sensitivity of calibrated week-2
@@ -242,6 +258,21 @@ References
 |   *Monthly Weather Review,* 136, 78-97.
 | 
 
+.. _Rodwell-2010:
+
+| Rodwell, M.J., D.S. Richardson, T.D. Hewson and T. Haiden, 2010. *A new equitable*
+|   *score suitable for verifying precipitation in numerical weather prediction*.
+|   Quarterly Journal of the Royal Meteorological Society, 136: 1344-1463.
+|   https://doi.org/10.1002/qj.656
+|
+
+.. _Rodwell-2011:
+
+| Rodwell, M.J., T. Haiden, D.S. Richardson, 2011. *Developments in Precipitation*
+|   *Verification*. ECMWF Newsletter Number 128.
+|   https://www.ecmwf.int/node/14595
+|
+
 .. _Saetra-2004:
 
 | Saetra O., H. Hersbach, J-R Bidlot, D. Richardson, 2004: Effects of