Add logic to Ensemble-Stat to handle an ensemble control member.

[JohnHalleyGotway]

Describe the Enhancement

When verifying ensembles the UK Met Office does not include the control member in the resulting ensemble statistics. This task is to coordinate with @mpm-meto and/or @robdarvell to determine how the Ensemble-Stat tool should handle a control member.

One big question is, if the control member is NOT included in the computation of statistics, who should it be used by Ensemble-Stat. Perhaps there is no need to pass it to Ensemble-Stat in the first place.

If we determine that a control member is needed and useful in Ensemble-Stat, enhance Ensemble-Stat to do so. Clearly, we need more discussion and definition here before this issue is actionable.

Time Estimate

Estimate the amount of work required here.
Issues should represent approximately 1 to 3 days of work.

Sub-Issues

Consider breaking the enhancement down into sub-issues.

• Add a checkbox for each sub-issue here.

Relevant Deadlines

Can move to future development cycles and/or versions depending on when we get the task well-defined.

Funding Source

2799991

Define the Metadata

Assignee

• Select engineer(s) or no engineer required
• Select scientist(s) or no scientist required

Labels

• Select component(s)
• Select priority
• Select requestor(s)

Projects and Milestone

• Select Repository and/or Organization level Project(s) or add alert: NEED PROJECT ASSIGNMENT label
• Select Milestone as the next official version or Future Versions

Define Related Issue(s)

Consider the impact to the other METplus components.

• METplus, MET, METdatadb, METviewer, METexpress, METcalcpy, METplotpy
• Add support for setting control members in EnsembleStat and GenEnsProd METplus#1236

Enhancement Checklist

See the METplus Workflow for details.

• Complete the issue definition above, including the Time Estimate and Funding Source.
• Fork this repository or create a branch of develop.
  Branch name: feature_<Issue Number>_<Description>
• Complete the development and test your changes.
• Add/update log messages for easier debugging.
• Add/update unit tests.
• Add/update documentation.
• Push local changes to GitHub.
• Submit a pull request to merge into develop.
  Pull request: feature <Issue Number> <Description>
• Define the pull request metadata, as permissions allow.
  Select: Reviewer(s) and Linked issues
  Select: Repository level development cycle Project for the next official release
  Select: Milestone as the next official version
• Iterate until the reviewer(s) accept and merge your changes.
• Delete your fork or branch.
• Close this issue.

[JohnHalleyGotway]

From @mpm-meto:

I’ve done some digging. This is the ticket completed about 3 years ago. Historically VER has always computed all the spread-related stats without the control but this was not explicitly stated. This was changed so that what used to be computed was renamed to explicitly state it was excluding the control and a new stat class was added which was for the stats to include the control (you could look at the classes a bit like MET line types).

There is an interesting comment about the anomaly that our calculation of the ensemble mean has always included the control member even when all the spread stats excluded the control.

Hope you can make out the details from the snapshots below. The key one is the difference in the normalisation, i.e. not dividing by n-2 (n-1) when the control is excluded (included). It’s a large change set but there is no easy way of sharing all of it with you. I hope I captured the most relevant bits.

[JohnHalleyGotway]

@mpm-meto and @robdarvell I am working on this issue in the feature_1905_ens_ctrl branch. The MET version 10.1.0 beta3 release includes the initial version of the gen_ens_prod tool which includes the "-ctrl" command line option.
While details are still fresh, I wanted to make sure we make the right changes in ensemble_stat as well.

I've added @j-opatz as an assignee on this issue so that we'll be able to discuss the details internally. Thinking about your comments and screenshots, it sounds like we want:

• ensemble mean always includes the control
• compute the spread both with and without the control member (if it was specified)

The ensemble spread appears in the ECNT line type. I'm wondering how we should handle this? Let's assume a control member was specified using the "-ctrl" command line option. Should we:

(1) Modify the existing ECNT line type to add several new columns to handle the spread with/without ctrl options? (This would be messy and require database changes.)

(2) Write 2 versions of the ECNT line type, one with the control member included, and one excluded? How to differentiate between those? Perhaps set the "DESC"ription column to some string? Or perhaps append some string to the FCST_VAR/OBS_VAR columns (e.g. FCST_VAR = "TMP" and "TMP_EXCL_CTRL")? This would minimize the impact of these changes on NOAA users who don't actually define a control member.

(3) Continue writing a single ECNT line type, but provide a configurable option for whether the control should be include/excluded from the spread. This is easiest, but if you were actually interested in both, you'd need to run Ensemble-Stat twice. And that's not very user friendly. Actually, we wouldn't even need a config file option. If you specify the control as a member, it'd be included in the spread. If you specify it as -ctrl, it'd be excluded. So I'll assume this is not the option you want.

How should we proceed?

If we go with (2), I should probably make a similar change in gen_ens_prod. Instead of just writing a single "spread" field to the NetCDF output, I'd add a "spread_excl_ctrl" variant.

[JohnHalleyGotway]

At this point, I'm going to proceed with the simplest available solution:

If the -cntrl member is specified on the command line, include it in the ensemble mean but exclude it from the spread. Ensemble-Stat will continue to compute a single version of spread, not separate ones with/without the control member included. This avoids the messy question of how to differentiate between the inclusion/exclusion of the control member in the output.

@robdarvell, @mpm-meto, and @j-opatz, if this is not a satisfactory solution, please advise. If you need more time for thought and testing, I recommend that we proceed with the simple solution now and create a new development issue to refine the logic later, as needed.