Enhance TC-Pairs consensus logic to compute the spread of the location, wind speed, and pressure.

[JohnHalleyGotway]

Describe the Enhancement

This enhancement request initially arose via METplus discussions in dtcenter/METplus#1341.

The METplus team met with NOAA/EMC on 2/2/2022 to discuss this issue and this GitHub issue is the result. Here are the steps:

• Enhance the consensus derivation logic in TC-Pairs to also compute the location spread and standard deviation of the wind speed and pressure values.
  • Compute the location spread as the mean of the great circle distance from the consensus location to the location of each of the members.
  • Compute the standard deviation of wind speeds and pressure values as you normally would.
• After these are computed, need to decide where/how to write these 3 additional values to the output.
  • One option to consider is adding 3 new columns to the end of the existing TCMPR line type. That wouldn't require many changes to the TC-Stat and the METdatadb load logic updates would be pretty easy. One downside is that the mean location and location spread would be very far from eachother in the output line. And that solution may conflict with the solution for Enhance TC-Pairs to read hurricane model diagnostic files (e.g. SHIPS) and TC-Stat to filter the new data #392.
  • Another option to consider is defining a new version of the TCMPR line type (perhaps "TCMPRCON" or something?) that is the same as the existing TCMPR line type, but with 3 extra columns added internally (not at the end). After the average location, speed, and pressure, we'd report the corresponding spread values. The downside is extra documentation and testing, upgrading TC-Stat to handle this, and upgrading METdata/METviewer to handle it. You'd often want to visualize TCMPR data from constituent models on the same plot at the TCMPRCON data. That may be difficult if they're stored in different line types.

Please see attached slides from the 2/2/2022 meeting:
J.Peng_discussion_20220202_new.pdf

Time Estimate

Estimate the amount of work required here.

Issues should represent approximately 1 to 3 days of work.

Sub-Issues

Consider breaking the enhancement down into sub-issues.

• Add a checkbox for each sub-issue here.

Relevant Deadlines

List relevant project deadlines here or state NONE.

Funding Source

27703542

Define the Metadata

Assignee

• Select engineer(s) or no engineer required
• Select scientist(s) or no scientist required

Labels

• Select component(s)
• Select priority
• Select requestor(s)

Projects and Milestone

• Select Repository and/or Organization level Project(s) or add alert: NEED PROJECT ASSIGNMENT label
• Select Milestone as the next official version or Future Versions

Define Related Issue(s)

Consider the impact to the other METplus components.

• METplus, MET, METdatadb, METviewer, METexpress, METcalcpy, METplotpy
• Define a METdatadb issue to account for any changes to the output format.
• Define a METviewer and/or METplotpy issue to include the spread info in plots.

Enhancement Checklist

See the METplus Workflow for details.

• Complete the issue definition above, including the Time Estimate and Funding Source.
• Fork this repository or create a branch of develop.
  Branch name: feature_<Issue Number>_<Description>
• Complete the development and test your changes.
• Add/update log messages for easier debugging.
• Add/update unit tests.
• Add/update documentation.
• Push local changes to GitHub.
• Submit a pull request to merge into develop.
  Pull request: feature <Issue Number> <Description>
• Define the pull request metadata, as permissions allow.
  Select: Reviewer(s) and Linked issues
  Select: Repository level development cycle Project for the next official release
  Select: Milestone as the next official version
• Iterate until the reviewer(s) accept and merge your changes.
• Delete your fork or branch.
• Close this issue.

[JohnHalleyGotway]

Mentioned in the gap analysis for EVS version 1.0 on 9/7/22.
@JiayiPeng-NOAA should be added as an assignee on this issue.

[JiayiPeng-NOAA]

Hi Seth, I prefer the first one: NUM_MEMBERS. While you are outputting TRACK_SPREAD, MSLP_SPREAD, MAX_WIND_SPREAD, NUM_MEMBERS, please also include: TRACK_MEAN (LAT_MEAN; LON_MEAN), MSLP_MEAN, MAX_WIND_MEAN in the same stat file. Thanks, Jiayi

…

On Mon, Oct 10, 2022 at 3:48 PM Seth Linden ***@***.***> wrote: @KathrynNewman <https://github.com/KathrynNewman> yep sounds good, ywa the track-spread is being output in nautical miles now. @JiayiPeng-NOAA <https://github.com/JiayiPeng-NOAA> we can certainly add a new output column to the TCMPR stat line type that shows the number of ensemble members. What do you think we should call it? NUM_MEMBERS? or CONS_MEMBERS? or SPREAD_MEMBERS? Open to suggestions? — Reply to this email directly, view it on GitHub <#2036 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALRTVNPOJS4527REUDFX5YDWCRXI5ANCNFSM5NNGSS2Q> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[sethlinden]

@JiayiPeng-NOAA sounds good. I think we may already be producing some of that output for the "consensus" model (TRACK_MEAN (LAT_MEAN; LON_MEAN), MSLP_MEAN, MAX_WIND_MEAN). But if we are not, I can certainly add those variables. I will also add output for NUM_MEMBERS. Thanks!

[JiayiPeng-NOAA]

Hi Seth, While you are calculating ensemble spread, you have all the ensemble mean data (Mean_LAT, Mean_LON, Mean_SLP and Mean_WindSpeed). Please output these variables in one file, which will help for further statistical analysis. Thanks, Jiayi

…

On Mon, Oct 10, 2022 at 6:55 PM Seth Linden ***@***.***> wrote: @JiayiPeng-NOAA <https://github.com/JiayiPeng-NOAA> sounds good. I think we may already be producing some of that output for the "consensus" model (TRACK_MEAN (LAT_MEAN; LON_MEAN), MSLP_MEAN, MAX_WIND_MEAN). But if we are not, I can certainly add those variables. I will also add output for NUM_MEMBERS. Thanks! — Reply to this email directly, view it on GitHub <#2036 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALRTVNPM6VY4BIDVY3TSVVTWCSNFTANCNFSM5NNGSS2Q> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[JohnHalleyGotway]

Received feedback from @JiayiPeng-NOAA on 11/14/2022 recommending that we rename the 5 new TCMPR output columns as follows. Note that I swapped the order of TRACK_SPREAD and TRACK_STDEV below to group all the STDEV columns together:

NUM_MEMBERS, TRACK_SPREAD, TRACK_STDEV, MSLP_STDEV, MAX_WIND_STDEV

I confirmed and approved these changes with @KathrynNewman.

Reopening this issue to handle this renaming.