Create an initial development version of a new TC-Diag tool to support the computation of tropical cyclone diagnostics

[JohnHalleyGotway]

Describe the New Feature

The METplus TC-Diagnostics group met on May 23, 2022 to define requirements for incorporating legacy hurricane diagnostics logic in METplus. Those requirement include the creation of a new tool named TC-Diag, for Tropical Cyclone Diagnostics.

The inputs for TC-Diag are the same as those for TC-RMW:

• a list of gridded input data files (-data)
• an ATCF track data file (-deck)
• a configuration file (-config)
• an output file name (-out)

For each track point, TC-Diag transforms the gridded model data to cylindrical coordinates centered on the storm location. This can be done relative to the radius of maximum winds-space, as is done in TC-RMW. However, additional conversion logic should also be supported, such as being based on shear.

ACTION: Need to define what derivation types should be supported.

After the requested fields are read and transformed in one or more ways, provide an option to write them to an output NetCDF file. However, recommend that this option be disabled by default.

Next, incorporate python code from Robert DeMaria directly into the MET repository to actually compute the diagnostics. The TC-Diag makes calls to that python code to derive the diagnostics. The number and type of diagnostics to be computed may depend on the type of cylindrical coordinate transformation that was performed. Ideally the cylindrical coordinate data is handed to python in memory. This logic should be made highly configurable. Provide a mechanism for users to write or modify a python derivation script.

ACTION: Need to define lots of details for how MET calls the python derivation scripts and how this can be made configurable. Also need to define source of those python scripts.

The python scripts hand back to TC-Diag the computed diagnostics. These will consist of a few dozen floating point numbers for each track location.

TC-Diag writes those diagnostic values out to one of two supported formats. The desired output format must be a configurable option:

1. LS Diag (Large Scale Diagnostics) format: See sample LS Diag formatted files only the NHC FTP site.
2. Model Diag format: Need to define documentation and sample data.

Acceptance Testing

Input: gridded model data, ATCF track file, ASCII config file
Output: Formatted ASCII output in the LS-Diag or Model-Diag formats

Time Estimate

2 weeks.

Sub-Issues

Consider breaking the new feature down into sub-issues.

• Add a checkbox for each sub-issue here.

Relevant Deadlines

The initial version of the TC-Diag tool should be ready by February 2023.

Funding Source

NOAA WPO - JTTI TCDiag Project (2770043)

Define the Metadata

Assignee

• Select engineer(s) or no engineer required: @JohnHalleyGotway
• Select scientist(s) or no scientist required: @jvigh and @KathrynNewman

Labels

• Select component(s)
• Select priority
• Select requestor(s)

Projects and Milestone

• Select Repository and/or Organization level Project(s) or add alert: NEED PROJECT ASSIGNMENT label
• Select Milestone as the next official version or Future Versions

Define Related Issue(s)

Consider the impact to the other METplus components.

• MET: Enhance TC-Pairs to read hurricane model diagnostic files (e.g. SHIPS) and TC-Stat to filter the new data #392
• METplus: List issue here
• METdatadb: List issue here
• METviewer: List issue here

New Feature Checklist

See the METplus Workflow for details.

• Complete the issue definition above, including the Time Estimate and Funding source.
• Fork this repository or create a branch of develop.
  Branch name: feature_<Issue Number>_<Description>
• Complete the development and test your changes.
• Add/update log messages for easier debugging.
• Add/update unit tests.
• Add/update documentation.
• Push local changes to GitHub.
• Submit a pull request to merge into develop.
  Pull request: feature <Issue Number> <Description>
• Define the pull request metadata, as permissions allow.
  Select: Reviewer(s) and Linked issues
  Select: Repository level development cycle Project for the next official release
  Select: Milestone as the next official version
• Iterate until the reviewer(s) accept and merge your changes.
• Delete your fork or branch.
• Close this issue.

[JohnHalleyGotway]

Some notes from the project discussion on 8/10/22:

• Need to create a TC-Diag tool to read GRIB data input files since WCOSS does NOT allow pygrib.
• TCDIAG must write to ASCII format while the NetCDF output is desired.
• Recommend also creating a TCDIAG line type written by TC-Pairs.
• The TC-DIAG tool.
  • Read GRIB data.
  • Calls Robert's derivation of diagnostics in Python
  • Writes ASCII output files (a NetCDF file is desired but not required).
• TC-Pairs reads them and incorporates it into a new TCDIAG output line type.

[JohnHalleyGotway]

@jvigh I've worked on the feature_2168_tc_diag branch over the last couple of days. So far all the changes have been plumbing, basically just creating a copy of the tc_rmw tool and renaming it tc_diag, adding a stub for its unit tests (unit_tc_diag.xml), and adding a largely empty chapter to the User's Guide about it.

I'm now ready to start working on substantive code changes, and I'm wondering how to get started.

I'll propose a time for us to meet to decide where to start.

[JohnHalleyGotway]

Based on our discussions, it sounds like TC-Diag will rely very heavily on python embedding in MET. As such, it should only be compiled when the --enable-python configuration option is chosen. Need to start figuring out the details of how we pass data back/forth between the C++ TC-Diag application and the python derivation scripts.

See this python embedding documentation for reference.