Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature 2215 ioda2nc message type #2217

Merged
merged 4 commits into from
Aug 2, 2022
Merged

Conversation

hsoh-u
Copy link
Collaborator

@hsoh-u hsoh-u commented Jul 29, 2022

  • Added "station_ob" variable to message_type variable name list (metadata_map at config file)
  • Added "station_id" variable to station_id variable name list (metadata_map at config file)
    • Not include XXX@RecMetaData into the metadata variable list
    • "station_id@RecMetaData" variable with dimension (nrecs, nstring) is different from "station_id@MetaData" variable with different dimension (nlocs, nstring).
    • No additional variable is available and not know how to map station_id from "station_id@RecMetaData"
  • If the message type metadata variable does not exist, set the message_type to "NA" instead of empty string "". This gives an option which prevents from filtering out by message type with the following MET tools
    • The empty message_type can not be mapped to any other existing message types
    • By having message_type "NA", they can be mapped to other message type

Expected Differences

  • Do these changes introduce new tools, command line arguments, or configuration file options? [No]

    If yes, please describe:

  • Do these changes modify the structure of existing or add new output data types (e.g. statistic line types or NetCDF variables)? [No]

    If yes, please describe:

Pull Request Testing

  • Describe testing already performed for these changes:

Run unit test.

  • the unit test "ioda2nc_var_all" will have the log message
DEBUG 1: process_ioda_file() -> The metadata variable for message type does not exist!
DEBUG 1: process_ioda_file() -> The metadata variable for station ID does not exist!

ioda2nc adds the message_type and station_id with seneca:/d1/personal/kalb/ioda/raob_all_v1_20201215T1200Z.nc4 with new IODA2NCCondig_default.

/d1/personal/hsoh/data/IODA_files/raob_all_v1_20201215T1200Z.nc4 out_raob_all_v1_20201215T1200Z_msg_type2.nc -config IODA2NCConfig_msg_type -obs_var air_temperature -v 4
  • Recommend testing for the reviewer(s) to perform, including the location of input datasets, and any additional instructions:

  • Do these changes include sufficient documentation updates, ensuring that no errors or warnings exist in the build of the documentation? [No]

  • Do these changes include sufficient testing updates? [No]

  • Will this PR result in changes to the test suite? [Yes]

    If yes, describe the new output and/or changes to the existing output:

The unit test "ioda2nc_var_all" does not have the message_type and station_id variables. The hdr_typ_table was "". It will be changed "NA" at ioda2nc/odb_sonde_16019_all.nc.

  • Please complete this pull request review by [Fill in date].

Pull Request Checklist

See the METplus Workflow for details.

  • Review the source issue metadata (required labels, projects, and milestone).
  • Complete the PR definition above.
  • Ensure the PR title matches the feature or bugfix branch name.
  • Define the PR metadata, as permissions allow.
    Select: Reviewer(s)
    Select: Organization level software support Project or Repository level development cycle Project
    Select: Milestone as the version that will include these changes
  • After submitting the PR, select Linked issues with the original issue number.
  • After the PR is approved, merge your changes. If permissions do not allow this, request that the reviewer do the merge.
  • Close the linked issue and delete your feature or bugfix branch from GitHub.

@hsoh-u hsoh-u requested a review from davidalbo July 29, 2022 21:18
@hsoh-u hsoh-u linked an issue Jul 29, 2022 that may be closed by this pull request
21 tasks
@hsoh-u hsoh-u requested review from JohnHalleyGotway and removed request for davidalbo August 1, 2022 19:02
@JohnHalleyGotway JohnHalleyGotway added this to the MET 11.0.0 milestone Aug 2, 2022
Copy link
Collaborator

@JohnHalleyGotway JohnHalleyGotway left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I approve of this PR.

Note that I did just update reformat_point.rst to make it consistent with the newly update ioda2nc config file. I reviewed the code changes and they look fine. I also inspected the diffs in the regression test output and see that this empty string:

ncdump -v hdr_typ_table odb_sonde_16019_all_TRUTH.nc
 hdr_typ_table =
  "" ;

Has now been corrected to be a non-empty "NA" string:

ncdump -v hdr_typ_table odb_sonde_16019_all_OUTPUT.nc
 hdr_typ_table =
  "NA" ;

@hsoh-u please proceed with the merge when you're ready and close issue #2215.

@hsoh-u hsoh-u merged commit 13f3fe0 into develop Aug 2, 2022
JohnHalleyGotway added a commit that referenced this pull request Aug 3, 2022
Co-authored-by: John Halley Gotway <johnhg@ucar.edu>
Co-authored-by: Julie Prestopnik <jpresto@seneca.rap.ucar.edu>
Co-authored-by: jprestop <jpresto@ucar.edu>
Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu>
Co-authored-by: j-opatz <59586397+j-opatz@users.noreply.github.com>
Co-authored-by: Randy Bullock <bullock@seneca.rap.ucar.edu>
Co-authored-by: davidfillmore <fillmore.winslow.david@gmail.com>
Co-authored-by: rgbullock <bullock@ucar.edu>
Co-authored-by: John Halley Gotway <johnhg@kiowa.rap.ucar.edu>
Co-authored-by: johnhg <johnhg@ucar.edu>
Co-authored-by: Seth Linden <linden@seneca.rap.ucar.edu>
Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com>
Co-authored-by: Seth Linden <linden@kiowa.rap.ucar.edu>
Co-authored-by: Howard Soh <hsoh@kiowa.rap.ucar.edu>
Co-authored-by: Seth Linden <linden@ucar.edu>
Co-authored-by: hsoh-u <hsoh@ucar.edu>
Co-authored-by: John Halley Gotway <johnhg@seneca.rap.ucar.edu>
Co-authored-by: MET Tools Test Account <met_test@seneca.rap.ucar.edu>
Co-authored-by: mo-mglover <78152252+mo-mglover@users.noreply.github.com>
Co-authored-by: davidalbo <dave@ucar.edu>
Co-authored-by: lisagoodrich <33230218+lisagoodrich@users.noreply.github.com>
Co-authored-by: Dan Adriaansen <dadriaan@ucar.edu>
Co-authored-by: Dave Albo <dave@seneca.rap.ucar.edu>
Co-authored-by: Molly Smith <molly.b.smith@noaa.gov>
Co-authored-by: Jonathan Vigh <jvigh@ucar.edu>
@hsoh-u hsoh-u deleted the feature_2215_ioda2nc_message_type branch August 10, 2022 16:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
No open projects
Development

Successfully merging this pull request may close these issues.

Add "station_ob" to metadata_map as a message_type metadata variable for ioda2nc
2 participants