Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix truncated station_id name in the output from IODA2NC #2216

Closed
22 tasks
hsoh-u opened this issue Jul 29, 2022 · 1 comment · Fixed by #2259
Closed
22 tasks

Fix truncated station_id name in the output from IODA2NC #2216

hsoh-u opened this issue Jul 29, 2022 · 1 comment · Fixed by #2259
Assignees
Labels
MET: PreProcessing Tools (Point) priority: high High Priority requestor: UK Met Office United Kingdom Met Office type: bug Fix something that is not working

Comments

@hsoh-u
Copy link
Collaborator

hsoh-u commented Jul 29, 2022

Replace italics below with details for this issue.

Describe the Problem

Provide a clear and concise description of the bug here.

Generated MET point obs NetCDF file by ioda2nc contains truncated station ids. The station id comes form "report_identifier@MetaData" variable. An IODA input file (at seneca:/d1/personal/kalb/ioda/raob_all_v1_20201215T1200Z.nc4) has "station_id@MetaData" variable instead of "report_identifier@MetaData". The MET's NetCDF dimension is 40 bytes (enough to contain the original station IDs).

The metadata variable for station_id@MetaData variable:
Station_ids from the IODA:
station_id@MetaData =
"89009 23 4gIUS02",
"89009 23 4gIUS04",
"89009 23 4gIUS02",
...
"68538-99-9gIUK02",
"68538-99-9gIUS10",
"68538-99-9gIUK04",
"68538-99-9gIUK06",

Generated MET point obs NetCDF file.
hdr_sid_table =
"89009 23",
"89664 23",
"89664 23 4n",
"89625 23",
...
"68538-99-9gIUK02",
"68538-99-9gIUS10",
"68538-99-9gIUK04",
"68538-99-9gIUK06",

Expected Behavior

Provide a clear and concise description of what you expected to happen here.

Do not truncate the string (just strip out trailing white space characters).

Environment

Describe your runtime environment:
*1. Machine: Linux Workstation (seneca)
*2. OS: RedHat Linux
3. Software version number(s) 11.0 beta2

To Reproduce

Describe the steps to reproduce the behavior:

  1. login to seneca
  2. Modify IODA2NC config
metadata_map = [
   { key = "message_type"; val = "msg_type,station_ob"; },
   { key = "station_id";   val = "station_id,report_identifier"; },
   { key = "pressure";     val = "air_pressure,pressure"; },
   { key = "height";       val = "height,height_above_mean_sea_level"; },
   { key = "elevation";    val = ""; }
];
  1. run ioda2nc
./ioda2nc /d1/personal/hsoh/data/IODA_files/raob_all_v1_20201215T1200Z.nc4 out_raob_all_air_temperature.nc /d1/personal/hsoh/git/features/feature_2215_ioda2nc_message_type/MET/share/met/config/IODA2NCConfig -obs_var air_temperature -v 4

Relevant Deadlines

List relevant project deadlines here or state NONE.

Funding Source

2799991

Define the Metadata

Assignee

  • Select engineer(s) or no engineer required
  • Select scientist(s) or no scientist required

Labels

  • Select component(s)
  • Select priority
  • Select requestor(s)

Projects and Milestone

  • Select Organization level Project for support of the current coordinated release
  • Select Repository level Project for development toward the next official release or add alert: NEED PROJECT ASSIGNMENT label
  • Select Milestone as the next bugfix version

Define Related Issue(s)

Consider the impact to the other METplus components.

Bugfix Checklist

See the METplus Workflow for details.

  • Complete the issue definition above, including the Time Estimate and Funding Source.
  • Fork this repository or create a branch of main_<Version>.
    Branch name: bugfix_<Issue Number>_main_<Version>_<Description>
  • Fix the bug and test your changes.
  • Add/update log messages for easier debugging.
  • Add/update unit tests.
  • Add/update documentation.
  • Push local changes to GitHub.
  • Submit a pull request to merge into main_<Version>.
    Pull request: bugfix <Issue Number> main_<Version> <Description>
  • Define the pull request metadata, as permissions allow.
    Select: Reviewer(s) and Linked issues
    Select: Organization level software support Project for the current coordinated release
    Select: Milestone as the next bugfix version
  • Iterate until the reviewer(s) accept and merge your changes.
  • Delete your fork or branch.
  • Complete the steps above to fix the bug on the develop branch.
    Branch name: bugfix_<Issue Number>_develop_<Description>
    Pull request: bugfix <Issue Number> develop <Description>
    Select: Reviewer(s) and Linked issues
    Select: Repository level development cycle Project for the next official release
    Select: Milestone as the next official version
  • Close this issue.
@hsoh-u hsoh-u added type: bug Fix something that is not working alert: NEED MORE DEFINITION Not yet actionable, additional definition required alert: NEED ACCOUNT KEY Need to assign an account key to this issue alert: NEED CYCLE ASSIGNMENT Need to assign to a release development cycle labels Jul 29, 2022
@hsoh-u hsoh-u self-assigned this Jul 29, 2022
@JohnHalleyGotway JohnHalleyGotway added requestor: METplus Team METplus Development Team MET: PreProcessing Tools (Point) priority: high High Priority and removed alert: NEED MORE DEFINITION Not yet actionable, additional definition required alert: NEED CYCLE ASSIGNMENT Need to assign to a release development cycle labels Aug 11, 2022
@JohnHalleyGotway JohnHalleyGotway added this to the MET 10.1.3 (bugfix) milestone Aug 11, 2022
@JohnHalleyGotway JohnHalleyGotway changed the title The station_id is truncated with ioda2.nc Fix the truncated station_id name in the output from IODA2NC Aug 11, 2022
@hsoh-u
Copy link
Collaborator Author

hsoh-u commented Aug 16, 2022

It's better to remove the trailing spaces only.

The IODA specifies the station_id as string type but no comment for the spaces at the station ID.
gsidiag_conv_uv_testinput at JCSDA-internal/ioda-converters is a sample input with space for the station ID.

Station_ID =
  "89512   ",
  "WV784   ",
  "IR224   ",
  "IR224   ",
  "0017164A",
  "0019616A",
...
  "IR 70   ",
  "IR 55   ",
  "IR 70   ",
  "IR 70   ",
  "WV 55   ",
  "WV 55   ",
  "WV 55   ",
...

@hsoh-u hsoh-u closed this as completed Aug 16, 2022
Repository owner moved this from To do to Done in Coordinated METplus-4.1 Support Aug 16, 2022
Repository owner moved this from To Do to Done in MET-11.0.0-beta3 (9/21/22) Aug 16, 2022
@hsoh-u hsoh-u reopened this Aug 16, 2022
Repository owner moved this from Done to In Progress in MET-11.0.0-beta3 (9/21/22) Aug 16, 2022
Repository owner moved this from Done to In progress in Coordinated METplus-4.1 Support Aug 16, 2022
@TaraJensen TaraJensen removed the alert: NEED ACCOUNT KEY Need to assign an account key to this issue label Sep 14, 2022
@hsoh-u hsoh-u linked a pull request Sep 14, 2022 that will close this issue
15 tasks
@hsoh-u hsoh-u moved this from In progress to Pull request review in Coordinated METplus-4.1 Support Sep 14, 2022
@hsoh-u hsoh-u moved this from In Progress to Pull Request Review in MET-11.0.0-beta3 (9/21/22) Sep 14, 2022
hsoh-u pushed a commit that referenced this issue Sep 15, 2022
Repository owner moved this from Pull request review to Done in Coordinated METplus-4.1 Support Sep 15, 2022
Repository owner moved this from Pull Request Review to Done in MET-11.0.0-beta3 (9/21/22) Sep 15, 2022
@JohnHalleyGotway JohnHalleyGotway changed the title Fix the truncated station_id name in the output from IODA2NC Fix truncated station_id name in the output from IODA2NC Dec 9, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
MET: PreProcessing Tools (Point) priority: high High Priority requestor: UK Met Office United Kingdom Met Office type: bug Fix something that is not working
Projects
No open projects
Development

Successfully merging a pull request may close this issue.

3 participants