Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bugfix: Fix PB2NC to report accurate total observation counts in log messages #2387

Closed
22 tasks
georgemccabe opened this issue Dec 22, 2022 · 0 comments · Fixed by #2400 or #2411
Closed
22 tasks

Bugfix: Fix PB2NC to report accurate total observation counts in log messages #2387

georgemccabe opened this issue Dec 22, 2022 · 0 comments · Fixed by #2400 or #2411
Assignees
Labels
MET: PreProcessing Tools (Point) priority: high High Priority reporting: DTC NCAR Base NCAR Base DTC Project reporting: DTC NOAA BASE NOAA Office of Atmospheric Research DTC Project requestor: METplus Team METplus Development Team type: bug Fix something that is not working

Comments

@georgemccabe
Copy link
Collaborator

georgemccabe commented Dec 22, 2022

This was discovered while going through the METplus tutorial first pb2nc example

Describe the Problem

Since MET 10.1, the incorrect total number of obs is reported in the logs. It looks like the total number is higher than the actual number of obs that are written to the output file.

Potential fixes (unverified):

  1. It looks like n_derived_obs was added and included in the total count (n_file_obs + n_derived_obs), but n_file_obs is also incremented when n_derived_obs is on this line. Removing n_file_obs++ from this line may fix the issue?
  2. There is a n_total_obs counter variable that is incremented but never used. Using this value in the log instead may produce the correct result.

Expected Behavior

The number of observations written to the NetCDF output file should match the number from the log noting the "Total observations retained or derived."

Environment

Describe your runtime environment:
1. Machine: (e.g. HPC name, Linux Workstation, Mac Laptop)
2. OS: (e.g. RedHat Linux, MacOS)
3. Software version number(s)

To Reproduce

Describe the steps to reproduce the behavior:
1. Go to the METplus tutorial first pb2nc example
2. Set the values from the instructions
3. Run the command (or something similar):

pb2nc \
${METPLUS_DATA}/met_test/data/sample_obs/prepbufr/ndas.t00z.prepbufr.tm12.20070401.nr \
tutorial_pb_run1.nc \
PB2NCConfig_tutorial_run1 \
-v 2

4. Confirm the log message looks like this:

DEBUG 2: Total observations retained or derived = 68870

5. Run again using pb2nc from MET 10.0.0 or earlier and confirm that the log message looks like this:

DEBUG 2: Total observations retained or derived = 52491

and the NetCDF output matches this number.

Relevant Deadlines

List relevant project deadlines here or state NONE.

Funding Source

2792542 NOAA Base

Define the Metadata

Assignee

  • Select engineer(s) or no engineer required
  • Select scientist(s) or no scientist required

Labels

  • Select component(s)
  • Select priority
  • Select requestor(s)

Projects and Milestone

  • Select Organization level Project for support of the current coordinated release
  • Select Repository level Project for development toward the next official release or add alert: NEED PROJECT ASSIGNMENT label
  • Select Milestone as the next bugfix version

Define Related Issue(s)

Consider the impact to the other METplus components.

Bugfix Checklist

See the METplus Workflow for details.

  • Complete the issue definition above, including the Time Estimate and Funding Source.
  • Fork this repository or create a branch of main_<Version>.
    Branch name: bugfix_<Issue Number>_main_<Version>_<Description>
  • Fix the bug and test your changes.
  • Add/update log messages for easier debugging.
  • Add/update unit tests.
  • Add/update documentation.
  • Push local changes to GitHub.
  • Submit a pull request to merge into main_<Version>.
    Pull request: bugfix <Issue Number> main_<Version> <Description>
  • Define the pull request metadata, as permissions allow.
    Select: Reviewer(s) and Development issue
    Select: Organization level software support Project for the current coordinated release
    Select: Milestone as the next bugfix version
  • Iterate until the reviewer(s) accept and merge your changes.
  • Delete your fork or branch.
  • Complete the steps above to fix the bug on the develop branch.
    Branch name: bugfix_<Issue Number>_develop_<Description>
    Pull request: bugfix <Issue Number> develop <Description>
    Select: Reviewer(s) and Development issue
    Select: Repository level development cycle Project for the next official release
    Select: Milestone as the next official version
  • Close this issue.
@georgemccabe georgemccabe added type: bug Fix something that is not working alert: NEED MORE DEFINITION Not yet actionable, additional definition required alert: NEED ACCOUNT KEY Need to assign an account key to this issue alert: NEED CYCLE ASSIGNMENT Need to assign to a release development cycle labels Dec 22, 2022
@georgemccabe georgemccabe added this to the MET 11.0.1 (bugfix) milestone Dec 22, 2022
@hsoh-u hsoh-u removed alert: NEED MORE DEFINITION Not yet actionable, additional definition required alert: NEED CYCLE ASSIGNMENT Need to assign to a release development cycle labels Jan 10, 2023
hsoh-u pushed a commit that referenced this issue Jan 10, 2023
hsoh-u pushed a commit that referenced this issue Jan 10, 2023
@hsoh-u hsoh-u linked a pull request Jan 12, 2023 that will close this issue
15 tasks
@hsoh-u hsoh-u removed the alert: NEED ACCOUNT KEY Need to assign an account key to this issue label Jan 13, 2023
@hsoh-u hsoh-u linked a pull request Jan 17, 2023 that will close this issue
15 tasks
hsoh-u added a commit that referenced this issue Jan 18, 2023
…in_v11.0

#2387 Do not increase n_file_obs when n_derived_obs is increased
@TaraJensen TaraJensen added reporting: DTC NCAR Base NCAR Base DTC Project reporting: DTC NOAA BASE NOAA Office of Atmospheric Research DTC Project labels Jan 25, 2023
@JohnHalleyGotway JohnHalleyGotway changed the title Bugfix: PB2NC total obs count log is incorrect Bugfix: Correct PB2NC to report accurate total observation counts in log messages Feb 21, 2023
@JohnHalleyGotway JohnHalleyGotway changed the title Bugfix: Correct PB2NC to report accurate total observation counts in log messages Bugfix: Fix PB2NC to report accurate total observation counts in log messages Feb 21, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
MET: PreProcessing Tools (Point) priority: high High Priority reporting: DTC NCAR Base NCAR Base DTC Project reporting: DTC NOAA BASE NOAA Office of Atmospheric Research DTC Project requestor: METplus Team METplus Development Team type: bug Fix something that is not working
Projects
Status: Done
Development

Successfully merging a pull request may close this issue.

4 participants