Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bugfix #2187 develop add_offset_and_scale_factor #2194

Merged
merged 4 commits into from
Jun 22, 2022

Conversation

hsoh-u
Copy link
Collaborator

@hsoh-u hsoh-u commented Jun 22, 2022

Expected Differences

The main issue was add_offset and scale_factor were applied only both attributes are defined. Now they are applied when any attribute is defined.
Note: The data_type for add_offset and scale_factor was changed from float to double. Some computed values are different.

    float add_offset, scale_factor;
    double *nc_data;

    4.218276, 4.999973, 4.016995, _, _, 1.070221, 2.826573, _, _, _, _, _, 
==>
    double add_offset, scale_factor;
    double *nc_data;

    4.218277, 4.999973, 4.016996, _, _, 1.070221, 2.826573, _, _, _, _, _,
  • Do these changes introduce new tools, command line arguments, or configuration file options? [No]

    If yes, please describe:

  • Do these changes modify the structure of existing or add new output data types (e.g. statistic line types or NetCDF variables)? [No]

    If yes, please describe:

Pull Request Testing

  • Describe testing already performed for these changes:

Note: this is an example: scale_factor is defined, but not add_offset attribute.

/d1/personal/hsoh/git/bugfixes/bugfix_2187_add_offset_and_scale_factor/MET/bin/plot_data_plane /d1/personal/hsoh/data/MET-2187/GETDL3_DAL_CONUS_2021182_1.0.nc scale_factor.ps 'name="HDAY"; level="(*,*)";' -v 9

before:
DEBUG 4: Data plane information:
DEBUG 4:    plane min: -30620
DEBUG 4:    plane max: 215363
===>
after:
DEBUG 7: apply_scale_factor(T) HDAY(d): unpacked data: count=2431643 out of 3640000, scale_factor=0.0001 add_offset=0. FillValue(<T>)=-9.999e+07
DEBUG 7: apply_scale_factor(T)  data range [-3.062 - 21.5363] raw data: [-30620 - 215363] Positive count: 2426664

DEBUG 4: Data plane information:
DEBUG 4:    plane min: -3.062
DEBUG 4:    plane max: 21.5363
  • Recommend testing for the reviewer(s) to perform, including the location of input datasets, and any additional instructions:

  • Do these changes include sufficient documentation updates, ensuring that no errors or warnings exist in the build of the documentation? [No]

  • Do these changes include sufficient testing updates? [No]

  • Will this PR result in changes to the test suite? [Yes]

    If yes, describe the new output and/or changes to the existing output:

The computed values are slightly different for following outputs:
point2grid_GOES_16_ADP.nc
point2grid_GOES_16_AOD_TO_G212_compute.nc
point2grid_GOES_16_AOD_TO_G212_gaussian.nc
point2grid_GOES_16_AOD_TO_G212_grid_map.nc
point2grid_GOES_16_AOD_TO_G212.nc
point2grid_GOES_16_AOD_TO_G212_unsigned.nc

  • Please complete this pull request review by [Fill in date].

Pull Request Checklist

See the METplus Workflow for details.

  • Review the source issue metadata (required labels, projects, and milestone).
  • Complete the PR definition above.
  • Ensure the PR title matches the feature or bugfix branch name.
  • Define the PR metadata, as permissions allow.
    Select: Reviewer(s)
    Select: Organization level software support Project or Repository level development cycle Project
    Select: Milestone as the version that will include these changes
  • After submitting the PR, select Linked issues with the original issue number.
  • After the PR is approved, merge your changes. If permissions do not allow this, request that the reviewer do the merge.
  • Close the linked issue and delete your feature or bugfix branch from GitHub.

@JohnHalleyGotway JohnHalleyGotway added this to the MET 11.0.0 milestone Jun 22, 2022
Copy link
Collaborator

@JohnHalleyGotway JohnHalleyGotway left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I approve of this PR.

  • The code changes look fine to me. I see that you updated the vx_data2d_nc_met, vx_data2d_nc_pinterp, and vx_data2d_nccf libraries to all call common code in the vx_nc_util library for reading data from NetCDF files. That's great.
  • I see that the add_offset and fill_value attributes are now being handled.
  • You note that you expected the regression test to fail... because some Point2Grid output would change. But that is NOT the case. The regression test via GitHub Actions ran fine and flagged no diffs.
  • I also compiled this branch and tested locally to confirm that add_offset and scale_factor really are being applied, even when only one is present...
plot_data_plane GETDL3_DAL_CONUS_2021182_1.0.nc plot.ps 'name="HDAY"; level="(*,*)";' -v 4
DEBUG 4:    plane min: -3.062
DEBUG 4:    plane max: 21.5363

It now applies the scale_factor as it should. Prior to this change the min/max values were:

DEBUG 4:       plane min: -30620
DEBUG 4:       plane max: 215363

Thanks!

@JohnHalleyGotway JohnHalleyGotway changed the title Bugfix 2187 add offset and scale factor Bugfix #2187 develop add_offset_and_scale_factor Jun 22, 2022
@JohnHalleyGotway JohnHalleyGotway mentioned this pull request Jun 22, 2022
15 tasks
@hsoh-u hsoh-u closed this Jun 22, 2022
@hsoh-u hsoh-u reopened this Jun 22, 2022
@hsoh-u hsoh-u merged commit c24957f into develop Jun 22, 2022
@hsoh-u hsoh-u deleted the bugfix_2187_add_offset_and_scale_factor branch June 22, 2022 16:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
No open projects
Development

Successfully merging this pull request may close these issues.

Fix NetCDF library code to process scale_factor and add_offset attributes independently.
2 participants