Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update Truth: Based on dtcenter/MET#2655 #2327

Closed
JohnHalleyGotway opened this issue Aug 29, 2023 · 1 comment · Fixed by #2331
Closed

Update Truth: Based on dtcenter/MET#2655 #2327

JohnHalleyGotway opened this issue Aug 29, 2023 · 1 comment · Fixed by #2331
Labels
alert: NEED ACCOUNT KEY Need to assign an account key to this issue component: CI/CD Continuous integration and deployment issues priority: blocker Blocker requestor: METplus Team METplus Development Team type: update truth Update truth dataset
Milestone

Comments

@JohnHalleyGotway
Copy link
Collaborator

JohnHalleyGotway commented Aug 29, 2023

Describe Expected Changes

Pull request dtcenter/MET#2655 for issue dtcenter/MET#2648 adds 7 new output columns to the MODE CTS line type. This will modify the output of any existing METplus use case that runs the MODE tool. This issue to validate these expected changes flagged as differences by GHA, and update the reference truth dataset for the develop branch to move past them.

Review the differences flagged in this METplus GHA testing workflow run:
https://github.com/dtcenter/METplus/actions/runs/5932181609

@JohnHalleyGotway JohnHalleyGotway added priority: blocker Blocker alert: NEED ACCOUNT KEY Need to assign an account key to this issue component: CI/CD Continuous integration and deployment issues requestor: METplus Team METplus Development Team labels Aug 29, 2023
@JohnHalleyGotway JohnHalleyGotway added this to the METplus-6.0.0 milestone Aug 29, 2023
@JohnHalleyGotway JohnHalleyGotway added the type: update truth Update truth dataset label Aug 29, 2023
@JohnHalleyGotway JohnHalleyGotway changed the title Update Truth: For dtcenter/MET#2655 Update Truth: Based on dtcenter/MET#2655 Aug 30, 2023
@JohnHalleyGotway
Copy link
Collaborator Author

Reviewing the diffs flagged in GHA testing workflow run:
https://github.com/dtcenter/METplus/actions/runs/5932181609

Differences are flagged in the output of 7 use case groups. All exit with an error code of 2 indicating that there are no runtime errors, just differences.

I downloaded all 7 difference artifacts:

diff-use_cases_climate_0-1.zip
diff-use_cases_marine_and_cryosphere_0-2.zip
diff-use_cases_met_tool_wrapper_0-29_59-62.zip
diff-use_cases_short_range_1.zip
diff-use_cases_short_range_13.zip
diff-use_cases_short_range_7.zip
diff-use_cases_short_range_8.zip

A total of 20 files differ and all 20 of them are MODE CTS output files:

find ./ -type f | grep output | wc -l
20
find ./ -type f | grep output | grep cts | wc -l
20

I used a combination of sed/awk/find to convert the output files back to the the truth format and confirmed that all diffs are due to adding 7 new output columns and modifying the version number from V11.1.0 to V12.0.0.

for output in `find ./ -name "mode*cts_output.txt";`; do
  truth=`echo $output | sed 's/output/truth/g'`;
  cat $truth | awk '{for(i=1;i<=41;i++) printf $i" "; print ""}' > truth;
  cat $output | sed 's/V12.0.0/V11.1.0/g' | awk '{for(i=1;i<=41;i++) printf $i" "; print ""}' > output; 
  echo $output `diff truth output`;
done

This confirms that all diffs are expected.
We should proceed with updating the METplus truth dataset to move past these diffs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
alert: NEED ACCOUNT KEY Need to assign an account key to this issue component: CI/CD Continuous integration and deployment issues priority: blocker Blocker requestor: METplus Team METplus Development Team type: update truth Update truth dataset
Projects
Status: 🏁 Done
Development

Successfully merging a pull request may close this issue.

2 participants