Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding checksum for latest snapshots artifacts #1574

Conversation

peterzhuamazon
Copy link
Member

Signed-off-by: Peter Zhu zhujiaxi@amazon.com

Description

Adding checksum for latest snapshots artifacts

Issues Resolved

#1497

Check List

  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Peter Zhu <zhujiaxi@amazon.com>
@peterzhuamazon peterzhuamazon requested a review from a team as a code owner January 31, 2022 19:31
@peterzhuamazon peterzhuamazon self-assigned this Jan 31, 2022
@peterzhuamazon peterzhuamazon added bug Something isn't working cicd labels Jan 31, 2022
@peterzhuamazon peterzhuamazon linked an issue Jan 31, 2022 that may be closed by this pull request
Signed-off-by: Peter Zhu <zhujiaxi@amazon.com>
Comment on lines +19 to +20
List <Closure> fileActions = ['createSha512Checksums']
this.registerLibTester(new UploadMinSnapshotsToS3LibTester( fileActions, 'tests/jenkins/data/opensearch-1.3.0.yml' ))
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we are not able to import the lib from vars, I treat this one as a string for comparison @dblock

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If there is a way to import it into unit test, please let me know about it.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand the problem, what would you like to write here instead of what's in the code?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The original code is:

         List <Closure> fileActions = [createSha512Checksums()] 

which does not run because it cannot figure out what is createSha512Checksums() as I cant seems to find a way to import it from /vars/createSha512Checksums.groovy.

Copy link
Member

@dblock dblock Jan 31, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see, you could pass another dummy function in here that is a member of the test class, instead of createSha512Checksums?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can, but that kinda defeat the purpose of testing this.
Will try in a different test tho.
Thanks.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@abhinavGupta16 thoughts?

Copy link
Contributor

@abhinavGupta16 abhinavGupta16 Feb 1, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dblock - A dummy function is a good idea. Although, I don't think a dummy function would do the job either since the passed function in the job is a closure call. The class.name fetches createSha512Checksums$closurecall_1 and not createSha512Checksums. Is it possible to create a dummy closure call in this class?

@peterzhuamazon - I think the test would still be competent since we'd only be passing a dummy function in the expected list instead of the list. The rest would still work the same. What do you think?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried declaring a closure but it doesn't get a name. I would just leave this as is.

vars/uploadMinSnapshotsToS3.groovy Outdated Show resolved Hide resolved
vars/uploadMinSnapshotsToS3.groovy Show resolved Hide resolved
Comment on lines +19 to +20
List <Closure> fileActions = ['createSha512Checksums']
this.registerLibTester(new UploadMinSnapshotsToS3LibTester( fileActions, 'tests/jenkins/data/opensearch-1.3.0.yml' ))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand the problem, what would you like to write here instead of what's in the code?

Signed-off-by: Peter Zhu <zhujiaxi@amazon.com>
@codecov-commenter
Copy link

codecov-commenter commented Jan 31, 2022

Codecov Report

Merging #1574 (50e7640) into main (e9ce1cd) will increase coverage by 0.02%.
The diff coverage is 100.00%.

Impacted file tree graph

@@             Coverage Diff              @@
##               main    #1574      +/-   ##
============================================
+ Coverage     94.32%   94.35%   +0.02%     
  Complexity       12       12              
============================================
  Files           148      149       +1     
  Lines          3154     3170      +16     
  Branches          8        9       +1     
============================================
+ Hits           2975     2991      +16     
  Misses          172      172              
  Partials          7        7              
Impacted Files Coverage Δ
...ts/jenkins/jobs/uploadMinSnapshotsToS3_Jenkinsfile 100.00% <100.00%> (ø)
src/system/execute.py 100.00% <0.00%> (ø)
src/paths/output_dir.py 100.00% <0.00%> (ø)
src/paths/tree_walker.py 100.00% <0.00%> (ø)
src/paths/build_output_dir.py 100.00% <0.00%> (ø)
src/system/working_directory.py 100.00% <0.00%> (ø)
src/system/thread_safe_counter.py 100.00% <0.00%> (ø)
tests/jenkins/jobs/BuildAssembleUpload_Jenkinsfile 100.00% <0.00%> (ø)
src/system/process.py 93.54% <0.00%> (+0.56%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e9ce1cd...50e7640. Read the comment docs.

Signed-off-by: Peter Zhu <zhujiaxi@amazon.com>
Copy link
Contributor

@abhinavGupta16 abhinavGupta16 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for going beyond the requirements for the task and adding the libtester for the existing UploadMinSnapshotsToS3LibTester.groovy library.

@peterzhuamazon peterzhuamazon merged commit ef83dea into opensearch-project:main Jan 31, 2022
@peterzhuamazon peterzhuamazon deleted the opensearch-latestchecksum-1497 branch January 31, 2022 22:47
peterzhuamazon added a commit to peterzhuamazon/opensearch-build that referenced this pull request Feb 16, 2022
* Adding checksum for latest snapshots artifacts

Signed-off-by: Peter Zhu <zhujiaxi@amazon.com>

* Change to snapshots dir name in path

Signed-off-by: Peter Zhu <zhujiaxi@amazon.com>

* Update test stack with new location

Signed-off-by: Peter Zhu <zhujiaxi@amazon.com>

* Simplify the naming in artifacts variables

Signed-off-by: Peter Zhu <zhujiaxi@amazon.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working cicd
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Bug]: checksum check failing for OpenSearch
5 participants