system test prerequisite problem - zip file is empty

[pshipton]

https://openj9-jenkins.osuosl.org/job/Test_openjdk17_j9_extended.system_x86-64_mac_Nightly_testList_1/546 - mac10-x86-3

21:47:20  build-java:
21:47:20      [javac] Compiling 58 source files to /Users/****/workspace/Test_openjdk17_j9_extended.system_x86-64_mac_Nightly_testList_1/aqa-tests/system/STF/stf.core/bin
21:47:20      [javac] error: error reading /Users/****/workspace/Test_openjdk17_j9_extended.system_x86-64_mac_Nightly_testList_1/aqa-tests/systemtest_prereqs/junit/junit.jar; zip file is empty
21:47:20      [javac] error: error reading /Users/****/workspace/Test_openjdk17_j9_extended.system_x86-64_mac_Nightly_testList_1/aqa-tests/systemtest_prereqs/junit/hamcrest-core.jar; zip file is empty
21:47:20      [javac] error: error reading /Users/****/workspace/Test_openjdk17_j9_extended.system_x86-64_mac_Nightly_testList_1/aqa-tests/systemtest_prereqs/log4j/log4j-api.jar; zip file is empty
21:47:20      [javac] error: error reading /Users/****/workspace/Test_openjdk17_j9_extended.system_x86-64_mac_Nightly_testList_1/aqa-tests/systemtest_prereqs/log4j

[pshipton]

@llxia fyi

[llxia]

This is a known issue with Jenkins copyArtifacts on UNB machine.

00:17:31.891  Timeout set to expire in 1 hr 0 min
[Pipeline] {
[Pipeline] copyArtifacts
01:17:31.892  Cancelling nested steps due to timeout
[Pipeline] }
[Pipeline] // timeout
[Pipeline] echo
01:17:32.069  Exception: org.jenkinsci.plugins.workflow.steps.FlowInterruptedException
[Pipeline] echo
01:17:32.071  Cannot run copyArtifacts from test.getDependency/systemtest.getDependency. Skipping copyArtifacts...

The solution for #4500 resolved the get dependencies (logic in TKG) - test.getDependency for all tests except for system tests. And it worked as expected. systemtest.getDependency has its own logic (in both STF and aqa-systemtest repo). It does not align with the logic in TKG.

If we want to pre-stage system test libs on the machines (same as the rest of tests in TKG), we need to look into system tests logic and update it accordingly.

btw, I do not think systemtest can check sha and re-download. And it uses SDK to build some test libs. https://openj9-jenkins.osuosl.org/job/systemtest.getDependency/36/consoleFull

[llxia]

Alternatively, we can leave https://ci.adoptium.net/job/systemtest.getDependency/ as it is. And leverage https://github.com/adoptium/TKG/blob/master/scripts/getDependencies.pl in TKG to pre-stage 3rd party jars for system tests.

[llxia]

@annaibm Please take a look when you have time. We can talk in more detail in the meeting.

[llxia]

Just for the record, we will break down this into several tasks:

• download the system test jars from https://ci.adoptium.net/job/systemtest.getDependency/ and ignore checksum if the jar is generated by systemtest.getDependency
• pre-stage system test jars in a separate folder on the machine (i.e., testDependency/system_lib)(some system test libs may be the same as functional tests libs with version differences)

Enhancements:

• update systemtest.getDependency to generate checksum when generating the jar
• move testDependency/system_lib into testDependency/lib (remove duplicate libs)
• create Jenkins job to clean testDependency/lib periodically