Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OpenJDK java/lang/StringBuilder/HugeCapacity Unexpected exit from test [exit code: 137] #17323

Open
pshipton opened this issue May 2, 2023 · 12 comments

Comments

@pshipton
Copy link
Member

pshipton commented May 2, 2023

This was seen once before, #17203 (comment)

https://openj9-jenkins.osuosl.org/job/Test_openjdk11_j9_sanity.openjdk_s390x_linux_Nightly/529 - ub20-390-4
jdk_lang_j9_0
-Xdump:system:none -Xdump:heap:none -Xdump:system:events=gpf+abort+traceassert+corruptcache -XX:-JITServerTechPreviewMessage -Xjit -Xgcpolicy:balanced -Xnocompressedrefs
java/lang/StringBuilder/HugeCapacity.java

I don't spot any diagnostics.

00:36:55  ACTION: main -- Failed. Unexpected exit from test [exit code: 137]
00:36:55  REASON: User specified action: run main/othervm -Xmx5G -XX:+CompactStrings HugeCapacity 
00:36:55  TIME:   17.562 seconds
00:36:55  messages:
00:36:55  command: main -Xmx5G -XX:+CompactStrings HugeCapacity
00:36:55  reason: User specified action: run main/othervm -Xmx5G -XX:+CompactStrings HugeCapacity 
00:36:55  Mode: othervm [/othervm specified]
00:36:55  elapsed time (seconds): 17.562
00:36:55  configuration:
00:36:55  STDOUT:
00:36:55  STDERR:
00:36:55  rerun:
00:36:55  cd /home/jenkins/workspace/Test_openjdk11_j9_sanity.openjdk_s390x_linux_Nightly/aqa-tests/TKG/output_16829975804315/jdk_lang_j9_0/work/scratch/0 && \
00:36:55  DISPLAY=:0 \
00:36:55  HOME=/home/jenkins \
00:36:55  LANG=en_US.UTF-8 \
00:36:55  PATH=/bin:/usr/bin:/usr/sbin \
00:36:55  CLASSPATH=/home/jenkins/workspace/Test_openjdk11_j9_sanity.openjdk_s390x_linux_Nightly/aqa-tests/TKG/output_16829975804315/jdk_lang_j9_0/work/classes/1/java/lang/StringBuilder/HugeCapacity.d:/home/jenkins/workspace/Test_openjdk11_j9_sanity.openjdk_s390x_linux_Nightly/aqa-tests/openjdk/openjdk-jdk/test/jdk/java/lang/StringBuilder:/home/jenkins/workspace/Test_openjdk11_j9_sanity.openjdk_s390x_linux_Nightly/jvmtest/openjdk/jtreg/lib/javatest.jar:/home/jenkins/workspace/Test_openjdk11_j9_sanity.openjdk_s390x_linux_Nightly/jvmtest/openjdk/jtreg/lib/jtreg.jar \
00:36:55      /home/jenkins/workspace/Test_openjdk11_j9_sanity.openjdk_s390x_linux_Nightly/openjdkbinary/j2sdk-image/bin/java \
00:36:55          -Dtest.vm.opts='-ea -esa -Xmx512m -Xdump:system:none -Xdump:heap:none -Xdump:system:events=gpf+abort+traceassert+corruptcache -XX:-JITServerTechPreviewMessage -Xjit -Xgcpolicy:balanced -Xnocompressedrefs' \
00:36:55          -Dtest.tool.vm.opts='-J-ea -J-esa -J-Xmx512m -J-Xdump:system:none -J-Xdump:heap:none -J-Xdump:system:events=gpf+abort+traceassert+corruptcache -J-XX:-JITServerTechPreviewMessage -J-Xjit -J-Xgcpolicy:balanced -J-Xnocompressedrefs' \
00:36:55          -Dtest.compiler.opts= \
00:36:55          -Dtest.java.opts= \
00:36:55          -Dtest.jdk=/home/jenkins/workspace/Test_openjdk11_j9_sanity.openjdk_s390x_linux_Nightly/openjdkbinary/j2sdk-image \
00:36:55          -Dcompile.jdk=/home/jenkins/workspace/Test_openjdk11_j9_sanity.openjdk_s390x_linux_Nightly/openjdkbinary/j2sdk-image \
00:36:55          -Dtest.timeout.factor=8.0 \
00:36:55          -Dtest.nativepath=/home/jenkins/workspace/Test_openjdk11_j9_sanity.openjdk_s390x_linux_Nightly/openjdkbinary/openjdk-test-image/jdk/jtreg/native \
00:36:55          -Dtest.root=/home/jenkins/workspace/Test_openjdk11_j9_sanity.openjdk_s390x_linux_Nightly/aqa-tests/openjdk/openjdk-jdk/test/jdk \
00:36:55          -Dtest.name=java/lang/StringBuilder/HugeCapacity.java \
00:36:55          -Dtest.file=/home/jenkins/workspace/Test_openjdk11_j9_sanity.openjdk_s390x_linux_Nightly/aqa-tests/openjdk/openjdk-jdk/test/jdk/java/lang/StringBuilder/HugeCapacity.java \
00:36:55          -Dtest.src=/home/jenkins/workspace/Test_openjdk11_j9_sanity.openjdk_s390x_linux_Nightly/aqa-tests/openjdk/openjdk-jdk/test/jdk/java/lang/StringBuilder \
00:36:55          -Dtest.src.path=/home/jenkins/workspace/Test_openjdk11_j9_sanity.openjdk_s390x_linux_Nightly/aqa-tests/openjdk/openjdk-jdk/test/jdk/java/lang/StringBuilder \
00:36:55          -Dtest.classes=/home/jenkins/workspace/Test_openjdk11_j9_sanity.openjdk_s390x_linux_Nightly/aqa-tests/TKG/output_16829975804315/jdk_lang_j9_0/work/classes/1/java/lang/StringBuilder/HugeCapacity.d \
00:36:55          -Dtest.class.path=/home/jenkins/workspace/Test_openjdk11_j9_sanity.openjdk_s390x_linux_Nightly/aqa-tests/TKG/output_16829975804315/jdk_lang_j9_0/work/classes/1/java/lang/StringBuilder/HugeCapacity.d \
00:36:55          -Dtest.class.path.prefix=/home/jenkins/workspace/Test_openjdk11_j9_sanity.openjdk_s390x_linux_Nightly/aqa-tests/TKG/output_16829975804315/jdk_lang_j9_0/work/classes/1/java/lang/StringBuilder/HugeCapacity.d:/home/jenkins/workspace/Test_openjdk11_j9_sanity.openjdk_s390x_linux_Nightly/aqa-tests/openjdk/openjdk-jdk/test/jdk/java/lang/StringBuilder \
00:36:55          -ea \
00:36:55          -esa \
00:36:55          -Xmx512m \
00:36:55          -Xdump:system:none \
00:36:55          -Xdump:heap:none \
00:36:55          -Xdump:system:events=gpf+abort+traceassert+corruptcache \
00:36:55          -XX:-JITServerTechPreviewMessage \
00:36:55          -Xjit \
00:36:55          -Xgcpolicy:balanced \
00:36:55          -Xnocompressedrefs \
00:36:55          -Djava.library.path=/home/jenkins/workspace/Test_openjdk11_j9_sanity.openjdk_s390x_linux_Nightly/openjdkbinary/openjdk-test-image/jdk/jtreg/native \
00:36:55          -Xmx5G \
00:36:55          -XX:+CompactStrings \
00:36:55          com.sun.javatest.regtest.agent.MainWrapper /home/jenkins/workspace/Test_openjdk11_j9_sanity.openjdk_s390x_linux_Nightly/aqa-tests/TKG/output_16829975804315/jdk_lang_j9_0/work/java/lang/StringBuilder/HugeCapacity.d/main.0.jta
00:36:55  
00:36:55  TEST RESULT: Failed. Unexpected exit from test [exit code: 137]
00:36:55  --------------------------------------------------
00:40:13  Test results: passed: 731; failed: 1
@pshipton
Copy link
Member Author

pshipton commented May 2, 2023

@pshipton
Copy link
Member Author

pshipton commented May 4, 2023

@pshipton
Copy link
Member Author

@dmitripivkine not sure if this belongs to GC. The failure started showing up when we starting testing with -Xgcpolicy:balanced not long ago. We've only seen it on zlinux. It's only seen on jdk11 because the test is excluded from jdk16 (17) due to #10582

@dmitripivkine
Copy link
Contributor

@pshipton @llxia Is there any way to run test differently to get failure details to look at? I can not find any results stored, just error message. There is no reason to run grinders until we figure out how to get meaningful results.

@pshipton
Copy link
Member Author

pshipton commented May 30, 2023

Checking if the failure can be reproduced when running just the failing test.
Using EXTRA_OPTIONS -Xgcpolicy:balanced -Xnocompressedrefs -Xdump:system:events=systhrow,filter=java/lang/OutOfMemoryError
50x x 5 grinder https://openj9-jenkins.osuosl.org/job/Grinder/2476/ - passed on ub20-390-1, 3, 4, 5, 6
50x x 5 grinder https://openj9-jenkins.osuosl.org/job/Grinder/2477/ - passed on ub20-390-3, 5, 6, rh7-390-1, 4

@pshipton
Copy link
Member Author

pshipton commented May 30, 2023

Try running all jdk_lang_j9_0
Using EXTRA_OPTIONS -Xdump:system:events=systhrow,filter=java/lang/OutOfMemoryError, which I expect will create a number of core files for passing tests.
3x x 5 https://openj9-jenkins.osuosl.org/job/Grinder/2478/

@pshipton
Copy link
Member Author

There is a test mismatch in the grinder, but HugeCapacity also failed in 2/5 of the runs (runs 0, 4 on ub20-390-3, 4). The other runs occurred on the rhel7 machines.

The test does only seem to fail on the ub20 machines. It seems like maybe there isn't enough memory to create the JVM and run the test. All the machines have about 8GB, but the ub20 machines seem to be configured with no swap space, while the
rhel7 machines do have swap space. I'll set this as an infra issue.

@JasonFengJ9
Copy link
Member

https://openj9-jenkins.osuosl.org/job/Test_openjdk11_j9_sanity.openjdk_s390x_linux_Release_testList_1/5/consoleFull

04:35:04  TEST RESULT: Failed. Unexpected exit from test [exit code: 137]
04:35:04  --------------------------------------------------
04:37:39  Test results: passed: 730; failed: 1
04:37:55  Report written to /home/jenkins/workspace/Test_openjdk11_j9_sanity.openjdk_s390x_linux_Release_testList_1/aqa-tests/TKG/output_16900990055977/jdk_lang_j9_0/report/html/report.html
04:37:55  Results written to /home/jenkins/workspace/Test_openjdk11_j9_sanity.openjdk_s390x_linux_Release_testList_1/aqa-tests/TKG/output_16900990055977/jdk_lang_j9_0/work
04:37:55  Error: Some tests failed or other problems occurred.
04:37:55  -----------------------------------
04:37:55  jdk_lang_j9_0_FAILED

@pshipton
Copy link
Member Author

pshipton commented Jan 4, 2024

Also fails on rh8-390-2, which also has no swap space. Since we aren't making any progress fixing the machines I will exclude the test until it can pass.

              total        used        free      shared  buff/cache   available
Mem:        7696108     1538712     5004988      368900     1152408     5672452
Swap:             0           0           0

adoptium/aqa-tests#4940

pshipton added a commit to pshipton/openjdk-tests that referenced this issue Jan 4, 2024
It's already excluded on jdk17+ for other reasons.

Issue eclipse-openj9/openj9#17323

Signed-off-by: Peter Shipton <Peter_Shipton@ca.ibm.com>
karianna pushed a commit to adoptium/aqa-tests that referenced this issue Jan 5, 2024
…4940)

It's already excluded on jdk17+ for other reasons.

Issue eclipse-openj9/openj9#17323

Signed-off-by: Peter Shipton <Peter_Shipton@ca.ibm.com>
sophia-guo pushed a commit to sophia-guo/openjdk-tests that referenced this issue Jan 11, 2024
…doptium#4940)

It's already excluded on jdk17+ for other reasons.

Issue eclipse-openj9/openj9#17323

Signed-off-by: Peter Shipton <Peter_Shipton@ca.ibm.com>
smlambert added a commit to adoptium/aqa-tests that referenced this issue Jan 11, 2024
* Use all cores for jtreg instead of halving (#4933)

Signed-off-by: Stewart X Addison <sxa@redhat.com>

* Enable headless mode on RISC-V (#4935)

* Exclude java/lang/StringBuilder/HugeCapacity.java on OpenJ9 zlinux (#4940)

It's already excluded on jdk17+ for other reasons.

Issue eclipse-openj9/openj9#17323

Signed-off-by: Peter Shipton <Peter_Shipton@ca.ibm.com>

* Move custom targets to special level (#4941)

Signed-off-by: Shelley Lambert <slambert@gmail.com>

* Add 21 and 22 as supported versions in external tests (#4950)

related: #4949

Signed-off-by: Lan Xia <Lan_Xia@ca.ibm.com>

* cherry-pick

e37ba10 c5e7de8 0d7c837 76695ca 37b48e3 a45c03c

Signed-off-by: Sophia Guo <sophia.gwf@gmail.com>

---------

Signed-off-by: Stewart X Addison <sxa@redhat.com>
Signed-off-by: Peter Shipton <Peter_Shipton@ca.ibm.com>
Signed-off-by: Shelley Lambert <slambert@gmail.com>
Signed-off-by: Lan Xia <Lan_Xia@ca.ibm.com>
Signed-off-by: Sophia Guo <sophia.gwf@gmail.com>
Co-authored-by: Stewart X Addison <6487691+sxa@users.noreply.github.com>
Co-authored-by: Ludovic Henry <git@ludovic.dev>
Co-authored-by: Peter Shipton <Peter_Shipton@ca.ibm.com>
Co-authored-by: Shelley Lambert <slambert@gmail.com>
Co-authored-by: Lan Xia <Lan_Xia@ca.ibm.com>
pshipton added a commit to pshipton/openjdk-tests that referenced this issue Feb 1, 2024
@pshipton
Copy link
Member Author

pshipton commented Feb 2, 2024

The machines are updated to have swap space, and seem to work. Unexclude the test.
adoptium/aqa-tests#5033

llxia pushed a commit to adoptium/aqa-tests that referenced this issue Feb 2, 2024
…ed (#5033)

Issue eclipse-openj9/openj9#17323

Signed-off-by: Peter Shipton <Peter_Shipton@ca.ibm.com>
@JasonFengJ9
Copy link
Member

openjdk11_j9_sanity.openjdk_s390x_linux(rhel9s390xrt-2)

[2024-11-02T03:21:55.033Z] variation: -Xdump:system:none -Xdump:heap:none -Xdump:system:events=gpf+abort+traceassert+corruptcache -XX:-JITServerTechPreviewMessage Mode501
[2024-11-02T03:21:55.033Z] JVM_OPTIONS:  -Xdump:system:none -Xdump:heap:none -Xdump:system:events=gpf+abort+traceassert+corruptcache -XX:-JITServerTechPreviewMessage -Xjit -Xgcpolicy:balanced -Xnocompressedrefs -Xverbosegclog 

[2024-11-02T03:55:01.772Z] TEST: java/lang/StringBuilder/HugeCapacity.java

[2024-11-02T03:55:01.773Z] TEST RESULT: Failed. Unexpected exit from test [exit code: 137]
[2024-11-02T03:55:01.773Z] --------------------------------------------------
[2024-11-02T04:00:20.340Z] Test results: passed: 738; failed: 1
[2024-11-02T04:01:01.759Z] Report written to /home/jenkins/workspace/Test_openjdk11_j9_sanity.openjdk_s390x_linux/aqa-tests/TKG/output_17305122238337/jdk_lang_j9_0/report/html/report.html
[2024-11-02T04:01:01.759Z] Results written to /home/jenkins/workspace/Test_openjdk11_j9_sanity.openjdk_s390x_linux/aqa-tests/TKG/output_17305122238337/jdk_lang_j9_0/work
[2024-11-02T04:01:01.759Z] Error: Some tests failed or other problems occurred.
[2024-11-02T04:01:01.759Z] -----------------------------------
[2024-11-02T04:01:01.759Z] jdk_lang_j9_0_FAILED

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants