some aix machine got error message : no space left #2448
Comments
|
|
|
You can repeat the test with the minor change to switch SDK_RESOURCE=nightly (which will take the nightly build from the API, versus having to wait for an upstream build to be present). |
|
System storage is updated on -3 and -4, enlarging /home to 40G |
|
Except for build-1 (adopt01), all osuosl systems have /home==40G. Re-running test on |
|
-- My conclusion is that the test string "Np space on Device:" is misleading. Later on there seems to be a core dump related failure: LABEL: CORE_DUMP Date/Time: Thu Jan 20 09:09:57 2022 Description Probable Causes User Causes Failure Causes Detail Data ADDITIONAL INFORMATION |
So, while we have this job - copying the internal link for regenerating the job:
|
|
|
please add label: testFail |
|
Ram again, and saved workspace. When running manually - as jenkins (su - jenkins from root) - I do not see any errors. Sadly. |
|
@sophia-guo You mentioned in the call today that you were still seeing this problem during this week. Can you provide a link to the log please. Also can you confirm if this is limited to |
|
No, this is not limited to The run on Earlier one
https://ci.adoptopenjdk.net/job/Test_openjdk17_hs_extended.openjdk_ppc64_aix_testList_1/20/ |
sophia-guo
changed the title
|
|
So, yesterday - not sure why yet - test-osuosl-aix715-ppc64-4 did run out of FS space. I know because there is an errpt: |
|
So, as I watch system 'in action' I am getting the impression that the way some 2G and 4G transfers are 'so-called' being performed is to transfer them via /tmp (just saw /tmp on test-osuosl-aix715-ppc64-3 fill up /tmp). So, in order to accomodate these new test commands I am restoring /home back top 32G and adding 8 G to /tmp (so it grows to 10G). |
|
Looking at test: ./openjdk/jdk/test/jdk/java/nio/channels/DatagramChannel/SendReceiveMaxSize.java I am wondering if the default udp network sizing is getting in the way:
|
|
OK - lots of detail.
Gory Details |
|
That machine currently has a 31Gb /home with 2.5Gb in use. If it hit an out of space message it would suggest some significant problem occurring during that job (Excessive number or size of core file sizes perhaps?). If we don't spot the failure it may be worth re-running it with It looks like @aixtools is currently re-running an equivalent Grinder at https://ci.adoptopenjdk.net/job/Grinder/5168/console on the machine so I will leave it to him to check |
|
re: no space left: Perhaps it is not /home that is overrunning, but /tmp or /var (or /) where information is being stored as part of the xfer. For whatever reason - |
|
IMHO - there are so many errors - focusing on this one is a waste of time - more likely an artifact of something else (e.g., 5+ core dumps with core files) I am more concerned by: Failedjava/nio/channels/FileChannel/Transfer2GPlus.java.Transfer2GPlus (from java_nio_channels_FileChannel_Transfer2GPlus.java) Failing for the past 1 build
(Since #5168 )
StacktraceExecution failed: `main' threw exception: java.lang.RuntimeException: Too few bytes transferred: 2126643200 < 2147484671 Standard OutputTransferred 2147484671 bytes
Standard Errorjava.lang.RuntimeException: Too few bytes transferred: 2126643200 < 2147484671 at Transfer2GPlus.testToWritableByteChannel(Transfer2GPlus.java:128) at Transfer2GPlus.main(Transfer2GPlus.java:59) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at com.sun.javatest.regtest.agent.MainActionHelper$AgentVMRunnable.run(MainActionHelper.java:312) at java.base/java.lang.Thread.run(Thread.java:833) |
|
re: filespace. Re-read the whole message tree - it seems to be /tmp that is running out of space (0 bytes atm): Will try increasing /tmp.. |
|
p.s.: https://ci.adoptopenjdk.net/job/Test_openjdk17_j9_extended.functional_ppc64_aix/143/ looks stalled - as /tmp had zero (0) bytes free. Maybe the run should be aborted - better restarted - now that /tmp has been enlarged to 8GB. |
|
Increasing the size of /tmp seems to be resolving this (and more??). I'll re-run on system ppc64-2 (as https://ci.adoptopenjdk.net/job/Grinder/5184/) and ppc64-3 (with old size of /tmp) as https://ci.adoptopenjdk.net/job/Grinder/5185/. |
|
@sophia-guo : you say critical, I say solved. In short, /tmp at 4G to too small for testing. The others, surprisedly had all been manually set to a larger value - which is why this did not occur on other systems (recently). imho: this can be closed. |
|
I'm a little surprised that the test is using |
32 tests failed with
no spaceleft.https://ci.adoptopenjdk.net/job/Test_openjdk17_hs_extended.openjdk_ppc64_aix_testList_2/19/#showFailuresLink
The text was updated successfully, but these errors were encountered: