UBI and Ubuntu 2404 test failure summary

[Haroon-Khel]

Using this issue to summarise the consistent failures of #3586 and #3501 as there were many common failures

UBI 8 and 9 JDK8 all platforms:
Known issue #2949
java/net/Inet6Address/B6206527.java
java/net/ipv6tests/B6521014.java

both are excluded on linux for openj9 https://github.com/adoptium/aqa-tests/blob/7e3811b9d4c6fcf265d2fc7e9852412034fe6260/openjdk/excludes/ProblemList_openjdk8-openj9.txt#L150 and https://github.com/adoptium/aqa-tests/blob/7e3811b9d4c6fcf265d2fc7e9852412034fe6260/openjdk/excludes/ProblemList_openjdk8-openj9.txt#L166

I recommend exclusion for jdk8

UBI8 and 9
Known issue #2631
java/nio/file/Files/probeContentType/Basic.java

excluded on windows for many jdk versions https://github.com/search?q=repo%3Aadoptium%2Faqa-tests+java%2Fnio%2Ffile%2FFiles%2FprobeContentType%2FBasic&type=code

UBI 8 and 9 and Ubuntu 2404, various tests within jdk_security3 are failing for jdk8 and 11, common ones are

sun/security/pkcs11/fips/TestTLS12.java
sun/security/pkcs11/Provider/Login.sh
sun/security/pkcs11/Secmod/AddPrivateKey.java
sun/security/pkcs11/Secmod/GetPrivateKey.java
sun/security/pkcs11/Secmod/JksSetPrivateKey.java
sun/security/pkcs11/Secmod/LoadKeystore.java
sun/security/pkcs11/Secmod/TrustAnchors.java
sun/security/tools/keytool/autotest.sh

Ubuntu 2404 and UBI 8 and 9, JDK 11 all platforms

java/time/test/java/time/format/TestUTCParse.java

[sxa]

Notes:

• Looking at the IPv6 test failures in the issue at infra#2949 it looks like it's passing on some machines and not others on the same host, and with the same OS. Is that correct? In which case we perhaps need to understand the difference instead of just exclude and see if we can make the failing machines work (Assuming this new problem is that same as that one in that issue). Also noting that the -1 machine mentioned in that issue as having been taken offline is now live again.
• Most of the pkcs11/JDK8 ones are covered by JDK8: Failing pkcs11 tests (libnss related?) on AL2023, UBI9+, F35+ aqa-tests#5380
• The exception to the issue in the previous bullet point is TestTLS12 which was noted in the April triage as passing when run on our RHEL6 host, so this likely needs a new issue raised to look at it. infra#2625 suggests there is a fix for this, although it has now been excluded specifically on Linux/ppc64le
• autotest.sh failure was seen on Amazon Linux 2023 although this comment suggests it passed on an Ubuntu 24.04 system.
• Also noting that it's interesting that the IBM943 failure in the nio suite has not been seen on UBI, so that appears to have been Amazon linux specific
• TestUTCParse has been noted elsewhere (including macos) e.g. April triage but I don't believe an issue has been raised, so that should be done.
• Is the java nio Basic test specific to JDK8 too? And if so does that mean JDK17+ are all ok on Ubuntu 24.04 (Which seems consistent with the riscv64 testing at Replace Ubuntu 23.10 Scaleway machines with 24.04 LTS #3598

[Haroon-Khel]

Raised #3658 for the failing TestUTCParse test

[Haroon-Khel]

There also 2 more tests affecting JDK21 for both UBI8/9 and Ubuntu2404

jdk/jfr/jcmd/TestJcmdDump.java
com/sun/jdi/FinalizerTest.java

[smlambert]

There also 2 more tests affecting JDK21 for both UBI8/9 and Ubuntu2404

Noting from our discussion that TestJcmdDump and FinalizerTest are intermittent failures that are considered non-blocking. (OutOfMemory)

[sxa]

We had a call today to discuss some of these failures with a goal to get approval to support Ubuntu 24.04 for the July release:

• @Haroon-Khel will try to install the shared-mime-info package referenced in jdk8 java/nio/file/Files/probeContentType/Basic.java failed on test-docker-fedora34-x64-1 #2631 (comment) to see if ti resolves the probeContentType failure (Noting that the equivalent package on Ubuntu may be different)
• @Haroon-Khel will attempt a run of the TestUTCParse test after adjusting the time zone on the machine to UTC
• @smlambert will take a look at the details of the autotest.sh failure (details+log) to see if there is any obvious cause
• @sxa will try to look into the PKCS11 failures potentially involving others, but we will likely exclude them for this PSU independently of that analysis, particularly since they only seem to affect JDK8.
• I have raised JDK8 fips/TestTLS12 failure on Ubuntu 24.04 aqa-tests#5420 specifically about the fips/TestTLS12 test. Again this is NOT specific to Ubuntu 24.04 as it has been seen on other distributions but it may be different from the pkcs11 failures.

[Haroon-Khel]

@Haroon-Khel will try to install the shared-mime-info package referenced in #2631 (comment) to see if ti resolves the probeContentType failure (Noting that the equivalent package on Ubuntu may be different)

Installing shared-mime-info allows the test to pass. Ive installed it on all of the affected machines and will add it to the dockerstatic dockerfiles

[Haroon-Khel]

@Haroon-Khel will attempt a run of the TestUTCParse test after adjusting the time zone on the machine to UTC

On the failing machines it looks like the time is already set to UTC

root@test-azure-ubuntu2404-x64-1:~# date
Fri Jul  5 16:37:29 UTC 2024

[smlambert]

On the failing machines it looks like the time is already set to UTC

and looking at passing machines do you see differences in output of date?

[sxa]

Noting that based on the table at adoptium/aqa-tests#5420 (comment) I'm ok with the PKCS11 and TestTLS12 failures being no worse on Ubuntu 24.04 than on 22.04 so I do not consider that a blocker.
Also @smlambert just to clarify the autotest.sh failure does not occur on Ubuntu 24.04 so is not a blocker for that version.

[Haroon-Khel]

and looking at passing machines do you see differences in output of date?

On test-docker-alpine319-x64-4 on which the test passes (https://ci.adoptium.net/job/Grinder/10549/console)

8f585a2be287:/# date
Tue Jul  9 15:57:37 UTC 2024

Same format as #3614 (comment)

[sxa]

Summary of follow-up to the action items on the earlier comment:

• The installation of shared-mime info allows the probeContentType test to pass
• TestUTCParse on JDK11 UBI8/9+U2404 is still problematic (Seems unrelated to whether the machine is showing UTC dates by default). Noting also that we've seen this test fail on macos
• keytool/autotest.sh failure seems specific to Amazon Linux 2023 (possibly also UBI)
• The pkcs11 and FIPS PKCS failures are documented in JDK8 fips/TestTLS12 failure on Ubuntu 24.04 aqa-tests#5420 (comment)
• Also the tools/jpackage failure which is due to a change specific to Ubuntu 24.04

[sxa]

Noting that #3521 (related to MD5 and excluded under adoptium/aqa-tests#5220) is also a problem on the newer distributions (including some prior to 24.04) and so should be covered under this issue. This was fixed upstream so that issue has been closed but I'm noting it here for completeness (this issue was created after it was excluded)