[native-image] Adds -H:+ForeignAPISupport for all GraalVM 24.2+ #44092

Karm · 2024-10-24T23:33:30Z

fixes #44001
enables a disabled test, discussed in #43997

Due to the AWT test being disabled it has been slipping under the radar that latest JDK, i.e. JDK 24+ uses JEP 454 API internally at places previously catered to by plain JNI. The fonts detection and fonts loading is such a place. We reflected this change in Mandrel test suite Karm/mandrel-integration-tests#277, but it is finding its way to Quarkus only now with this PR.

See GraalVM notes on the support of this API in Native-image: foreign-interface

I tested this PR with latest JDK 24 based Mandrel dev build Mandrel-24.2.0-dev59c335af5b8 (build 24-beta+20-ea)
and the current GA Mandrel-24.1.1.0-Final (build 23.0.1+11).

Question:

Do we enable the API unconditionally for JDK 24+ based native-image? Or do we detect AWT extension usage? Which other use cases might trigger this in the JDK code?

Explanation of the debug

...for posterity.

The error message is cryptic for information leakage reasons as discussed on openjdk/jdk#20169 (comment).

Caused by: java.io.IOException: Problem reading font data.
	at java.desktop@24-beta/java.awt.Font.createFont0(Font.java:1205)
	at java.desktop@24-beta/java.awt.Font.createFont(Font.java:1076)
	at io.quarkus.awt.it.Application.init(Application.java:63)
	at io.quarkus.awt.it.Application_Bean.doCreate(Unknown Source)
	... 26 more

So the root cause became apparent to me during a gdb session:

(gdb) p *(t->cause->detailMessage->value)
$1054 = {<byte []> = {<java.lang.Object> = {<_objhdr> = {
        hub = 0xe093b8 <io.netty.handler.codec.http2.DefaultHttp2FrameReader::readHeadersFrame(io.netty.channel.ChannelHandlerContext*, 
io.netty.buffer.ByteBuf*, io.netty.handler.codec.http2.Http2FrameListener*)+1096>}, <No data fields>}, len = 101, 
data = 0x7fffed638eb0 "Support for the Java Foreign Function and Memory API is not active: enable with -H:+ForeignAPISupport"}, <No data fields>}
(gdb)

...and only then I recalled the old Karm/mandrel-integration-tests#277.

quarkus-bot · 2024-10-24T23:33:33Z

Thanks for your pull request!

Your pull request does not follow our editorial rules. Could you have a look?

title should preferably start with an uppercase character (if it makes sense!)

_{This message is automatically generated by a bot.}

quarkus-bot · 2024-10-24T23:33:49Z

/cc @galderz (awt)

gsmet · 2024-10-25T15:18:38Z

core/deployment/src/main/java/io/quarkus/deployment/pkg/steps/NativeImageBuildStep.java

+                /*
+                 * Foreign Function and Memory API in Native Image, JDK's JEP 454
+                 * This is needed for JDK 24+ internal native calls due to AWT.
+                 * TODO: Do we enable it globally like so or do we only enable it when AWT extension is used?


I think I would rather be consistent rather than enabling obscure options only when a very specific extension is enabled.

So +1 for your current approach but I also would like @zakkak 's opinion on this.

My main concern is that this option is experimental and that the FFM API support is still WIP.

@Karm is this necessary because we are using internal JDK code that depends on the FFM API or because some library in the classpath detects that we are using JDK 24 and tries to use FFM API? In the first case I guess we can't do much and I would prefer we enable the flag only when necessary (at least for now). In the latter case I believe we should see how to make the said library avoid the use of the FFM API.

Suggested change

* TODO: Do we enable it globally like so or do we only enable it when AWT extension is used?

Karm · 2024-10-29T11:01:54Z

Just a rebase to account for #44139

Karm · 2024-10-29T14:52:20Z

@zakkak If you have any specific suggestion as to what to explore, gimme a hint. I think we eventually need this when Brotli4j switches to FFI too...

zakkak · 2024-10-29T18:06:20Z

@zakkak If you have any specific suggestion as to what to explore, gimme a hint. I think we eventually need this when Brotli4j switches to FFI too...

At the moment it's not clear to me what code is using the FFM API when using AWT. Is it java.awt itself or some 3rd party library?

Karm · 2024-10-30T00:10:02Z

@zakkak If you have any specific suggestion as to what to explore, gimme a hint. I think we eventually need this when Brotli4j switches to FFI too...

At the moment it's not clear to me what code is using the FFM API when using AWT. Is it java.awt itself or some 3rd party library?

It's JDK itself, e.g. in sun.java2d, right off the bat: https://bugs.openjdk.org/browse/JDK-8337237

i.e. this Quarkus PR is not to satisfy any specific third party lib like TwelveMonkeys or Bioformats etc.

zakkak · 2024-10-30T07:10:53Z

Good, in that case I guess enabling it by default (when using GraalVM for JDK >= 24) makes sense.

We could have a check to only do it when the corresponding internal libraries are reached but that will quickly get outdated.

zakkak

Thank you Karm.

Please remove the TODO and it's good to go :)

zakkak · 2024-10-30T07:11:36Z

core/deployment/src/main/java/io/quarkus/deployment/pkg/steps/NativeImageBuildStep.java

+                /*
+                 * Foreign Function and Memory API in Native Image, JDK's JEP 454
+                 * This is needed for JDK 24+ internal native calls due to AWT.
+                 * TODO: Do we enable it globally like so or do we only enable it when AWT extension is used?


Suggested change

* TODO: Do we enable it globally like so or do we only enable it when AWT extension is used?

Karm · 2024-10-30T10:24:12Z

@zakkak Can't merge it until you waive it I guess " Merging can be performed automatically once the requested changes are addressed. "

quarkus-bot · 2024-10-30T16:00:27Z

Status for workflow `Quarkus CI`

This is the status report for running Quarkus CI on commit 42d7a13.

✅ The latest workflow run for the pull request has completed successfully.

It should be safe to merge provided you have a look at the other checks in the summary.

You can consult the Develocity build scans.

Flaky tests - Develocity

⚙️ JVM Tests - JDK 17 Windows

📦 extensions/opentelemetry/deployment

✖ io.quarkus.opentelemetry.deployment.metrics.HttpServerMetricsTest.collectsHttpRouteFromEndAttributes - History

Assertion condition defined as a Lambda expression in io.quarkus.opentelemetry.deployment.common.exporter.InMemoryMetricExporter expected: <true> but was: <false> within 5 seconds. - org.awaitility.core.ConditionTimeoutException

org.awaitility.core.ConditionTimeoutException: Assertion condition defined as a Lambda expression in io.quarkus.opentelemetry.deployment.common.exporter.InMemoryMetricExporter expected: <true> but was: <false> within 5 seconds.
	at org.awaitility.core.ConditionAwaiter.await(ConditionAwaiter.java:167)
	at org.awaitility.core.AssertionCondition.await(AssertionCondition.java:119)
	at org.awaitility.core.AssertionCondition.await(AssertionCondition.java:31)
	at org.awaitility.core.ConditionFactory.until(ConditionFactory.java:1006)
	at org.awaitility.core.ConditionFactory.untilAsserted(ConditionFactory.java:790)
	at io.quarkus.opentelemetry.deployment.common.exporter.InMemoryMetricExporter.assertCountPointsAtLeast(InMemoryMetricExporter.java:131)
	at io.quarkus.opentelemetry.deployment.common.exporter.InMemoryMetricExporter_ClientProxy.assertCountPointsAtLeast(Unknown Source)

Karm requested review from gsmet and zakkak October 24, 2024 23:33

Karm self-assigned this Oct 24, 2024

quarkus-bot bot added area/core area/graphics labels Oct 24, 2024

Karm added the area/native-image label Oct 24, 2024

This comment has been minimized.

Sign in to view

quarkus-bot bot added the triage/flaky-test label Oct 25, 2024

Karm removed the triage/flaky-test label Oct 25, 2024

gsmet approved these changes Oct 25, 2024

View reviewed changes

Karm force-pushed the issue-44001 branch from 8c18d75 to 27b8eb7 Compare October 29, 2024 11:01

This comment has been minimized.

Sign in to view

quarkus-bot bot added the triage/flaky-test label Oct 29, 2024

zakkak requested changes Oct 30, 2024

View reviewed changes

Karm mentioned this pull request Oct 30, 2024

AWT native extension needs porting to JDK 24 ea #44001

Closed

Karm force-pushed the issue-44001 branch from 27b8eb7 to eb483ad Compare October 30, 2024 10:21

Karm requested a review from zakkak October 30, 2024 10:22

[native-image] Adds -H:+ForeignAPISupport for all GraalVM 24.2+

42d7a13

Karm force-pushed the issue-44001 branch from eb483ad to 42d7a13 Compare October 30, 2024 10:22

zakkak approved these changes Oct 30, 2024

View reviewed changes

zakkak added the triage/waiting-for-ci Ready to merge when CI successfully finishes label Oct 30, 2024

zakkak merged commit 45dbff7 into quarkusio:main Oct 30, 2024
52 checks passed

quarkus-bot bot added kind/bugfix and removed triage/waiting-for-ci Ready to merge when CI successfully finishes labels Oct 30, 2024

quarkus-bot bot added this to the 3.17 - main milestone Oct 30, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[native-image] Adds -H:+ForeignAPISupport for all GraalVM 24.2+ #44092

[native-image] Adds -H:+ForeignAPISupport for all GraalVM 24.2+ #44092

Karm commented Oct 24, 2024

quarkus-bot bot commented Oct 24, 2024

quarkus-bot bot commented Oct 24, 2024

This comment has been minimized.

gsmet Oct 25, 2024

zakkak Oct 29, 2024

zakkak Oct 30, 2024

Karm commented Oct 29, 2024

Karm commented Oct 29, 2024

This comment has been minimized.

zakkak commented Oct 29, 2024

Karm commented Oct 30, 2024

zakkak commented Oct 30, 2024

zakkak left a comment

zakkak Oct 30, 2024

Karm commented Oct 30, 2024

quarkus-bot bot commented Oct 30, 2024 •

edited by github-actions bot

Loading

[native-image] Adds -H:+ForeignAPISupport for all GraalVM 24.2+ #44092

[native-image] Adds -H:+ForeignAPISupport for all GraalVM 24.2+ #44092

Conversation

Karm commented Oct 24, 2024

Question:

Explanation of the debug

quarkus-bot bot commented Oct 24, 2024

quarkus-bot bot commented Oct 24, 2024

This comment has been minimized.

gsmet Oct 25, 2024

Choose a reason for hiding this comment

zakkak Oct 29, 2024

Choose a reason for hiding this comment

zakkak Oct 30, 2024

Choose a reason for hiding this comment

Karm commented Oct 29, 2024

Karm commented Oct 29, 2024

This comment has been minimized.

zakkak commented Oct 29, 2024

Karm commented Oct 30, 2024

zakkak commented Oct 30, 2024

zakkak left a comment

Choose a reason for hiding this comment

zakkak Oct 30, 2024

Choose a reason for hiding this comment

Karm commented Oct 30, 2024

quarkus-bot bot commented Oct 30, 2024 • edited by github-actions bot Loading

Status for workflow Quarkus CI

Flaky tests - Develocity

⚙️ JVM Tests - JDK 17 Windows

📦 extensions/opentelemetry/deployment

quarkus-bot bot commented Oct 30, 2024 •

edited by github-actions bot

Loading

Status for workflow `Quarkus CI`