Quarkus Awt extension doesnt work after upgrading to 3.2.3-Final when using native build #35256

sreekanth-pricer · 2023-08-08T08:08:30Z

Describe the bug

We use Font class from java-awt to parse the font. After upgrading the quarkus from 3.0.3 to 3.2.3-Final. The feature is broken and i get the error "no awt in java.library.path" with the exception ""java.lang.UnsatisfiedLinkError".

Expected behavior

The expected is to not to get the exceptions and should be able to use Font class from java.awt

Actual behavior

i get the error "no awt in java.library.path" with the exception ""java.lang.UnsatisfiedLinkError".

How to Reproduce?

No response

Output of `uname -a` or `ver`

No response

Output of `java -version`

No response

GraalVM version (if different from Java)

No response

Quarkus version or git rev

No response

Build tool (ie. output of `mvnw --version` or `gradlew --version`)

No response

Additional information

No response

The text was updated successfully, but these errors were encountered:

geoand · 2023-08-08T08:18:21Z

cc @Karm

gsmet · 2023-08-08T08:42:40Z

Is it when starting your app from command line or in a Docker container?

sreekanth-pricer · 2023-08-08T09:32:51Z

It is failing in the docker container and running in kubernetes

gsmet · 2023-08-08T09:58:02Z

Yeah, so that's unfortunately expected. GraalVM changed how .so dependencies are handled.

See how it was handled here: quarkusio/quarkus-quickstarts@a6c7cfc#diff-89914cf05ea1b37167e374cb1625de38c8f5d48823d4ad228b15b0a3b34ca9e7R30-R32 .

I don't see it in the migration guide, though. I'll have a look later.

sreekanth-pricer · 2023-08-11T08:40:21Z

I still get the same error even if i do the same changes as mentioned in quarkusio/quarkus-quickstarts@a6c7cfc#diff-89914cf05ea1b37167e374cb1625de38c8f5d48823d4ad228b15b0a3b34ca9e7R30-R32 .

Karm · 2023-08-14T19:20:05Z

I still get the same error even if i do the same changes as mentioned in quarkusio/quarkus-quickstarts@a6c7cfc#diff-89914cf05ea1b37167e374cb1625de38c8f5d48823d4ad228b15b0a3b34ca9e7R30-R32 .

The message just means that the chain of native .so dependencies that was supposed to be loaded in order for awt to work failed.
It does a poor job at stating exactly what failed, which .so was missing or which .so failed to load .e.g due to a missing symbol etc.

Can you share exactly which builder image or GraalVM/Mandrel distribution you use?

sreekanth-pricer · 2023-08-15T08:35:46Z

Hi, Please find the details below ( extracted from our pipeline)

Java version: 17.0.8+7, vendor version: Mandrel-23.0.1.2-Final

Builder Image :
Pulling from quarkus/ubi-quarkus-mandrel-builder-image
Digest: sha256:bfb896cd4e6fff84c77bceeb722dd3e07d23f9e18709b01e09af0d44e9feb0f6
Status: Image is up to date for quay.io/quarkus/ubi-quarkus-mandrel-builder-image:jdk-17
quay.io/quarkus/ubi-quarkus-mandrel-builder-image:jdk-17

Karm · 2023-08-15T08:47:06Z

@sreekanth-pricer , so, how does your app differ from this:

$ git clone https://github.com/quarkusio/quarkus-quickstarts.git
$ cd quarkus-quickstarts/awt-graphics-rest-quickstart/
$ ./mvnw package -Pnative -Dquarkus.native.container-build=true -Dquarkus.native.builder-image=quay.io/quarkus/ubi-quarkus-mandrel-builder-image:jdk-17
$ docker build -f src/main/docker/Dockerfile.native -t quarkus/awt-graphics-rest .
$ docker run -i --rm -p 8080:8080 quarkus/awt-graphics-rest

?

I invite you specifically to check the freetype and fontconfig runtime image dependencies in the Dockerfile.

There is still a chance that your app does something that requires a lib I don't have covered in tests. In that case, would it be possible for you to fork and modify the quickstart app to include your Font parsing code as a quick reproducer?

sreekanth-pricer · 2023-08-15T09:03:11Z

I have a doubt which docker file was supposed to be used, because the Dockerfile.native-micro contains the libfreetype dependency which we use. The other docker Dockerfile.native doesn't have any of these. Could you please explain how it works.

Karm · 2023-08-15T09:13:18Z

@sreekanth-pricer

Dockerfile.native - installs dependencies using microdnf as you can see if you take a look https://github.com/quarkusio/quarkus-quickstarts/blob/main/awt-graphics-rest-quickstart/src/main/docker/Dockerfile.native#L24
Dockerfile.native-micro is targeted to create even smaller image, does not have a capacity for running package manager, so it copies needed files over from another image.

If you are doing this yourself, using your own base and runtime images, note the BEWARE warning:
https://github.com/quarkusio/quarkus-quickstarts/blob/main/awt-graphics-rest-quickstart/src/main/docker/Dockerfile.native-micro#L7

e.g. it won't work if you are trying to copy libraries from newer glibc based Ubuntu Latest to an older glibc based RHEL 8 etc.

It would really make things much faster if you could share more about your setup. e.g. "We have this Linux image X, it's roughly based on XYZ etc."

Karm · 2023-08-15T09:14:56Z

NOTE: The freetype and fontconfig dependency is not new, so if it worked for you before the Quarkus upgrade, it should keep working fine. The only news is the suite of .so files that must go with your quarkus native executable now.

sreekanth-pricer · 2023-08-15T10:46:29Z

We have used jib to build the native image. So I have created a base image for jib as below. The same has been used for both the quarkus versions 3.0 and 3.2 . Worked for 3.0 but not 3.2

FROM quay.io/quarkus/quarkus-micro-image:2.0
COPY --from=BUILD
/lib64/libfreetype.so.6
/lib64/libgcc_s.so.1
/lib64/libbz2.so.1
/lib64/libpng16.so.16
/lib64/libm.so.6
/lib64/libbz2.so.1
/lib64/libexpat.so.1
/lib64/libuuid.so.1
/lib64/

COPY --from=BUILD
/usr/lib64/libfontconfig.so.1
/usr/lib64/

COPY --from=BUILD
/usr/share/fonts /usr/share/fonts

COPY --from=BUILD
/usr/share/fontconfig /usr/share/fontconfig

COPY --from=BUILD
/usr/lib/fontconfig /usr/lib/fontconfig

COPY --from=BUILD
/etc/fonts /etc/fonts

ENV LD_LIBRARY_PATH='/usr/lib64:/lib64'

Karm · 2023-08-15T11:08:06Z

@sreekanth-pricer Hmmm, when you inspect the final runtime image, can you see your quarkus application executable and its necessary .so files along with it? I mean e.g. , in my case:

$ docker run -it --entrypoint /bin/ls quarkus/awt-graphics-rest-quickstart /work/ -1
application
libawt.so
libawt_headless.so
libawt_xawt.so
libfontmanager.so
libjava.so
libjavajpeg.so
libjvm.so
liblcms.so
libmlib_image.so

billapepper · 2023-08-23T00:31:05Z

I can confirm that the quarkusio/quarkus-quickstarts@a6c7cfc#diff-89914cf05ea1b37167e374cb1625de38c8f5d48823d4ad228b15b0a3b34ca9e7R30-R32 changes don't seem to help in the docker container... I added the COPY --chown=1001:root target/*.so /work/ line into my docker file, but awt error continues:

Caused by: java.lang.UnsatisfiedLinkError: no awt in java.library.path
	at org.graalvm.nativeimage.builder/com.oracle.svm.core.jdk.NativeLibrarySupport.loadLibraryRelative(NativeLibrarySupport.java:136)
	at java.base@17.0.8/java.lang.ClassLoader.loadLibrary(ClassLoader.java:50)
	at java.base@17.0.8/java.lang.Runtime.loadLibrary0(Runtime.java:818)
	at java.base@17.0.8/java.lang.System.loadLibrary(System.java:1989)
	at java.desktop@17.0.8/java.awt.Toolkit$2.run(Toolkit.java:1388)
	at java.desktop@17.0.8/java.awt.Toolkit$2.run(Toolkit.java:1386)
	at java.base@17.0.8/java.security.AccessController.executePrivileged(AccessController.java:171)
	at java.base@17.0.8/java.security.AccessController.doPrivileged(AccessController.java:318)
	at java.desktop@17.0.8/java.awt.Toolkit.loadLibraries(Toolkit.java:1385)
	at java.desktop@17.0.8/java.awt.Toolkit.initStatic(Toolkit.java:1423)
	at java.desktop@17.0.8/java.awt.Toolkit.<clinit>(Toolkit.java:1397)
	at java.base@17.0.8/java.lang.Class.ensureInitialized(DynamicHub.java:579)
	at java.desktop@17.0.8/sun.awt.AppContext$2.run(AppContext.java:273)
	at java.desktop@17.0.8/sun.awt.AppContext$2.run(AppContext.java:262)
	at java.base@17.0.8/java.security.AccessController.executePrivileged(AccessController.java:171)
	at java.base@17.0.8/java.security.AccessController.doPrivileged(AccessController.java:318)
	at java.desktop@17.0.8/sun.awt.AppContext.initMainAppContext(AppContext.java:262)
	at java.desktop@17.0.8/sun.awt.AppContext$3.run(AppContext.java:315)
	at java.desktop@17.0.8/sun.awt.AppContext$3.run(AppContext.java:298)
	at java.base@17.0.8/java.security.AccessController.executePrivileged(AccessController.java:171)
	at java.base@17.0.8/java.security.AccessController.doPrivileged(AccessController.java:318)
	at java.desktop@17.0.8/sun.awt.AppContext.getAppContext(AppContext.java:297)
	at java.desktop@17.0.8/javax.imageio.spi.IIORegistry.getDefaultInstance(IIORegistry.java:129)
	at java.desktop@17.0.8/javax.imageio.ImageIO.<clinit>(ImageIO.java:66)
        ...

Has anyone had any luck getting around this error yet?

Quarkus 3.2.4.Final
Java 17.0.8

(Upgrading from 2.x, but was working without issues in 2.x)

Karm · 2023-08-24T21:02:26Z

I can confirm that the quarkusio/quarkus-quickstarts@a6c7cfc#diff-89914cf05ea1b37167e374cb1625de38c8f5d48823d4ad228b15b0a3b34ca9e7R30-R32 changes don't seem to help in the docker container... I added the COPY --chown=1001:root target/*.so /work/ line into my docker file, but awt error continues:

Caused by: java.lang.UnsatisfiedLinkError: no awt in java.library.path
	at org.graalvm.nativeimage.builder/com.oracle.svm.core.jdk.NativeLibrarySupport.loadLibraryRelative(NativeLibrarySupport.java:136)
	at java.base@17.0.8/java.lang.ClassLoader.loadLibrary(ClassLoader.java:50)
	at java.base@17.0.8/java.lang.Runtime.loadLibrary0(Runtime.java:818)
	at java.base@17.0.8/java.lang.System.loadLibrary(System.java:1989)
	at java.desktop@17.0.8/java.awt.Toolkit$2.run(Toolkit.java:1388)
	at java.desktop@17.0.8/java.awt.Toolkit$2.run(Toolkit.java:1386)
	at java.base@17.0.8/java.security.AccessController.executePrivileged(AccessController.java:171)
	at java.base@17.0.8/java.security.AccessController.doPrivileged(AccessController.java:318)
	at java.desktop@17.0.8/java.awt.Toolkit.loadLibraries(Toolkit.java:1385)
	at java.desktop@17.0.8/java.awt.Toolkit.initStatic(Toolkit.java:1423)
	at java.desktop@17.0.8/java.awt.Toolkit.<clinit>(Toolkit.java:1397)
	at java.base@17.0.8/java.lang.Class.ensureInitialized(DynamicHub.java:579)
	at java.desktop@17.0.8/sun.awt.AppContext$2.run(AppContext.java:273)
	at java.desktop@17.0.8/sun.awt.AppContext$2.run(AppContext.java:262)
	at java.base@17.0.8/java.security.AccessController.executePrivileged(AccessController.java:171)
	at java.base@17.0.8/java.security.AccessController.doPrivileged(AccessController.java:318)
	at java.desktop@17.0.8/sun.awt.AppContext.initMainAppContext(AppContext.java:262)
	at java.desktop@17.0.8/sun.awt.AppContext$3.run(AppContext.java:315)
	at java.desktop@17.0.8/sun.awt.AppContext$3.run(AppContext.java:298)
	at java.base@17.0.8/java.security.AccessController.executePrivileged(AccessController.java:171)
	at java.base@17.0.8/java.security.AccessController.doPrivileged(AccessController.java:318)
	at java.desktop@17.0.8/sun.awt.AppContext.getAppContext(AppContext.java:297)
	at java.desktop@17.0.8/javax.imageio.spi.IIORegistry.getDefaultInstance(IIORegistry.java:129)
	at java.desktop@17.0.8/javax.imageio.ImageIO.<clinit>(ImageIO.java:66)
        ...

Has anyone had any luck getting around this error yet?

* Quarkus 3.2.4.Final

* Java 17.0.8

(Upgrading from 2.x, but was working without issues in 2.x)

Could you trace my steps in my previous comment to see what actual files you have there? Also, it's important what's the builder system and its libs, e.g. glibc and the target one.
I will fix the error message in JDK to be more accurate.

Karm · 2023-08-25T20:53:24Z

@sreekanth-pricer ?

sreekanth-pricer · 2023-08-28T07:52:38Z

Hi Karm, I had all the necessary files for my font parsing as i have used a separate base image for the jib as mentioned here #35256 (comment) . Also, we have moved out this font parsing from our quarkus application as we had to fix asap.

gefloh · 2023-08-29T13:54:38Z

We have the exact same problem from Quarkus 3.2.x on. The only problem in our case is: The application is not running inside a Docker container. We are using the quarkus-amazon-lambda extension and natively compile into an AWS lambda. So I am not really sure where to copy these .so files due to the lack of a Dockerfile. What to do @Karm, any idea?

Karm · 2023-08-31T12:13:59Z

@gefloh Could you whip up a small reproducer project I can use with my AWS account? AWS Lambda is a Linux running somewhere and we must copy the needed libs alongside the main Quarkus application executable to it.

gefloh · 2023-08-31T12:24:59Z

Ok, I will fiddle something.

gefloh · 2023-08-31T14:21:40Z

@Karm I created a small reproducer here which tries to convert some HTML into a PDF. When you compile this natively, upload the function.zip to AWS and run the lambda, you get the above mentioned UnsatisfiedLinkError.

AlexCraddock · 2023-08-31T19:07:16Z

I have managed to solve the bug for our case that @billapepper was asking about.

In our case we had this in our docker native build in a file called .dockerignore

*
!target/*-runner
!target/*-runner.jar
!target/lib/*
!target/quarkus-app/*

When the native build runs it adds the native libraries to the builds /target directory.

When the Dockerfile.native was building the copy command

COPY target/*.so /work

It was ignoring the *.so files due to the * in .dockerignore.

I have updated the .dockerignore file to

*
!target/*.so
!target/*-runner
!target/*-runner.jar
!target/lib/*
!target/quarkus-app/*

Now the native libraries are included in our docker image and the AWT processing for image resizing works.

As a note a lot of the examples in https://github.com/quarkusio/quarkus-quickstarts/tree/main for example
https://github.com/quarkusio/quarkus-quickstarts/blob/main/spring-web-quickstart/.dockerignore have the same incorrect setup, as a lot of people possibly copy these examples as a starting point, might be worth checking there to make sure that the *.so files are actually being included in your docker image under /work.

One thing to note the files in the AWT example https://github.com/quarkusio/quarkus-quickstarts/blob/main/awt-graphics-rest-quickstart/.dockerignore actually has the allow in the docker ifgnore for the *.so libraries

Karm · 2023-09-01T13:22:28Z

@AlexCraddock That is an excellent catch.
I will try to propagate this throughout the ecosystem.

Karm · 2023-09-01T13:22:58Z

@gefloh Thx. I'll take a look with my AWS.

Karm · 2023-09-04T11:16:02Z

@Karm I created a small reproducer here which tries to convert some HTML into a PDF. When you compile this natively, upload the function.zip to AWS and run the lambda, you get the above mentioned UnsatisfiedLinkError.

Hello @gefloh, the problem is indeed just the missing libraries.
I used your code to create my own AWS Lambda and it fails the way you described:

❌

✔️

When I manually added the .so files and re-uploaded the Lambda, it all went well:

...returning it me a bunch base64 encoded bytes. When I decoded it and jammed it into a .pdf file, it opened just fine for me, showing a caption "Test":

I'll proceed and fix this in Quarkus presently.

gefloh · 2023-09-04T11:35:57Z

Hello @Karm , ah perfect, thanks for your support.

rsvoboda · 2023-09-20T07:51:53Z

PR #35718 is merged, does it also fix this issue or just #35713?

Karm · 2023-10-03T21:14:14Z

Closing. Users need to copy libs and install libs as necessary as indicted above.

aylwyne · 2023-10-09T14:01:40Z

@Karm I have a question on how you solved this for an AWS Lambda function. You said that you "manually added the .so files and re-uploaded the Lambda, it all went well". Is there a way to get this to happen as part of the quarkus build? I have a build that runs as a github action that uses the quarkus maven command to build the function zip file then uses terraform to apply this to AWS. I'm not sure I see how I would get those .so files included in the function.zip.

gefloh · 2023-10-12T14:01:36Z

@aylwyne Just use a Quarkus version >=3.4.0. The build adds the necessary files automatically to the function.zip.

aylwyne · 2023-10-12T14:09:57Z

@aylwyne Just use a Quarkus version >=3.4.0. The build adds the necessary files automatically to the function.zip.

Thanks. I somehow missed the comment a few back linking to the specific fix in 3.4.0.

sreekanth-pricer added the kind/bug Something isn't working label Aug 8, 2023

quarkus-bot bot added the triage/needs-triage label Aug 8, 2023

geoand added area/native-image and removed triage/needs-triage labels Aug 8, 2023

Karm self-assigned this Aug 15, 2023

This was referenced Sep 4, 2023

AWS Lambda extension does not pack necessary .so files when AWT is used #35713

Closed

Packs libraries alongside executable in function.zip #35718

Merged

Karm closed this as completed Oct 3, 2023

melloware mentioned this issue Aug 15, 2024

Could not initialize class org.apache.pdfbox.pdmodel.PDDocument in quarkus-quickstart/tika-quickstart quarkiverse/quarkus-tika#198

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Quarkus Awt extension doesnt work after upgrading to 3.2.3-Final when using native build #35256

Quarkus Awt extension doesnt work after upgrading to 3.2.3-Final when using native build #35256

sreekanth-pricer commented Aug 8, 2023

geoand commented Aug 8, 2023

gsmet commented Aug 8, 2023

sreekanth-pricer commented Aug 8, 2023 •

edited

Loading

gsmet commented Aug 8, 2023

sreekanth-pricer commented Aug 11, 2023

Karm commented Aug 14, 2023

sreekanth-pricer commented Aug 15, 2023

Karm commented Aug 15, 2023

sreekanth-pricer commented Aug 15, 2023

Karm commented Aug 15, 2023

Karm commented Aug 15, 2023

sreekanth-pricer commented Aug 15, 2023

Karm commented Aug 15, 2023

billapepper commented Aug 23, 2023

Karm commented Aug 24, 2023

Karm commented Aug 25, 2023

sreekanth-pricer commented Aug 28, 2023

gefloh commented Aug 29, 2023

Karm commented Aug 31, 2023

gefloh commented Aug 31, 2023

gefloh commented Aug 31, 2023

AlexCraddock commented Aug 31, 2023 •

edited

Loading

Karm commented Sep 1, 2023

Karm commented Sep 1, 2023

Karm commented Sep 4, 2023 •

edited

Loading

gefloh commented Sep 4, 2023

rsvoboda commented Sep 20, 2023

Karm commented Oct 3, 2023

aylwyne commented Oct 9, 2023

gefloh commented Oct 12, 2023

aylwyne commented Oct 12, 2023

Quarkus Awt extension doesnt work after upgrading to 3.2.3-Final when using native build #35256

Quarkus Awt extension doesnt work after upgrading to 3.2.3-Final when using native build #35256

Comments

sreekanth-pricer commented Aug 8, 2023

Describe the bug

Expected behavior

Actual behavior

How to Reproduce?

Output of uname -a or ver

Output of java -version

GraalVM version (if different from Java)

Quarkus version or git rev

Build tool (ie. output of mvnw --version or gradlew --version)

Additional information

geoand commented Aug 8, 2023

gsmet commented Aug 8, 2023

sreekanth-pricer commented Aug 8, 2023 • edited Loading

gsmet commented Aug 8, 2023

sreekanth-pricer commented Aug 11, 2023

Karm commented Aug 14, 2023

sreekanth-pricer commented Aug 15, 2023

Karm commented Aug 15, 2023

sreekanth-pricer commented Aug 15, 2023

Karm commented Aug 15, 2023

Karm commented Aug 15, 2023

sreekanth-pricer commented Aug 15, 2023

Karm commented Aug 15, 2023

billapepper commented Aug 23, 2023

Karm commented Aug 24, 2023

Karm commented Aug 25, 2023

sreekanth-pricer commented Aug 28, 2023

gefloh commented Aug 29, 2023

Karm commented Aug 31, 2023

gefloh commented Aug 31, 2023

gefloh commented Aug 31, 2023

AlexCraddock commented Aug 31, 2023 • edited Loading

Karm commented Sep 1, 2023

Karm commented Sep 1, 2023

Karm commented Sep 4, 2023 • edited Loading

❌

✔️

gefloh commented Sep 4, 2023

rsvoboda commented Sep 20, 2023

Karm commented Oct 3, 2023

aylwyne commented Oct 9, 2023

gefloh commented Oct 12, 2023

aylwyne commented Oct 12, 2023

Output of `uname -a` or `ver`

Output of `java -version`

Build tool (ie. output of `mvnw --version` or `gradlew --version`)

sreekanth-pricer commented Aug 8, 2023 •

edited

Loading

AlexCraddock commented Aug 31, 2023 •

edited

Loading

Karm commented Sep 4, 2023 •

edited

Loading