Make Netty native transport work on Linux AArch64 and RISC-V (besides x86-64)

[mkurz]

I run Linux on a MacBook M1 Pro:

$ uname -m
aarch64

Also it seems more cloud services offer Linux arm64 (=aarch64) virtual machines nowadays. The problem is currently when running Play with the netty backend on such CPUs with play.server.netty.transport = "native" configured, you end up with this error:

2024-02-26 13:41:19 DEBUG i.n.u.internal.NativeLibraryLoader  -Dio.netty.native.workdir: /tmp (io.netty.tmpdir)
2024-02-26 13:41:19 DEBUG i.n.u.internal.NativeLibraryLoader  -Dio.netty.native.deleteLibAfterLoading: true
2024-02-26 13:41:19 DEBUG i.n.u.internal.NativeLibraryLoader  -Dio.netty.native.tryPatchShadedId: true
2024-02-26 13:41:19 DEBUG i.n.u.internal.NativeLibraryLoader  -Dio.netty.native.detectNativeLibraryDuplicates: true
Oops, cannot start the server.
java.lang.UnsatisfiedLinkError: failed to load the required native library
        at io.netty.channel.epoll.Epoll.ensureAvailability(Epoll.java:81)
        at io.netty.channel.epoll.EpollEventLoop.<clinit>(EpollEventLoop.java:57)
        at io.netty.channel.epoll.EpollEventLoopGroup.newChild(EpollEventLoopGroup.java:189)
        at io.netty.channel.epoll.EpollEventLoopGroup.newChild(EpollEventLoopGroup.java:37)
        at io.netty.util.concurrent.MultithreadEventExecutorGroup.<init>(MultithreadEventExecutorGroup.java:84)
        at io.netty.util.concurrent.MultithreadEventExecutorGroup.<init>(MultithreadEventExecutorGroup.java:60)
        at io.netty.util.concurrent.MultithreadEventExecutorGroup.<init>(MultithreadEventExecutorGroup.java:49)
        at io.netty.channel.MultithreadEventLoopGroup.<init>(MultithreadEventLoopGroup.java:59)
        at io.netty.channel.epoll.EpollEventLoopGroup.<init>(EpollEventLoopGroup.java:117)
        at io.netty.channel.epoll.EpollEventLoopGroup.<init>(EpollEventLoopGroup.java:104)
        at io.netty.channel.epoll.EpollEventLoopGroup.<init>(EpollEventLoopGroup.java:81)
        at play.core.server.NettyServer.<init>(NettyServer.scala:111)
        at play.core.server.NettyServerProvider.createServer(NettyServer.scala:430)
        at play.core.server.ServerProvider.createServer(ServerProvider.scala:30)
        at play.core.server.ServerProvider.createServer$(ServerProvider.scala:27)
        at play.core.server.NettyServerProvider.createServer(NettyServer.scala:422)
        at play.core.server.ProdServerStart$.start(ProdServerStart.scala:58)
        at play.core.server.ProdServerStart$.main(ProdServerStart.scala:28)
        at play.core.server.ProdServerStart.main(ProdServerStart.scala)
Caused by: java.lang.UnsatisfiedLinkError: could not load a native library: netty_transport_native_epoll_aarch_64
        at io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:239)
        at io.netty.channel.epoll.Native.loadNativeLibrary(Native.java:323)
        at io.netty.channel.epoll.Native.<clinit>(Native.java:85)
        at io.netty.channel.epoll.Epoll.<clinit>(Epoll.java:40)
        ... 18 more
        Suppressed: java.lang.UnsatisfiedLinkError: could not load a native library: netty_transport_native_epoll
                at io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:239)
                at io.netty.channel.epoll.Native.loadNativeLibrary(Native.java:326)
                ... 20 more
        Caused by: java.io.FileNotFoundException: META-INF/native/libnetty_transport_native_epoll.so
                at io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:186)
                ... 21 more
                Suppressed: java.lang.UnsatisfiedLinkError: no netty_transport_native_epoll in java.library.path: [/usr/java/packages/lib, /lib, /usr/lib, /usr/lib64, /lib64]
                        at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2678)
                        at java.base/java.lang.Runtime.loadLibrary0(Runtime.java:830)
                        at java.base/java.lang.System.loadLibrary(System.java:1890)
                        at io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
                        at io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:396)
                        at io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:161)
                        ... 21 more
                        Suppressed: java.lang.UnsatisfiedLinkError: no netty_transport_native_epoll in java.library.path: [/usr/java/packages/lib, /lib, /usr/lib, /usr/lib64, /lib64]
                                at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2678)
                                at java.base/java.lang.Runtime.loadLibrary0(Runtime.java:830)
                                at java.base/java.lang.System.loadLibrary(System.java:1890)
                                at io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
                                at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
                                at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
                                at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
                                at java.base/java.lang.reflect.Method.invoke(Method.java:566)
                                at io.netty.util.internal.NativeLibraryLoader$1.run(NativeLibraryLoader.java:430)
                                at java.base/java.security.AccessController.doPrivileged(Native Method)
                                at io.netty.util.internal.NativeLibraryLoader.loadLibraryByHelper(NativeLibraryLoader.java:422)
                                at io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:388)
                                ... 22 more
Caused by: java.io.FileNotFoundException: META-INF/native/libnetty_transport_native_epoll_aarch_64.so
        at io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:186)
        ... 21 more
        Suppressed: java.lang.UnsatisfiedLinkError: no netty_transport_native_epoll_aarch_64 in java.library.path: [/usr/java/packages/lib, /lib, /usr/lib, /usr/lib64, /lib64]
                at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2678)
                at java.base/java.lang.Runtime.loadLibrary0(Runtime.java:830)
                at java.base/java.lang.System.loadLibrary(System.java:1890)
                at io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
                at io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:396)
                at io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:161)
                ... 21 more
                Suppressed: java.lang.UnsatisfiedLinkError: no netty_transport_native_epoll_aarch_64 in java.library.path: [/usr/java/packages/lib, /lib, /usr/lib, /usr/lib64, /lib64]
                        at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2678)
                        at java.base/java.lang.Runtime.loadLibrary0(Runtime.java:830)
                        at java.base/java.lang.System.loadLibrary(System.java:1890)
                        at io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
                        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
                        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
                        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
                        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
                        at io.netty.util.internal.NativeLibraryLoader$1.run(NativeLibraryLoader.java:430)
                        at java.base/java.security.AccessController.doPrivileged(Native Method)
                        at io.netty.util.internal.NativeLibraryLoader.loadLibraryByHelper(NativeLibraryLoader.java:422)
                        at io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:388)
                        ... 22 more

The solution is quite simple, just adding the linux-aarch_64 native library as dependency works:

2024-02-26 13:39:07 DEBUG i.n.u.internal.NativeLibraryLoader  -Dio.netty.native.workdir: /tmp (io.netty.tmpdir)
2024-02-26 13:39:07 DEBUG i.n.u.internal.NativeLibraryLoader  -Dio.netty.native.deleteLibAfterLoading: true
2024-02-26 13:39:07 DEBUG i.n.u.internal.NativeLibraryLoader  -Dio.netty.native.tryPatchShadedId: true
2024-02-26 13:39:07 DEBUG i.n.u.internal.NativeLibraryLoader  -Dio.netty.native.detectNativeLibraryDuplicates: true
2024-02-26 13:39:07 DEBUG i.n.u.internal.NativeLibraryLoader  Successfully loaded the library /tmp/libnetty_transport_native_epoll_aarch_6412027135138498234731.so

I also added the RISC-V native library, because it's also available now, just since December:

• Add support for RISC-V netty/netty#13670

Currently no other native transports (on Linux) are available: https://repo1.maven.org/maven2/io/netty/netty-transport-native-epoll/4.1.107.Final/

Adding those native transport does not do any harm (besides occupying a couple of bytes on the disk if not used), but make the out of the box experience of play.server.netty.transport = "native" much nicer. (Only the library for the used architecture gets loaded: https://github.com/netty/netty/blob/netty-4.1.107.Final/transport-classes-epoll/src/main/java/io/netty/channel/epoll/Native.java#L319-L332 - of course, because the others would fail to load if trying to load on the wrong architecture)

More links:

• https://netty.io/wiki/native-transports.html
• pom.xml: adding suuport for AARCH64 architecture in Netty/Transport/N… netty/netty#7933

[mkurz]

@Mergifyio backport 2.9.x

[mergify]

backport 2.9.x

✅ Backports have been created

• #12406 [2.9.x] Make Netty native transport work on Linux AArch64 and RISC-V (besides x86-64) (backport #12405) by @mkurz has been created for branch 2.9.x but encountered conflicts