OSGi native libraries take 3 #277

io7m · 2017-02-04T10:35:14Z

Hello again!

It seems that I may have spoken too soon at the end of #216: What I had was actually working by accident due to a quirk of the OS I was using at the time.

Currently, LWJGL still can't be made to work on OSGi due to the way it looks up native libraries. I'll try to walk through what happens.

Firstly, I'm working from the assumption that I'll be repackaging the existing LWJGL jar files such that they become OSGi bundles, and therefore native libraries relative to each bundle will be embedded into each bundle. So, for example, the libjemalloc.so library will be inserted into an lwjgl-jemalloc bundle. Because OSGi (in addition to languages like Ceylon, and systems such as JBoss modules) uses a "peer to peer" classloading model, each bundle is loaded in its own classloader. This means that code can't pick an essentially arbitrary classloader and try to load a library from that classloader as if it were a resource; the file will most likely not be accessible via that specific classloader.

With that in mind, let's follow the execution path of Library.loadSystem("lwjgl")...

        @SuppressWarnings("try")
	public static void loadSystem(String name) throws UnsatisfiedLinkError {
		apiLog("Loading library (system): " + name);
		if ( Paths.get(name).isAbsolute() ) {
			System.load(name);
			apiLog("\tSuccess");
			return;
		}

The isAbsolute path won't be taken, because "lwjgl" isn't absolute. Therefore...

                String libName = Platform.get().mapLibraryName(name);

		URL libURL = Library.class.getResource("/" + libName);
		if ( libURL == null ) {
			// Try org.lwjgl.librarypath
			if ( loadSystem(libName, Configuration.LIBRARY_PATH) )
				return;
		} else {

The returned libName will be liblwjgl.so on this platform. Due to the peer-to-peer classloading described above, libURL will be null, so we jump to another overload of loadSystem...

private static boolean loadSystem(String libName, Configuration<String> property) {
	String paths = property.get();
	return paths != null && loadSystem(libName, property.getProperty(), paths);
}

Now this is conditional based on the current configuration, but in an OSGi context, I would expect the property to be unset, therefore we return false here. As a result, we end up...

		// Then java.library.path
		String paths = System.getProperty(JAVA_LIBRARY_PATH);
		if ( paths != null ) {
			try {
				System.loadLibrary(name);

				Path libFile = findLibrary(paths, libName);
				if ( libFile != null ) {
					apiLog(String.format("\tLoaded from %s: %s", JAVA_LIBRARY_PATH, libFile));
					checkHash(libFile);
				}
				return;
			} catch (Throwable t) {
				apiLog(String.format("\t%s not found in %s", libName, JAVA_LIBRARY_PATH));
			}
		}

		printError(true);
		throw new UnsatisfiedLinkError("Failed to locate library: " + libName);

Now, again, in an OSGi context, we'd expect JAVA_LIBRARY_PATH not to be set. This means that the call to System.loadLibrary won't happen and an exception is raised. We can set JAVA_LIBRARY_PATH to some nonsense value such as /nonexistent, but this is pretty damn ugly and I've no idea if it would even work properly across platforms.

The situation is slightly worse for libraries loaded via loadNative because although most of the above logic is the same, the method doesn't fall back to System.loadLibrary and therefore the library will be loaded from the host's filesystem, even though we've bundled a guaranteed-compatible native library for libraries such as jemalloc and glfw!

I think I'd like to restate my position that the cleanest way to deal with all of this is to provide a property that can be enabled that simply bypasses all of the above and uses System.loadLibrary directly. I appreciate that you've had support problems with this in the past as it's not really traceable, but it's currently required to work in systems like OSGi and languages like Ceylon and is reliable in the context of those systems (because they have standardized, well-understood ways of including native code).

The text was updated successfully, but these errors were encountered:

Spasi · 2017-02-04T16:19:28Z

Now, again, in an OSGi context, we'd expect JAVA_LIBRARY_PATH not to be set. This means that the call to System.loadLibrary won't happen and an exception is raised.

This is a bug and easily fixable. The current implementation indeed assumes that in order for System.loadLibrary to succeed, java.library.path must not be empty. Obviously this is not the case with OSGi, where ClassLoader.findLibrary is used.

The situation is slightly worse for libraries loaded via loadNative because although most of the above logic is the same, the method doesn't fall back to System.loadLibrary and therefore the library will be loaded from the host's filesystem, even though we've bundled a guaranteed-compatible native library for libraries such as jemalloc and glfw!

This cannot be fixed with System.loadLibrary, because these libraries are not JNI libraries. As I said here, loading them with System.loadNative is required so that LWJGL has a handle to the native library and can resolve symbols.

In theory they could be built as JNI libraries, but this would defeat a very important feature of LWJGL: being able to replace those libraries with custom builds. You may want to do that for a few reasons: enable debugging or stricter validation (all LWJGL libraries are built for top performance), include a fix or feature not present in the LWJGL release, or simply because you don't trust the LWJGL binaries.

There are two ways to resolve this:

Do not use Bundle-NativeCode for such libraries. Treat them as resources instead of libraries and simply add them to the classpath. LWJGL will take care of extracting and loading them, just like ClassLoader.findLibrary does. This means losing the benefits of OSGi metadata.
Keep using Bundle-NativeCode, somehow force OSGi to extract the native library, figure out where the library was extracted and use Configuration.<libname>_LIBRARY_NAME options to set the corresponding absolute path.

I'm not sure how 2) could be implemented. Maybe the OSGi's ClassLoader.findLibrary could be called using reflection?

Spasi · 2017-02-04T20:39:29Z

Well, reflection works on Java 8, but not on Java 9 without --add-opens java.base/java.lang=ALL-UNNAMED.

Is there an OSGi API that can be used instead?

io7m · 2017-02-04T21:49:22Z

I'm still giving it some consideration, but I suspect that 2 is impossible. The reason being is that OSGi doesn't seem to have any APIs at all related to native code; the native code is supposed to be configured entirely via static metadata with the only interface being the standard Java loadLibrary methods. Additionally, each OSGi container implementation handles the actual loading of native libraries differently, so it's not really something we can predict.

I think handling the JNI libraries via System.loadLibrary once the java.library.path bug is fixed will be fine. As for the non-JNI libraries, I think it would be better to treat them as resources in the manner you suggested.

There is still the issue of actually accessing resources when there are multiple classloaders involved, but could this be as simple as just passing in a reference to the right classloader? For example, the code that loads libglfw could pass a reference to its own classloader to Library.loadNative and the passed in classloader would be responsible for returning the resource that LWJGL then unpacks.

…progress This lets the ClassLoader find the library if possible, useful for example with OSGi bundles.

…277 in progress

Spasi · 2017-02-04T22:42:19Z

Please try the next snapshot (3.1.2 build 14). The first issue should be fixed and I have added an option to enable System.loadLibrary emulation in Library.loadNative (using reflection, Java 9 caveat still applies).

There is still the issue of actually accessing resources when there are multiple classloaders involved, but could this be as simple as just passing in a reference to the right classloader?

Could you explain this a bit more? Do you mean that different LWJGL bindings are loaded from different ClassLoaders?

Currently LWJGL uses Library.class.getResource() to resolve shared libraries. The new System.loadLibrary emulation also uses Library.class.getClassLoader(). One solution would be to add an overload that accepts a user-specified Class and uses that instead of Library.

io7m · 2017-02-05T09:36:13Z

Please try the next snapshot (3.1.2 build 14).

I'll give it a go today, thanks!

Could you explain this a bit more? Do you mean that different LWJGL bindings are loaded from different ClassLoaders?

Let's assume that I create one bundle per LWJGL module. For example, I put lwjgl-jemalloc and all of its natives into one bundle, and then I put lwjgl and its natives into a separate bundle.

Then, JEmalloc calls Library.loadNative which then calls Library.class.getResource("/libjemalloc.so"); to locate the native library as a resource. The problem here is that Library is loaded by a different classloader than JEmalloc, because they're in different bundles. The resources of JEmalloc are not accessible via Library.

I've put together a small example of resource lookups across bundles here:

https://github.com/io7m/osgi-resource-lookups-20170205

There's a class in the loader module that presents an interface somewhat like Library.loadNative in that it tries to load a given resource. An example class in the module0 module tries various calls to the loader and prints the results. The loadFromLoaderClass("/module0.txt") → null case is the one that matches what LWJGL currently does: The com.io7m.cltest.module0.Example class asks the Loader class in a different bundle to load /module0.txt. It fails, because /module0.txt is not in the same bundle as Loader.

Going by the results obtained above, I think what needs to happen is for JEmalloc to pass its own class reference (JEmalloc.class) to Library.loadNative and that class reference is then used look up the resource. This matches the loadFromGivenClass(class com.io7m.cltest.module0.Example, "/module0.txt") and as you can see, it appears to work correctly.

One alternative to this stuff is to pack literally every LWJGL module into one large bundle. But the reasons for not doing this are the same reasons that you don't currently pack all of the LWJGL modules into one large jar the way you used to. 😄

io7m · 2017-02-05T15:30:45Z

A bit of experimentation appears to show that the JNI libraries are loaded correctly in an OSGi context in the 3.1.2 snapshot. 👍

I'm currently investigating a failure to load non-JNI libraries via the new reflection path which I think may actually be a bug in the OSGi runtime I'm using, but from the results I posted in the example code above, I suspect that there's an issue in the LWJGL code too: Loading should fail because you're invoking the findLibrary method on Library's classloader instead of the classloader of a possibly different bundle (such as the hypothetical lwjgl-jemalloc bundle).

https://github.com/LWJGL/lwjgl3/blob/master/modules/core/src/main/java/org/lwjgl/system/Library.java#L219

io7m · 2017-02-06T08:52:41Z

Nice! Any chance you could push a snapshot so that I can verify this?

Spasi · 2017-02-06T12:35:25Z

Sorry for the delay, the CI builds broke last night and it was too late to fix them. 3.1.2 build 15 is available now.

io7m · 2017-02-06T22:04:32Z

Thanks! I'll be giving this a go tomorrow. Couldn't get to it today. From reading the changes, I'm optimistic that it'll work.

io7m · 2017-02-07T12:25:00Z

Hi. Small issue discovered when testing the latest snapshot:

[INFO] --- maven-bundle-plugin:3.2.0:bundle (default-bundle) @ org.lwjgl.monolithic ---
[ERROR] Bundle com.io7m.bundles:org.lwjgl.monolithic:bundle:3.1.1 : Classes found in the wrong directory: {META-INF/versions/9/org/lwjgl/system/StackWalkUtil.class=org.lwjgl.system.StackWalkUtil}
[ERROR] Error(s) found in bundle configuration

This is a bug in the maven-bundle-plugin plugin, so my testing of the LWJGL changes may be delayed slightly until I can get that one fixed. Guess they aren't quite Java 9 ready yet...

io7m · 2017-02-07T13:12:57Z

Er, by "they", I mean the maven-bundle-plugin maintainers.

io7m · 2017-02-10T18:38:17Z

Hi!

Well, there's still something not quite right: It seems like in the code path I end up taking via OSGi, there are no calls made to either System.load() or System.loadLibrary() for libGL.so.1. Is this expected behaviour? I don't understand enough of the LWJGL internals to determine how it handles native code in all cases. It seems like either one of those calls would need to happen though in order for native methods to be resolved. If this isn't expected behaviour, I'll try to trace the sequence of calls and get more information. The other libraries such as libjemalloc.so, libglfw.so, etc, are loaded correctly and work properly.

The actual symptom I'm seeing is that everything appears to work correctly right up until the first call to GL11.glClearColor(...) whereupon I straight away receive:

[LWJGL] Loading library: libGL.so.1
[LWJGL] 	libGL.so.1 not found in org.lwjgl.librarypath=/tmp/lwjglsomeone/3.1.2-SNAPSHOT
[LWJGL] 	Loaded from java.library.path: /usr/lib64/libGL.so.1
[com.io7m.bundles.org.lwjgl.example.LWJGLExample][renderer]: clearing window
Exception in thread "renderer" java.lang.UnsatisfiedLinkError: org.lwjgl.opengl.GL11.glClearColor(FFFF)V
	at org.lwjgl.opengl.GL11.glClearColor(Native Method)
	at com.io7m.bundles.org.lwjgl.example.LWJGLExample.doExample(LWJGLExample.java:104)
	at java.lang.Thread.run(Thread.java:745)

Spasi · 2017-02-11T00:06:13Z

It seems like in the code path I end up taking via OSGi, there are no calls made to either System.load() or System.loadLibrary() for libGL.so.1. Is this expected behaviour?

Yes, libGL.so is loaded with Library.loadNative. It just so happens in your case that java.library.path includes /usr/lib64, so it is loaded via the code path that emulates System.loadLibrary.

The actual symptom I'm seeing is that everything appears to work correctly right up until the first call to GL11.glClearColor(...)

Has liblwjgl_opengl.so loaded correctly? The OpenGL bindings require both libGL and liblwjgl_opengl.

io7m · 2017-02-11T09:22:17Z

I don't get any errors when loading liblwjgl_opengl.so:

[LWJGL] Version: 3.1.2 SNAPSHOT
[LWJGL] 	 OS: Linux v4.8.13-1-ARCH
[LWJGL] 	JRE: 1.8.0_121 amd64
[LWJGL] 	JVM: OpenJDK 64-Bit Server VM v25.121-b13 by Oracle Corporation
[LWJGL] Loading library (system): lwjgl
[LWJGL] 	Using SharedLibraryLoader...
[LWJGL] 	Found at: /tmp/lwjglsomeone/3.1.2-SNAPSHOT/liblwjgl.so
[LWJGL] 	Loaded from org.lwjgl.librarypath: /tmp/lwjglsomeone/3.1.2-SNAPSHOT/liblwjgl.so
[LWJGL] MemoryUtil accessor: MemoryAccessorUnsafe
[LWJGL] Loading library: jemalloc
[LWJGL] 	Using SharedLibraryLoader...
[LWJGL] 	Found at: /tmp/lwjglsomeone/3.1.2-SNAPSHOT/libjemalloc.so
[LWJGL] 	Loaded from org.lwjgl.librarypath: /tmp/lwjglsomeone/3.1.2-SNAPSHOT/libjemalloc.so
[LWJGL] MemoryUtil allocator: JEmallocAllocator
[LWJGL] Loading library: glfw
[LWJGL] 	Using SharedLibraryLoader...
[LWJGL] 	Found at: /tmp/lwjglsomeone/3.1.2-SNAPSHOT/libglfw.so
[LWJGL] 	Loaded from org.lwjgl.librarypath: /tmp/lwjglsomeone/3.1.2-SNAPSHOT/libglfw.so
[com.io7m.bundles.org.lwjgl.example.LWJGLExample][renderer]: creating window
[LWJGL] Loading library (system): lwjgl_opengl
[LWJGL] 	Using SharedLibraryLoader...
[LWJGL] 	Found at: /tmp/lwjglsomeone/3.1.2-SNAPSHOT/liblwjgl_opengl.so
[LWJGL] 	Loaded from org.lwjgl.librarypath: /tmp/lwjglsomeone/3.1.2-SNAPSHOT/liblwjgl_opengl.so

Are there any methods I can call that would provoke an error had it not been loaded correctly?

Spasi · 2017-02-13T10:02:33Z

I cannot reproduce this. Could you please try a simple non-OSGi test on your system to see if it triggers the same error?

io7m · 2017-02-14T09:14:36Z

Oh, it works correctly outside of OSGi; that's never been an issue! Running the same code with the latest snapshot outside of OSGi works without issue.

I'm still trying to work out exactly what assumption the OSGi environment changes that LWJGL apparently depends on.

What I'm really looking for right now is a method that would raise some sort of error if and only if lwjgl_opengl hadn't been loaded correctly.

io7m · 2017-02-14T10:20:00Z

It appears to be a classloader issue. If I pack all of the LWJGL libraries and classes into a single bundle (so that they're all loaded via the same classloader), everything works correctly. This is less than ideal (and makes some of the work you did on LWJGL kind of pointless) but it's at least available as a last resort if I can't work out why LWJGL doesn't like multiple classloaders.

Spasi · 2017-02-14T10:55:23Z

The problem may be related to Reflection.getCallerClass() in System.loadLibrary. Even though the loading of lwjgl_opengl is initiated from GL.java, the class that makes the System.loadLibrary call is Library.java, which is in the core bundle, not the OpenGL one. But, if my understanding of OSGi is correct, would it fail to find lwjgl_opengl is that were the case? I.e. you would see an error earlier, not when calling glClearColor.

io7m · 2017-02-14T12:51:59Z

I'm not sure. I think I may need to enlist the help of someone who knows the OSGi internals a little better. I'll send a message to the osgi-dev mailing list.

Still not completely working due to LWJGL/lwjgl3#277, but this is certainly an improvement over the previous attempts.

io7m · 2017-02-14T13:01:49Z

https://mail.osgi.org/pipermail/osgi-dev/2017-February/006123.html

httpdigest · 2017-02-14T13:41:18Z

You are missing this in your org.lwjgl repo's pom.xml:

<repositories>
  <repository>
    <id>oss.sonatype.org</id>
    <url>https://oss.sonatype.org/content/repositories/snapshots/</url>
    <snapshots>
      <enabled>true</enabled>
    </snapshots>
  </repository>
</repositories>

People with a clean Maven local repository are otherwise not able to build your project.

io7m · 2017-02-14T13:42:12Z

Good point, I had this configured as a local profile. I'll add it to the pom.xml now.

Spasi · 2017-02-17T05:28:16Z

Could you please test if there's a difference with 3.1.2 build 16?

io7m · 2017-02-17T09:29:44Z

I'll give it a shot today, thanks!

io7m · 2017-02-17T13:41:38Z

No change, unfortunately, but I think your suspicion about Reflection.getCallerClass() is correct. Tracing the execution of the attempt to load lwjgl_opengl in the debugger shows that Runtime.load0 is called with the first argument set to Library.class despite the explicit passing around of class references such as GL.class:

I suspect that the only way to work around this is going to be by subverting package-private accessibility and calling the Runtime.load0 method reflectively.

Spasi · 2017-02-18T13:08:03Z

I suspect that the only way to work around this is going to be by subverting package-private accessibility and calling the Runtime.load0 method reflectively.

The code path that leads to Runtime.load0 is expected to fail under OSGi.

There was a bug in the previous commit, which I fixed. But then I realized that any reflection-based solution does not work under Java 9 with the default JVM settings. I ended up using method references so that System.loadLibrary is called in the context of the class that calls Library.loadSystem. Please try the next 3.1.2 build (17).

io7m · 2017-02-18T15:51:00Z

Still no luck with build 17, unfortunately.

The code path that leads to Runtime.load0 is expected to fail under OSGi.

This is actually the only code path that's taken when attempting to load lwjgl_opengl. Is this expected?

I've tried to build the bindings myself to try out a few things, but unfortunately they don't compile at the moment:

compile:
[javac: Core] Compiling 1256 source files to /home/someone/git/com.github/LWJGL/lwjgl3/bin/Core
[javac: Core] /home/someone/git/com.github/LWJGL/lwjgl3/modules/core/src/generated/java/org/lwjgl/nanovg/LibNanoVG.java:17: error: no suitable method found for loadSystem(Class<LibNanoVG>,String)
[javac: Core] 		Library.loadSystem(LibNanoVG.class, libName);
[javac: Core] 		       ^
[javac: Core]     method Library.loadSystem(String) is not applicable
[javac: Core]       (actual and formal argument lists differ in length)
[javac: Core]     method Library.loadSystem(Consumer<String>,Class<?>,String) is not applicable
[javac: Core]       (actual and formal argument lists differ in length)
[javac: Core]     method Library.loadSystem(Class<?>,String,Configuration<String>) is not applicable
[javac: Core]       (actual and formal argument lists differ in length)
[javac: Core]     method Library.loadSystem(Class<?>,String,String,String) is not applicable
[javac: Core]       (actual and formal argument lists differ in length)
[javac: Core] /home/someone/git/com.github/LWJGL/lwjgl3/modules/core/src/generated/java/org/lwjgl/nuklear/Nuklear.java:1182: error: no suitable method found for loadSystem(Class<Nuklear>,String)
[javac: Core] 	static { Library.loadSystem(Nuklear.class, Platform.mapLibraryNameBundled("lwjgl_nuklear")); }
[javac: Core] 	                ^
[javac: Core]     method Library.loadSystem(String) is not applicable
[javac: Core]       (actual and formal argument lists differ in length)
[javac: Core]     method Library.loadSystem(Consumer<String>,Class<?>,String) is not applicable
[javac: Core]       (actual and formal argument lists differ in length)
[javac: Core]     method Library.loadSystem(Class<?>,String,Configuration<String>) is not applicable
[javac: Core]       (actual and formal argument lists differ in length)
[javac: Core]     method Library.loadSystem(Class<?>,String,String,String) is not applicable
[javac: Core]       (actual and formal argument lists differ in length)
[javac: Core] /home/someone/git/com.github/LWJGL/lwjgl3/modules/core/src/generated/java/org/lwjgl/openvr/OpenVR.java:39: error: no suitable method found for loadSystem(Class<OpenVR>,String)
[javac: Core] 		Library.loadSystem(OpenVR.class, libName);
[javac: Core] 		       ^
[javac: Core]     method Library.loadSystem(String) is not applicable
[javac: Core]       (actual and formal argument lists differ in length)
[javac: Core]     method Library.loadSystem(Consumer<String>,Class<?>,String) is not applicable
[javac: Core]       (actual and formal argument lists differ in length)
[javac: Core]     method Library.loadSystem(Class<?>,String,Configuration<String>) is not applicable
[javac: Core]       (actual and formal argument lists differ in length)
[javac: Core]     method Library.loadSystem(Class<?>,String,String,String) is not applicable
[javac: Core]       (actual and formal argument lists differ in length)
[javac: Core] /home/someone/git/com.github/LWJGL/lwjgl3/modules/core/src/generated/java/org/lwjgl/stb/LibSTB.java:17: error: no suitable method found for loadSystem(Class<LibSTB>,String)
[javac: Core] 		Library.loadSystem(LibSTB.class, libName);
[javac: Core] 		       ^
[javac: Core]     method Library.loadSystem(String) is not applicable
[javac: Core]       (actual and formal argument lists differ in length)
[javac: Core]     method Library.loadSystem(Consumer<String>,Class<?>,String) is not applicable
[javac: Core]       (actual and formal argument lists differ in length)
[javac: Core]     method Library.loadSystem(Class<?>,String,Configuration<String>) is not applicable
[javac: Core]       (actual and formal argument lists differ in length)
[javac: Core]     method Library.loadSystem(Class<?>,String,String,String) is not applicable
[javac: Core]       (actual and formal argument lists differ in length)
[javac: Core] /home/someone/git/com.github/LWJGL/lwjgl3/modules/core/src/generated/java/org/lwjgl/util/lmdb/LMDB.java:327: error: no suitable method found for loadSystem(Class<LMDB>,String)
[javac: Core] 	static { Library.loadSystem(LMDB.class, Platform.mapLibraryNameBundled("lwjgl_lmdb")); }
[javac: Core] 	                ^
[javac: Core]     method Library.loadSystem(String) is not applicable
[javac: Core]       (actual and formal argument lists differ in length)
[javac: Core]     method Library.loadSystem(Consumer<String>,Class<?>,String) is not applicable
[javac: Core]       (actual and formal argument lists differ in length)
[javac: Core]     method Library.loadSystem(Class<?>,String,Configuration<String>) is not applicable
[javac: Core]       (actual and formal argument lists differ in length)
[javac: Core]     method Library.loadSystem(Class<?>,String,String,String) is not applicable
[javac: Core]       (actual and formal argument lists differ in length)
[javac: Core] /home/someone/git/com.github/LWJGL/lwjgl3/modules/core/src/generated/java/org/lwjgl/util/nfd/LibNFD.java:17: error: no suitable method found for loadSystem(Class<LibNFD>,String)
[javac: Core] 		Library.loadSystem(LibNFD.class, libName);
[javac: Core] 		       ^
[javac: Core]     method Library.loadSystem(String) is not applicable
[javac: Core]       (actual and formal argument lists differ in length)
[javac: Core]     method Library.loadSystem(Consumer<String>,Class<?>,String) is not applicable
[javac: Core]       (actual and formal argument lists differ in length)
[javac: Core]     method Library.loadSystem(Class<?>,String,Configuration<String>) is not applicable
[javac: Core]       (actual and formal argument lists differ in length)
[javac: Core]     method Library.loadSystem(Class<?>,String,String,String) is not applicable
[javac: Core]       (actual and formal argument lists differ in length)
[javac: Core] /home/someone/git/com.github/LWJGL/lwjgl3/modules/core/src/generated/java/org/lwjgl/util/par/LibPar.java:17: error: no suitable method found for loadSystem(Class<LibPar>,String)
[javac: Core] 		Library.loadSystem(LibPar.class, libName);
[javac: Core] 		       ^
[javac: Core]     method Library.loadSystem(String) is not applicable
[javac: Core]       (actual and formal argument lists differ in length)
[javac: Core]     method Library.loadSystem(Consumer<String>,Class<?>,String) is not applicable
[javac: Core]       (actual and formal argument lists differ in length)
[javac: Core]     method Library.loadSystem(Class<?>,String,Configuration<String>) is not applicable
[javac: Core]       (actual and formal argument lists differ in length)
[javac: Core]     method Library.loadSystem(Class<?>,String,String,String) is not applicable
[javac: Core]       (actual and formal argument lists differ in length)
[javac: Core] /home/someone/git/com.github/LWJGL/lwjgl3/modules/core/src/generated/java/org/lwjgl/util/simd/LibSSE.java:15: error: no suitable method found for loadSystem(Class<LibSSE>,String)
[javac: Core] 		Library.loadSystem(LibSSE.class, libName);
[javac: Core] 		       ^
[javac: Core]     method Library.loadSystem(String) is not applicable
[javac: Core]       (actual and formal argument lists differ in length)
[javac: Core]     method Library.loadSystem(Consumer<String>,Class<?>,String) is not applicable
[javac: Core]       (actual and formal argument lists differ in length)
[javac: Core]     method Library.loadSystem(Class<?>,String,Configuration<String>) is not applicable
[javac: Core]       (actual and formal argument lists differ in length)
[javac: Core]     method Library.loadSystem(Class<?>,String,String,String) is not applicable
[javac: Core]       (actual and formal argument lists differ in length)
[javac: Core] /home/someone/git/com.github/LWJGL/lwjgl3/modules/core/src/generated/java/org/lwjgl/util/tinyexr/TinyEXR.java:73: error: no suitable method found for loadSystem(Class<TinyEXR>,String)
[javac: Core] 	static { Library.loadSystem(TinyEXR.class, Platform.mapLibraryNameBundled("lwjgl_tinyexr")); }
[javac: Core] 	                ^
[javac: Core]     method Library.loadSystem(String) is not applicable
[javac: Core]       (actual and formal argument lists differ in length)
[javac: Core]     method Library.loadSystem(Consumer<String>,Class<?>,String) is not applicable
[javac: Core]       (actual and formal argument lists differ in length)
[javac: Core]     method Library.loadSystem(Class<?>,String,Configuration<String>) is not applicable
[javac: Core]       (actual and formal argument lists differ in length)
[javac: Core]     method Library.loadSystem(Class<?>,String,String,String) is not applicable
[javac: Core]       (actual and formal argument lists differ in length)
[javac: Core] /home/someone/git/com.github/LWJGL/lwjgl3/modules/core/src/generated/java/org/lwjgl/util/tinyfd/TinyFileDialogs.java:35: error: no suitable method found for loadSystem(Class<TinyFileDialogs>,String)
[javac: Core] 		Library.loadSystem(TinyFileDialogs.class, Platform.mapLibraryNameBundled("lwjgl_tinyfd"));
[javac: Core] 		       ^
[javac: Core]     method Library.loadSystem(String) is not applicable
[javac: Core]       (actual and formal argument lists differ in length)
[javac: Core]     method Library.loadSystem(Consumer<String>,Class<?>,String) is not applicable
[javac: Core]       (actual and formal argument lists differ in length)
[javac: Core]     method Library.loadSystem(Class<?>,String,Configuration<String>) is not applicable
[javac: Core]       (actual and formal argument lists differ in length)
[javac: Core]     method Library.loadSystem(Class<?>,String,String,String) is not applicable
[javac: Core]       (actual and formal argument lists differ in length)
[javac: Core] /home/someone/git/com.github/LWJGL/lwjgl3/modules/core/src/generated/java/org/lwjgl/util/xxhash/LibXXHash.java:17: error: no suitable method found for loadSystem(Class<LibXXHash>,String)
[javac: Core] 		Library.loadSystem(LibXXHash.class, libName);
[javac: Core] 		       ^
[javac: Core]     method Library.loadSystem(String) is not applicable
[javac: Core]       (actual and formal argument lists differ in length)
[javac: Core]     method Library.loadSystem(Consumer<String>,Class<?>,String) is not applicable
[javac: Core]       (actual and formal argument lists differ in length)
[javac: Core]     method Library.loadSystem(Class<?>,String,Configuration<String>) is not applicable
[javac: Core]       (actual and formal argument lists differ in length)
[javac: Core]     method Library.loadSystem(Class<?>,String,String,String) is not applicable
[javac: Core]       (actual and formal argument lists differ in length)
[javac: Core] /home/someone/git/com.github/LWJGL/lwjgl3/modules/core/src/generated/java/org/lwjgl/util/yoga/Yoga.java:309: error: no suitable method found for loadSystem(Class<Yoga>,String)
[javac: Core] 	static { Library.loadSystem(Yoga.class, Platform.mapLibraryNameBundled("lwjgl_yoga")); }
[javac: Core] 	                ^
[javac: Core]     method Library.loadSystem(String) is not applicable
[javac: Core]       (actual and formal argument lists differ in length)
[javac: Core]     method Library.loadSystem(Consumer<String>,Class<?>,String) is not applicable
[javac: Core]       (actual and formal argument lists differ in length)
[javac: Core]     method Library.loadSystem(Class<?>,String,Configuration<String>) is not applicable
[javac: Core]       (actual and formal argument lists differ in length)
[javac: Core]     method Library.loadSystem(Class<?>,String,String,String) is not applicable
[javac: Core]       (actual and formal argument lists differ in length)
[javac: Core] 12 errors

$ git log -n 1
commit 396b02333a77d54722182c99c73736b6f02bdf44
Author: Ioannis Tsakpinis <iotsakp@gmail.com>
Date:   Sat Feb 18 15:00:26 2017 +0200

    Update kotlinc to 1.1-rc

I'm curious what will happen if the following change is made to loadSystem but I can't try it locally due to the above compilation error:

    private static boolean loadSystem(Class<?> context, String libName, String property, String paths) {
        Path libFile = findLibrary(paths, libName);
        if (libFile == null) {
            apiLog(String.format("\t%s not found in %s=%s", libName, property, paths));
            return false;
        }

        try {
            final Method m = ClassLoader.class.getDeclaredMethod("loadLibrary", Class.class, String.class, boolean.class);
            m.setAccessible(true);
            m.invoke(null, context, libFile.toAbsolutePath().toString(), true);
        } catch (NoSuchMethodException | IllegalAccessException | InvocationTargetException e) {
            apiLog(String.format("\tError loading library (%s) via reflective ClassLoader call: %s %s", libName, e, e.getMessage()));
        }

        System.load(libFile.toAbsolutePath().toString());
        apiLog(String.format("\tLoaded from %s: %s", property, libFile));
        checkHash(context, libFile);
        return true;
    }

Spasi · 2017-02-18T16:12:46Z

This is actually the only code path that's taken when attempting to load lwjgl_opengl. Is this expected?

Oh, I see. No, this is not expected if you use OSGi bundles for all libraries. System.load is only used with absolute paths to shared libraries. In the screenshot above I see /tmp/lwjglsomeone/3.1.2-SNAPSHOT/, which is a path generated by the SharedLibraryLoader. In a pure OSGi run, that path shouldn't be used except for non-JNI libraries.

Not sure what is happening exactly, but I would recommend cleaning up your setup so that we can test each issue separately. Delete the snapshot folder and make sure the application is launched without -Djava.library.path or -Dorg.lwjgl.librarypath.

I've tried to build the bindings myself to try out a few things, but unfortunately they don't compile at the moment:

You need to regenerate the bindings. Running ant compile-templates should do it.

I'm curious what will happen if the following change is made to loadSystem

This is effectively what was done in the previous commit. As I said, it should have worked in theory (if it didn't have the bug), but I want to avoid reflection on java.lang internals as it is not permitted (by default) on Java 9. Also, the necessary calls are different on Android (that I need to worry about from now on).

io7m · 2017-02-18T18:36:54Z

Not sure what is happening exactly, but I would recommend cleaning up your setup so that we can test each issue separately. Delete the snapshot folder and make sure the application is launched without -Djava.library.path or -Dorg.lwjgl.librarypath.

Agreed. To be clear, I'm running the program with:

-Dorg.lwjgl.util.Debug=true -Dorg.lwjgl.system.EmulateSystemLoadLibrary=true -Dorg.lwjgl.util.DebugLoader=true -Xcheck:jni -verbose:jni

I have to run a Maven task to repackage the published LWJGL jars into OSGi bundles each time, so I'll be sure to routinely nuke ~/.m2/repository/org/lwjgl and ~/.m2/repository/com/io7m/bundles/ each time from now on.

You need to regenerate the bindings. Running ant compile-templates should do it.

Got it. Seemed like I needed an ant clean as well, for some reason.

I want to avoid reflection on java.lang internals as it is not permitted (by default) on Java 9.

Yes, definitely best to avoid this stuff if at all possible.

io7m · 2017-02-18T21:18:35Z

Hah, making the change I described instead gave me:

Exception in thread "renderer" java.lang.UnsatisfiedLinkError: Native Library /tmp/lwjglsomeone/3.1.2-SNAPSHOT/liblwjgl_opengl.so already loaded in another classloader
	at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1907)
	at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1824)
	at java.lang.Runtime.load0(Runtime.java:809)
	at java.lang.System.load(System.java:1086)
	at org.lwjgl.system.Library.loadSystem(Library.java:170)
	at org.lwjgl.system.Library.loadSystem(Library.java:152)
	at org.lwjgl.system.Library.loadSystem(Library.java:119)
	at org.lwjgl.opengl.GL.<clinit>(GL.java:76)
	at com.io7m.bundles.org.lwjgl.example.LWJGLExample.doExample(LWJGLExample.java:101)
	at java.lang.Thread.run(Thread.java:745)

I can't help but be amused by how awkward this problem is.

io7m · 2017-02-18T21:30:02Z

Eliminating the second call to System.load seems to work correctly:

private static boolean loadSystem(Class<?> context, String libName, String property, String paths) {
        Path libFile = findLibrary(paths, libName);
        if (libFile == null) {
            apiLog(String.format("\t%s not found in %s=%s", libName, property, paths));
            return false;
        }

        try {
            final Method m = ClassLoader.class.getDeclaredMethod("loadLibrary", Class.class, String.class, boolean.class);
            m.setAccessible(true);
            m.invoke(null, context, libFile.toAbsolutePath().toString(), true);
        } catch (NoSuchMethodException | IllegalAccessException | InvocationTargetException e) {
            apiLog(String.format("\tError loading library (%s) reflectively: %s %s", libName, e, e.getMessage()));
        }

        apiLog(String.format("\tLoaded from %s: %s", property, libFile));
        checkHash(context, libFile);
        return true;
    }

This code is wrong, obviously: Assuming that the call was going to be made reflectively, then presumably there should be an attempt made to call System.load if the reflective call failed.

Spasi · 2017-02-18T22:10:48Z

Again, this right here makes no sense to me: /tmp/lwjglsomeone/3.1.2-SNAPSHOT/liblwjgl_opengl.so.

Have you cleared that folder before testing the latest build? liblwjgl_opengl.so being there is only possible if LWJGL was able to find it in the classpath and extract it. That is NOT the case in an OSGi bundle (afaict) and that's what this entire issue is about.

System.load is called when:

The library name is an absolute path.
The library is found in java.library.path.
The library is found in org.lwjgl.librarypath.
The library is found in the classpath and extracted to the temp org.lwjgl.librarypath.

The lwjgl_opengl library is loaded in the static initializer of the GL class. Using the latest build, this is what should happen when running under OSGi:

The path is not absolute, so System.load is skipped.
GL.getResource("/liblwjgl_opengl.so") should return null, so the SharedLibraryLoader is not used.
org.lwjgl.librarypath is not set, so nothing is loaded from there.
Finally, System.loadLibrary is called, from a class that exists in the OpenGL module/bundle. Reflection.getCallerClass() should return the correct class, which then returns the correct ClassLoader, which then triggers OSGi's findLibrary.

Could you please verify the above?

io7m · 2017-02-19T09:50:45Z

GL.getResource("/liblwjgl_opengl.so") should return null, so the SharedLibraryLoader is not used.

This assumption is incorrect. Thanks to the change you made a couple of commits back:

        // METHOD 2: org.lwjgl.librarypath
        URL libURL = context.getResource("/" + libName);
        if (libURL == null) {
            if (loadSystem(context, libName, Configuration.LIBRARY_PATH))
                return;
        } else {
            // Always use the SLL if the library is found in the classpath,
            // so that newer versions can be detected.
            boolean debugLoader = Configuration.DEBUG_LOADER.get(false);
            try {
                if (debugLoader)
                    apiLog("\tUsing SharedLibraryLoader...");
                // Extract from classpath and try org.lwjgl.librarypath
                try (FileChannel ignored = SharedLibraryLoader.load(name, libName, libURL)) {
                    if (loadSystem(context, libName, Configuration.LIBRARY_PATH))
                        return;
                }
            } catch (Exception e) {
                if (debugLoader)
                    e.printStackTrace(DEBUG_STREAM);
            }
        }

For lwjgl_opengl, the returned URL looks like bundle://10.0:1/liblwjgl_opengl.so which is sufficient for OSGi to locate the resource and for LWJGL to extract it. A reference to the library is also present in the OSGi metadata so we still get the benefits of that.

io7m · 2017-02-19T15:40:52Z

To clarify: Yes, I'm deleting the temporary directory between each build/run.

I think the main issues up to this point have been:

Locating the native libraries as resources so that LWJGL could extract them. This was originally broken because LWJGL was assuming one global classloader but was fixed when you started passing around explicit class references.
Loading the extracted native libraries in the right class loader. This one's still an issue but works given my to-be-avoided reflective call to ClassLoader.loadLibrary. This one seems to be fixable if we can just get Reflection.getCallerClass() to return the right value via System.load. I think your previous changes would have worked, if not for the misunderstanding about which path the code would be taking on OSGi (which may've been my fault, perhaps I didn't communicate it clearly).

Spasi · 2017-02-19T17:04:38Z

Build 18 is up. The call sites are a bit ridiculous now, if this works I'll try to figure out a cleaner implementation when I have more time.

io7m · 2017-02-20T20:48:18Z

SUCCESS!

🎉

Nice work, thank you! This has certainly been one of the best responses I've had to a bug report in recent memory. It's very life-affirming when developers work to assist with use-cases that they themselves don't have.

Right now, my plan is to maintain a set of OSGi bundles published to Maven Central under the com.io7m.bundles groupId. I'd be happy to move these into a repos in the LWJGL organization (perhaps with packages published under a separate org.lwjgl.osgi groupId) and maintain them in perpetuity, if you want them. They're really just a set of Maven POMs that turn the published LWJGL jars into OSGi bundles. The nice thing about this is that it can be kept strictly separated from any existing build processes or systems that you have, and can be applied after-the-fact to LWJGL releases. This is actually how I produce Maven packages for JOGL, although there haven't been too many of those recently.

Thanks to LWJGL/lwjgl3#277, the bundles are now fully working with the latest LWJGL snapshots.

Spasi · 2017-02-21T09:46:39Z

Awesome! Thanks for the great feedback and for having the patience to go through this process.

I have created the lwjgl3-osgi repository and made you a collaborator. It's currently empty and you're welcome to push anything to it. When I have a better understanding of the OSGi packaging process, I'll let you know how to proceed (if we'll move the scripts to a different repo, what groupId to use on Maven etc).

Lets use the new repository for further discussions.

io7m · 2017-02-24T12:50:48Z

Filed a ticket for this, just for future reference: LWJGL/lwjgl3-osgi#4

Spasi · 2017-10-01T17:54:30Z

Hey @io7m, could you please verify that the latest snapshot (LWJGL 3.1.4 build 3) doesn't break the OSGi bundles? See here for details.

io7m · 2017-10-01T18:06:43Z

Doing it now!

io7m · 2017-10-01T18:44:16Z

It works correctly.

I had to introduce a couple of small bundle metadata fixes for OSGi containers that are running on Java 9 VMs, but other than that, the bundles continue to work on both OpenJDK 8 and OpenJDK 9.

Spasi added the Type: Bug label Feb 4, 2017

Spasi added a commit that referenced this issue Feb 4, 2017

Use System.loadLibrary even when java.library.path is empty. #277 in …

3e008c3

…progress This lets the ClassLoader find the library if possible, useful for example with OSGi bundles.

Spasi added a commit that referenced this issue Feb 4, 2017

Add experimental System.loadLibrary emulation to Library.loadNative. #…

355a8ab

…277 in progress

Spasi closed this as completed in ad0e93d Feb 6, 2017

io7m added a commit to io7m/org.lwjgl that referenced this issue Feb 14, 2017

Update bundle definitions

f675e32

Still not completely working due to LWJGL/lwjgl3#277, but this is certainly an improvement over the previous attempts.

Spasi added a commit that referenced this issue Feb 16, 2017

More ClassLoader fixes for #277

141c75b

Spasi added a commit that referenced this issue Feb 18, 2017

Refactor the fix for #277 to avoid reflection

4defb48

Spasi added a commit that referenced this issue Feb 19, 2017

Apply the #277 caller class fix to System.load too

cb145f6

io7m added a commit to io7m/org.lwjgl that referenced this issue Feb 21, 2017

Working bundles!

f018abf

Thanks to LWJGL/lwjgl3#277, the bundles are now fully working with the latest LWJGL snapshots.

saudet mentioned this issue Aug 16, 2019

Separate the Maven Plugin from JavaCPP for OSGi bytedeco/javacpp#328

Merged

saudet mentioned this issue Oct 23, 2019

Java API for onnxruntime microsoft/onnxruntime#2215

Merged

OSGi native libraries take 3 #277

OSGi native libraries take 3 #277

Comments

io7m commented Feb 4, 2017 • edited Loading

Spasi commented Feb 4, 2017

Spasi commented Feb 4, 2017

io7m commented Feb 4, 2017

Spasi commented Feb 4, 2017

io7m commented Feb 5, 2017

io7m commented Feb 5, 2017 • edited Loading

io7m commented Feb 6, 2017

Spasi commented Feb 6, 2017

io7m commented Feb 6, 2017

io7m commented Feb 7, 2017

io7m commented Feb 7, 2017

io7m commented Feb 10, 2017 • edited Loading

Spasi commented Feb 11, 2017

io7m commented Feb 11, 2017

Spasi commented Feb 13, 2017

io7m commented Feb 14, 2017

io7m commented Feb 14, 2017

Spasi commented Feb 14, 2017

io7m commented Feb 14, 2017

io7m commented Feb 14, 2017

httpdigest commented Feb 14, 2017

io7m commented Feb 14, 2017

Spasi commented Feb 17, 2017

io7m commented Feb 17, 2017

io7m commented Feb 17, 2017

Spasi commented Feb 18, 2017

io7m commented Feb 18, 2017

Spasi commented Feb 18, 2017

io7m commented Feb 18, 2017

io7m commented Feb 18, 2017

io7m commented Feb 18, 2017

Spasi commented Feb 18, 2017

io7m commented Feb 19, 2017

io7m commented Feb 19, 2017 • edited Loading

Spasi commented Feb 19, 2017

io7m commented Feb 20, 2017 • edited Loading

Spasi commented Feb 21, 2017

io7m commented Feb 24, 2017

Spasi commented Oct 1, 2017

io7m commented Oct 1, 2017

io7m commented Oct 1, 2017

io7m commented Feb 4, 2017 •

edited

Loading

io7m commented Feb 5, 2017 •

edited

Loading

io7m commented Feb 10, 2017 •

edited

Loading

io7m commented Feb 19, 2017 •

edited

Loading

io7m commented Feb 20, 2017 •

edited

Loading