java.net.MalformedURLException in executable jar with OSGi Connect (Atomos)

[fipro78]

I have created a minimal OSGi runtime example to see what is possible using OSGi R8 and OSGi Connect. Similar to the example created by @rotty3000 (https://github.com/rotty3000/osgi-config-aff)

When running the build (I used Java 17, so the projects are configured for Java 17 even though not directly needed), I get default executable jars for Equinox and Felix (two app projects) and also two executable jars that contain Atomos.

The executable jars that contain Atomos are not working. A java.net.MalformedURLException is thrown:

org.osgi.framework.BundleException: Error reading bundle content.
        at org.eclipse.osgi.storage.Storage.getContentConnection(Storage.java:623)
        at org.eclipse.osgi.storage.Storage.install(Storage.java:667)
        at org.eclipse.osgi.internal.framework.BundleContextImpl.installBundle(BundleContextImpl.java:182)
        at org.eclipse.osgi.internal.framework.BundleContextImpl.installBundle(BundleContextImpl.java:175)
        at aQute.launcher.Launcher.installEmbedded(Launcher.java:912)
        at aQute.launcher.Launcher.update(Launcher.java:629)
        at aQute.launcher.Launcher.activate(Launcher.java:574)
        at aQute.launcher.Launcher.launch(Launcher.java:403)
        at aQute.launcher.Launcher.run(Launcher.java:185)
        at aQute.launcher.Launcher.main(Launcher.java:161)
        at aQute.launcher.pre.EmbeddedLauncher.executeWithRunPath(EmbeddedLauncher.java:170)
        at aQute.launcher.pre.EmbeddedLauncher.findAndExecute(EmbeddedLauncher.java:119)
        at aQute.launcher.pre.EmbeddedLauncher.main(EmbeddedLauncher.java:52)
Caused by: java.net.MalformedURLException: no protocol: jar/org.apache.felix.gogo.command-1.1.2.jar
        at java.base/java.net.URL.<init>(URL.java:674)
        at java.base/java.net.URL.<init>(URL.java:569)
        at java.base/java.net.URL.<init>(URL.java:516)
        at org.eclipse.osgi.storage.Storage.createURL(Storage.java:663)
        at org.eclipse.osgi.storage.Storage.getContentConnection(Storage.java:648)
        at org.eclipse.osgi.storage.Storage.getContentConnection(Storage.java:619)
        ... 12 more

Once the executable jar is packaged using jlink, everything works as expected.

Attached my sample project. The same behaviour can be seen in the osgi-config-aff example.

org.fipro.osgi.runtime.evaluation.zip

[rotty3000]

thanks @fipro78 I will investigate.

[fipro78]

@rotty3000
Did you have time to investigate on this? IIUC it should work. Atomos is providing an example that shows a similar approach with an executable jar using Spring Loader https://github.com/apache/felix-atomos/tree/master/atomos.examples/atomos.examples.springloader

[rotty3000]

Sadly I have not :(

[fipro78]

@rotty3000 and @tjwatson

I noticed that the executable jar examples from the Atomos project talk about assumptions on how the included jars are located.

The Atomos Spring Loader Example assumes the jars are located in BOOT-INF/lib/. The Atomos Index Example assumes the jars are located in an atomos/ subfolder. Bndtools places the embedded jars in a /jar folder.

Does this make a difference with the Atomos framework?

[tjwatson]

I'm unsure how the executable JAR bnd creates is related to Atomos. Does it embed Atomos?

[fipro78]

Well, I have created an executable jar with bndtools that includes Atomos. So does @rotty3000 on his example for creating a custom runtime image using jlink. Interestingly the custom jre is working but the executable jar is not.

[rotty3000]

is this reproducible on my poc repo @fipro78 ?

[fipro78]

@rotty3000 yes

[pkriens]

@fipro78 I am coming in from the cold for this. However, if you make the smallest possible github repo that exhibits this then I can take a look.

[fipro78]

@pkriens
Need to see what I can do to create something smaller. But actually this is the configuration in my repo

https://github.com/fipro78/osgi_deployment_options/blob/main/org.fipro.service.app/atomos-equinox-app.bndrun

I will try to extract the necessary parts so you can focus on the topic. But I will need some time to do this.

[pkriens]

np, I mark this as waiting. If it gets flagged as stale by dependabot I will close it but then you can always reopen it.

[fipro78]

I have reduced my example to a minimum to be able to focus on this issue:
https://github.com/fipro78/atomos_bndlauncher_issue

After the build via Maven there are two executable jars. The one with Atomos is showing the above mentioned issue.

Please let me know if the example is simplified enough to focus on the issue. I hope it helps to solve it.

[pkriens]

Thanks, this made it clear.

What's happening is that your launching code uses service loader to make a ConnectFrameworkFactory & ModuleConnector available. This triggers bnd's launcher in the connect mode. I implemented this a couple of years ago to experiment. You're probably one of the first to run into this.

In the code, when we're in the connect mode, I somehow did bundle install differently. Looking at the code, I seem to only accept the jrt: protocol and I have no idea what that is anymore. If not the jrt protocol, I take the relative path in the Jar. This is what Eclipse does not like.

I will make a PR to always use the URL. However, the ModuleConnector might have special requirements. As far as O recall, the Module Connector handles the actual loading. If you know more details about those, let me know.

[fipro78]

@pkriens
Unfortunately I don't know details about the module connector. But hopefully @tjwatson can give some insights on this.

[pkriens]

thanks for the quick reply. Waiting for @tjwatson ...

[tjwatson]

I have not built the reproducer. But looking at the repo I am unclear how Atomos is being configured. It does not appear that atomos.content.install nor atomos.content.start framework properties are being set to false when creating the framework with the module connector. Without setting these then Atomos will auto-install and auto-start all discovered bundles in the environment.

Yet then we seem to have BND also installing them? I'm not really sure how this BND launcher integration with a ModuleConnector is supposed to work for any off the shelf ModuleConnector. The ModuleConnector needs to associate the location you are using to install a bundle with one of its ConnectModules. For Atomos this is controlled with the Atomos API, or you let Atomos auto-install the bundles. By default Atomos will auto-configure the connected locations as well as install all the connected locations into the framework at initialization time. But all the connected locations will have a prefix of atomos: by default. For example, if Atomos discovered two bundles on the classpath with URLs of file://bundles/b1.jar and file://bundles/b2.jar then I would expect two bundles to be installed at locations atomos:file://bundles/b1.jar and atomos:file://bundles/b2.jar. But now we seem to also have this BND code trying to install the bundles again, but BND will be using an unconnected location so the Framework has no choice but to try and read the content into its own storage area and load the bundle again, and if you do that then I would expect a BSN/version collision to happen and fail to install.

[pkriens]

@tjwatson thanks for the quick reply! This was code written long ago when I was experimenting with Android at the time.

The reason this code does not work @fipro78 is that it includes org.apache.felix.atomos on the -runpath. And looking at the setup I am now puzzled what you try to achieve?As Thomas says, you need a Module Connector that is knows how you load the bundles. I've no idea what org.apache.felix.atomos but it is very unlikely that it is compatible out of the box with the bnd launcher. You seem to have two captains on the ship.

So @fipro78, what are you trying to do here? It does not look executable Jars are applicable here?

[fipro78]

Hm, good question. I was following the example from @rotty3000 and created different variants for deployments, trying to reduce container size and compare the effect on startup time.

Using the executable jar I am able to create a custom jlink image. Even if atomos is included. Therefore I expected that the executable jar should also work. If that is not the case, also fine. Then my basic results are final and the statement that atomos in an executable jar is not working stays true and there is no possible solution. :)

[tjwatson]

Atomos absolutely can work in an executable JAR itself. There are currently two approaches, using an atomos index, or using the the spring loader:

https://github.com/apache/felix-atomos/blob/master/atomos.examples/atomos.examples.index/README.md
https://github.com/apache/felix-atomos/blob/master/atomos.examples/atomos.examples.springloader/README.md

Both cases have advantages over the way BND does executable JARS (at least with my understanding) because using the Atomos approach loads the content of the bundles directly from the executable JAR without having to extract the content to disk for the framework to load it.

[tjwatson]

I should mention that I prefer the spring loader approach because it leaves the bundle JARs intact and embedded into the final executable JAR.

[fipro78]

Sorry my statement was not precise enough. I meant atomos is not working in a executable jar with bnd. At least this is what I understand from the discussion. Although I don't really understand in detail why.

[rotty3000]

I should mention that I prefer the spring loader approach because it leaves the bundle JARs intact and embedded into the final executable JAR.

I wanted so badly to implement that for BND executable jars but there was so much framework stuff going on under the hood there...

[pkriens]

Hmm, things are slowly coming back to me. So bear with me. I remember I made a version that flattened all the JARs in the executable Jar and then had some special directories for bundles where it would store the manifest and resources.

I remember it was hard to do generally since it requires all the bundles to not have any overlapping content when you have a single class loader.

@fipro78 Atomos requires the format of the Jar to be in a special way and that it also likely puts some requirements on the bundles you use. We could provide an exporter for Atomos if there is interest in energy.

However, I think we can close this issue?

[fipro78]

I just wanted to proof things and thought it is an issue. I understand that it can't be fixed easily or even not at all and we would need something new.

From my point of view we can close this issue if my understanding is correct. Currently there is no need for further actions. If it becomes necessary we can open a new ticket.

Thanks for the investigation and the responses.

[pkriens]

ywlc.