getTracer() fails because SenderFactory and ThriftSenderFactory are loaded from different ClassLoaders

[dbuchhorn]

In our project we have an ear which contains multiple jars and a seperated web service war. Both are deployed in a WildFly 13.
In the ear there is a api.jar which contains interfaces and other classes which should be visible for other deployments and a core.jar with implementations of the api.jar.
Through pom dependencies and jboss-deployment-structure.xml the war can use the classes from api.jar. In the ear there is also a lib directory with libraries used the jars in the ear, for example the libraries of opentracing and jaeger.
The war also has dependencies to the jaeger libraries.

All worked fine until we tried to update the jaeger libraries from version 0.27.0 to 0.30.4.

When we call the web service the following happens. The call gets to a web filter which among other things sets up a jaeger tracer for logging. This happens in the core.jar. We use the "Configuration via Environment" way to obtain a tracer. But since the new version we get this exception:

ERROR [io.undertow.request] (default task-1) UT005023: Exception handling request to /TestWebService/hello: java.util.ServiceConfigurationError: io.jaegertracing.spi.SenderFactory: Provider io.jaegertracing.thrift.internal.senders.ThriftSenderFactory not a subtype
	at java.util.ServiceLoader.fail(ServiceLoader.java:239)
	at java.util.ServiceLoader.access$300(ServiceLoader.java:185)
	at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:376)
	at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
	at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
	at io.jaegertracing.internal.senders.SenderResolver.resolve(SenderResolver.java:59)
	at io.jaegertracing.Configuration$SenderConfiguration.getSender(Configuration.java:634)
	at io.jaegertracing.Configuration$ReporterConfiguration.getReporter(Configuration.java:520)
	at io.jaegertracing.Configuration$ReporterConfiguration.access$000(Configuration.java:480)
	at io.jaegertracing.Configuration.getTracerBuilder(Configuration.java:214)
	at io.jaegertracing.Configuration.getTracer(Configuration.java:230)

From version 0.29.0 to 0.30.0 a change was made to io.jaegertracing.Configuration.SenderConfiguration.getSender() method which now calls the SenderResolver.resolve(SenderConfiguration) method. In this method a java.util.ServiceLoader is used to load available implementations of io.jaegertracing.spi.SenderFactory, the ServiceLoader is created like this: ServiceLoader.load(SenderFactory.class). Because this happens in the core.jar and uses libraries of the ear the SenderFactory class is loaded with the class loader of the ear. The ServiceLoader then does find the class io.jaegertracing.thrift.internal.senders.ThriftSenderFactory as an implementation of SenderFactory BUT this class is loaded by the class loader of the war. That is because the method ServiceLoader.load(SenderFactory.class) uses Thread.currentThread().getContextClassLoader() as loader. The current thread originally was started in the web service deployment.

A possible suggestion could be to call ServiceLoader.load(SenderFactory.class, SenderFactory.class.getClassLoader()) in SenderResolver.resolve(SenderConfiguration). But we don't know which impact this would have.

We found a workaround by calling Thread.currentThread().setContextClassLoader(SenderFactory.class.getClassLoader()) before we call getTracer() on the Configuration.

Thanks for your help

[jpkrohling]

Can you share your jboss-deployment-structure.xml and possibly your META-INF/MANIFEST? You might need to declare that services must be imported.

A possible suggestion could be to call ServiceLoader.load(SenderFactory.class, SenderFactory.class.getClassLoader()) in SenderResolver.resolve(SenderConfiguration). But we don't know which impact this would have.

+1, would you like to send a PR?

[jpkrohling]

By the way: I got a PR merged last week adding transparent Jaeger and OT support for WildFly. It'll be available from WF 14 Beta2 and on and I'm really interested in getting feedback about it.

https://github.com/wildfly/wildfly/blob/master/docs/src/main/asciidoc/_admin-guide/subsystem-configuration/MicroProfile_OpenTracing_SmallRye.adoc

[dbuchhorn]

We already declared the import of the services and through debugging I managed to see that META-INF/io.jaegertracing.spi.SenderFactory from jaeger-thrift is found and the class name io.jaegertracing.thrift.internal.senders.ThriftSenderFactory is read, but the problem is the class loader this class is then loaded with.
All jaeger libraries are in the lib directory of the ear file.

What stands PR for?

[jpkrohling]

problem is the class loader this class is then loaded with

Looks like you nailed it.

What stands PR for?

PR stands for pull request :) If you want to contribute with a fix, let me know and I can guide you.

[dbuchhorn]

PR created.