Brave 4.18

Brave 4.18 packs in a lot of new features, notably a redesigned integration model for
servlet-based instrumentation, dependency linking for messaging spans, and a safer
customization implementation.

Much of this was driven by users testing Spring Cloud Sleuth 2.0, which is based
on Brave now. Many thanks and patience to the Spring Boot 2 users testing this.

There was also significant design discussions for all of this, individuals thanked as
we go along below.

Redo of integrations that layer over servlet

All servlet-based frameworks collaborate with TracingFilter vs replicating lifecycle
in framework-specific ways. Even when the replicated code is less than 100 lines, there
are a number of reasons explained below for this choice.

Problems with mixed instrumentation starting server spans

Before, frameworks stacked over servlet, such as JAX-RS and Spring WebMVC, had
their own full-stack instrumentation. Even if only a few dozen lines, starting
"server spans" independently per-framework caused issues in integration. For
example, some frameworks could be "cut off" prior to their handlers kicking in.
Some frameworks, notably JAX-RS had very poor quality traces due to the lack of
available hooks. Finally, duplicate instrumentation was possible, especially in
environments like Spring Boot which support multiple coexisting frameworks. This
could materialize as multiple "spans" for the same server request unless
configured very carefully.

Using brave.servlet.TracingFilter whenever possible

The Micrometer team had similar problems trying to
reliably measure web request latency and found normalizing at the Servlet layer
helpful. This makes sense as the servlet layer is the most reused and tested code.
We expect the least amount of new bugs there. Also, its pros and cons are well
understood, as is its configuration.

So, we now recommend using the normal brave.servlet.TracingFilter whenever
possible. We also provide brave.spring.webmvc.DelegatingTracingFilter for
those needing to bootstrap that way.

For example, here's a snippet from our legacy app example, which
sets up the tracing filter in XML.

  <!-- ContextLoaderListener makes sure delegating filters can read the application context -->
  <listener>
    <listener-class>org.springframework.web.context.ContextLoaderListener</listener-class>
  </listener>

  <!-- Add the delegate to the standard tracing filter and map it to all paths -->
  <filter>
    <filter-name>tracingFilter</filter-name>
    <filter-class>brave.spring.webmvc.DelegatingTracingFilter</filter-class>
  </filter>
  <filter-mapping>
    <filter-name>tracingFilter</filter-name>
    <url-pattern>/*</url-pattern>
  </filter-mapping>

Leveraging framework-specific data

While standardizing on Servlet where possible solves some problems, it doesn't
naturally know anything resting above it. For example, Jersey and Spring WebMVC
know about the controller that processed a request and the route that led to it.
We now employ lighter and less error-prone "span customizing" extensions for
frameworks that stack over servlet.

To make this concrete. Let's take the legacy application example. If just using the
normal servlet instrumentation, you'd see a trace like this:

The above is skimpy on details but tells you about what happened at the http abstraction. Let's
say you took that several year old application and added this bean to it.

  <mvc:interceptors>
    <bean class="brave.spring.webmvc.SpanCustomizingHandlerInterceptor" />
  </mvc:interceptors>

As you'll see below, we've coordinated the layers such that the "mvc" component can contribute
tags and even a better span name with no effort on your part.

This isn't limited to Spring applications; it is available for anything that leverages
servlet libraries, including our 1st party instrumentation and whatever you contribute next.

For brave-instrumentation-jaxrs2

• TracingContainerFilter -> SpanCustomizingContainerFilter

For brave-instrumentation-jersey-server

• TracingApplicationEventListener -> SpanCustomizingApplicationEventListener
  • Use TracingApplicationEventListener when your container is native Jersey

For brave-instrumentation-spring-webmvc

• TracingAsyncHandlerInterceptor -> SpanCustomizingAsyncHandlerInterceptor
• TracingHandlerInterceptor -> SpanCustomizingHandlerInterceptor
  • Note: SpanCustomizingHandlerInterceptor is only for Spring 2.5

For brave-instrumentation-sparkjava

• There is no span customizing hooks right now. Let us know if you need them.

Credits roll

We only made it this far due to extended design and review discussions. Thanks very
much to @jplock (dropwizard) @jkschneider (micrometer) @marcingrzejszczak (sleuth) @wu-sheng (skywalking) and indirectly @nicmunroe as wingtips code and docs around servlet are fantastic.

Big changes can be intimidating, so special thanks also to Zipkin regulars
@shakuzen and @zeagord for taking time to review design points.

Messaging spans default to make dependency links

Before, using brave-instrumentation-kafka or brave-instrumentation-spring-rabbit
did not affect the Zipkin dependency graph. In other words, you wouldn't see any links
between the app producing a message to a broker and from the broker to a consumer.
This configuration oversight led to some support questions. Thanks for reporting them!

You can now set the remoteServiceName builder property to indicate the name of
your broker. These default to "kafka" and "rabbitmq" respectively.

For example, if using default rabbit instrumentation, the following trace:

Shows itself routing through the rabbitmq (default name) broker in the dependency diagram:

Thanks notably to @shakuzen and @jonathan-lo for reviewing this change.

Tracer.currentSpanCustomizer() and Span.customizer() for safer customization

Our "user-safe" type is SpanCustomizer. This type will have zero changes for its
tenure in the version 4.x line. It also it has no lifecycle support which means it can't
abnormally end your traces. SpanCustomizer happened because we wanted an
api platform developers needed be worried about exposing to users or 3rd parties.

We noticed a glitch which was it was easy to accidentally expose the underlying
span to users (due to class hierarchy ex. Span is a subtype of SpanCustomizer).

We've now fixed this and encourage Tracer.currentSpanCustomizer() and
Span.customizer() to provide where needed.

Thanks @yschimke and @zeagord for reviewing this!

Future work will introduce UnsafeSpan, which can rely on these methods to
ensure higher performance in single-threaded scenarios such as synchronous
calls. Watch the corresponding issue for updates.