-
Notifications
You must be signed in to change notification settings - Fork 40.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add auto-configuration for OTLP span exporter #34508
Closed
Closed
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
71 changes: 71 additions & 0 deletions
71
...va/org/springframework/boot/actuate/autoconfigure/tracing/otlp/OtlpAutoConfiguration.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,71 @@ | ||
/* | ||
* Copyright 2012-2023 the original author or authors. | ||
* | ||
* Licensed under the Apache License, Version 2.0 (the "License"); | ||
* you may not use this file except in compliance with the License. | ||
* You may obtain a copy of the License at | ||
* | ||
* https://www.apache.org/licenses/LICENSE-2.0 | ||
* | ||
* Unless required by applicable law or agreed to in writing, software | ||
* distributed under the License is distributed on an "AS IS" BASIS, | ||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
* See the License for the specific language governing permissions and | ||
* limitations under the License. | ||
*/ | ||
|
||
package org.springframework.boot.actuate.autoconfigure.tracing.otlp; | ||
|
||
import java.util.Map.Entry; | ||
|
||
import io.micrometer.tracing.otel.bridge.OtelTracer; | ||
import io.opentelemetry.api.OpenTelemetry; | ||
import io.opentelemetry.exporter.otlp.http.trace.OtlpHttpSpanExporter; | ||
import io.opentelemetry.exporter.otlp.http.trace.OtlpHttpSpanExporterBuilder; | ||
import io.opentelemetry.sdk.trace.SdkTracerProvider; | ||
|
||
import org.springframework.boot.actuate.autoconfigure.tracing.ConditionalOnEnabledTracing; | ||
import org.springframework.boot.autoconfigure.AutoConfiguration; | ||
import org.springframework.boot.autoconfigure.EnableAutoConfiguration; | ||
import org.springframework.boot.autoconfigure.condition.ConditionalOnClass; | ||
import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean; | ||
import org.springframework.boot.context.properties.EnableConfigurationProperties; | ||
import org.springframework.context.annotation.Bean; | ||
|
||
/** | ||
* {@link EnableAutoConfiguration Auto-configuration} for OTLP. Brave does not support | ||
* OTLP, so we only configure it for OpenTelemetry. OTLP defines three transports that are | ||
* supported: gRPC (/protobuf), HTTP/protobuf, HTTP/JSON. From these transports HTTP/JSON | ||
* is not supported by the OTel Java SDK, and it seems there are no plans supporting it in | ||
* the future, see: <a href= | ||
* "https://github.com/open-telemetry/opentelemetry-java/issues/3651">opentelemetry-java#3651</a>. | ||
* Because this class configures components from the OTel SDK, it can't support HTTP/JSON. | ||
* To keep things simple, we only auto-configure HTTP/protobuf. If you want to use gRPC, | ||
* please disable this auto-configuration and create a bean. | ||
* | ||
* @author Jonatan Ivanov | ||
* @since 3.1.0 | ||
*/ | ||
@AutoConfiguration | ||
@ConditionalOnEnabledTracing | ||
@ConditionalOnClass({ OtelTracer.class, SdkTracerProvider.class, OpenTelemetry.class, OtlpHttpSpanExporter.class }) | ||
@EnableConfigurationProperties(OtlpProperties.class) | ||
public class OtlpAutoConfiguration { | ||
|
||
@Bean | ||
@ConditionalOnMissingBean | ||
OtlpHttpSpanExporter otlpHttpSpanExporter(OtlpProperties properties) { | ||
OtlpHttpSpanExporterBuilder builder = OtlpHttpSpanExporter.builder() | ||
.setEndpoint(properties.getEndpoint()) | ||
.setTimeout(properties.getTimeout()) | ||
.setCompression(properties.getCompression().name().toLowerCase()); | ||
|
||
for (Entry<String, String> header : properties.getHeaders().entrySet()) { | ||
builder.addHeader(header.getKey(), header.getValue()); | ||
} | ||
|
||
return builder.build(); | ||
|
||
} | ||
|
||
} |
103 changes: 103 additions & 0 deletions
103
...main/java/org/springframework/boot/actuate/autoconfigure/tracing/otlp/OtlpProperties.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,103 @@ | ||
/* | ||
* Copyright 2012-2023 the original author or authors. | ||
* | ||
* Licensed under the Apache License, Version 2.0 (the "License"); | ||
* you may not use this file except in compliance with the License. | ||
* You may obtain a copy of the License at | ||
* | ||
* https://www.apache.org/licenses/LICENSE-2.0 | ||
* | ||
* Unless required by applicable law or agreed to in writing, software | ||
* distributed under the License is distributed on an "AS IS" BASIS, | ||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
* See the License for the specific language governing permissions and | ||
* limitations under the License. | ||
*/ | ||
|
||
package org.springframework.boot.actuate.autoconfigure.tracing.otlp; | ||
|
||
import java.time.Duration; | ||
import java.util.HashMap; | ||
import java.util.Map; | ||
|
||
import org.springframework.boot.context.properties.ConfigurationProperties; | ||
|
||
/** | ||
* Configuration properties for {@link OtlpAutoConfiguration}. | ||
* | ||
* @author Jonatan Ivanov | ||
* @since 3.1.0 | ||
*/ | ||
@ConfigurationProperties("management.otlp.tracing") | ||
public class OtlpProperties { | ||
|
||
/** | ||
* URL to the OTel collector's HTTP API. | ||
*/ | ||
private String endpoint = "http://localhost:4318/v1/traces"; | ||
|
||
/** | ||
* Call timeout for the OTel Collector to process an exported batch of data. This | ||
* timeout spans the entire call: resolving DNS, connecting, writing the request body, | ||
* server processing, and reading the response body. If the call requires redirects or | ||
* retries all must complete within one timeout period. | ||
*/ | ||
private Duration timeout = Duration.ofSeconds(10); | ||
|
||
/** | ||
* The method used to compress the payload. | ||
*/ | ||
private Compression compression = Compression.NONE; | ||
|
||
/** | ||
* Custom HTTP headers you want to pass to the collector, for example auth headers. | ||
*/ | ||
private Map<String, String> headers = new HashMap<>(); | ||
|
||
public String getEndpoint() { | ||
return this.endpoint; | ||
} | ||
|
||
public void setEndpoint(String endpoint) { | ||
this.endpoint = endpoint; | ||
} | ||
|
||
public Duration getTimeout() { | ||
return this.timeout; | ||
} | ||
|
||
public void setTimeout(Duration timeout) { | ||
this.timeout = timeout; | ||
} | ||
|
||
public Compression getCompression() { | ||
return this.compression; | ||
} | ||
|
||
public void setCompression(Compression compression) { | ||
this.compression = compression; | ||
} | ||
|
||
public Map<String, String> getHeaders() { | ||
return this.headers; | ||
} | ||
|
||
public void setHeaders(Map<String, String> headers) { | ||
this.headers = headers; | ||
} | ||
|
||
enum Compression { | ||
|
||
/** | ||
* Gzip compression. | ||
*/ | ||
GZIP, | ||
|
||
/** | ||
* No compression. | ||
*/ | ||
NONE | ||
|
||
} | ||
|
||
} |
20 changes: 20 additions & 0 deletions
20
...c/main/java/org/springframework/boot/actuate/autoconfigure/tracing/otlp/package-info.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,20 @@ | ||
/* | ||
* Copyright 2012-2023 the original author or authors. | ||
* | ||
* Licensed under the Apache License, Version 2.0 (the "License"); | ||
* you may not use this file except in compliance with the License. | ||
* You may obtain a copy of the License at | ||
* | ||
* https://www.apache.org/licenses/LICENSE-2.0 | ||
* | ||
* Unless required by applicable law or agreed to in writing, software | ||
* distributed under the License is distributed on an "AS IS" BASIS, | ||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
* See the License for the specific language governing permissions and | ||
* limitations under the License. | ||
*/ | ||
|
||
/** | ||
* Auto-configuration for tracing with OTLP. | ||
*/ | ||
package org.springframework.boot.actuate.autoconfigure.tracing.otlp; |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
118 changes: 118 additions & 0 deletions
118
...mework/boot/actuate/autoconfigure/tracing/otlp/OtlpAutoConfigurationIntegrationTests.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,118 @@ | ||
/* | ||
* Copyright 2012-2023 the original author or authors. | ||
* | ||
* Licensed under the Apache License, Version 2.0 (the "License"); | ||
* you may not use this file except in compliance with the License. | ||
* You may obtain a copy of the License at | ||
* | ||
* https://www.apache.org/licenses/LICENSE-2.0 | ||
* | ||
* Unless required by applicable law or agreed to in writing, software | ||
* distributed under the License is distributed on an "AS IS" BASIS, | ||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
* See the License for the specific language governing permissions and | ||
* limitations under the License. | ||
*/ | ||
|
||
package org.springframework.boot.actuate.autoconfigure.tracing.otlp; | ||
|
||
import java.io.IOException; | ||
import java.nio.charset.StandardCharsets; | ||
import java.util.concurrent.TimeUnit; | ||
|
||
import io.micrometer.tracing.Tracer; | ||
import io.opentelemetry.exporter.otlp.http.trace.OtlpHttpSpanExporter; | ||
import io.opentelemetry.sdk.common.CompletableResultCode; | ||
import io.opentelemetry.sdk.trace.export.SpanExporter; | ||
import okhttp3.mockwebserver.MockResponse; | ||
import okhttp3.mockwebserver.MockWebServer; | ||
import okhttp3.mockwebserver.RecordedRequest; | ||
import okio.Buffer; | ||
import okio.GzipSource; | ||
import org.junit.jupiter.api.AfterEach; | ||
import org.junit.jupiter.api.BeforeEach; | ||
import org.junit.jupiter.api.Test; | ||
|
||
import org.springframework.boot.actuate.autoconfigure.observation.ObservationAutoConfiguration; | ||
import org.springframework.boot.actuate.autoconfigure.tracing.MicrometerTracingAutoConfiguration; | ||
import org.springframework.boot.actuate.autoconfigure.tracing.OpenTelemetryAutoConfiguration; | ||
import org.springframework.boot.autoconfigure.AutoConfigurations; | ||
import org.springframework.boot.test.context.runner.ApplicationContextRunner; | ||
|
||
import static org.assertj.core.api.Assertions.assertThat; | ||
|
||
/** | ||
* Integration tests for {@link OtlpAutoConfiguration}. | ||
* | ||
* @author Jonatan Ivanov | ||
*/ | ||
class OtlpAutoConfigurationIntegrationTests { | ||
|
||
private final ApplicationContextRunner contextRunner = new ApplicationContextRunner() | ||
.withPropertyValues("management.tracing.sampling.probability=1.0") | ||
.withConfiguration( | ||
AutoConfigurations.of(ObservationAutoConfiguration.class, MicrometerTracingAutoConfiguration.class, | ||
OpenTelemetryAutoConfiguration.class, OtlpAutoConfiguration.class)); | ||
|
||
private MockWebServer mockWebServer; | ||
|
||
@BeforeEach | ||
void setUp() throws IOException { | ||
this.mockWebServer = new MockWebServer(); | ||
this.mockWebServer.start(); | ||
} | ||
|
||
@AfterEach | ||
void tearDown() throws IOException { | ||
this.mockWebServer.close(); | ||
} | ||
|
||
@Test | ||
void httpSpanExporterShouldUseProtoBufAndNoCompression() { | ||
this.mockWebServer.enqueue(new MockResponse()); | ||
this.contextRunner | ||
.withPropertyValues("management.otlp.tracing.endpoint=http://localhost:%d/v1/traces" | ||
.formatted(this.mockWebServer.getPort()), "management.otlp.tracing.headers.custom=42") | ||
.run((context) -> { | ||
context.getBean(Tracer.class).nextSpan().name("test").end(); | ||
assertThat(context.getBean(OtlpHttpSpanExporter.class).flush()) | ||
.isSameAs(CompletableResultCode.ofSuccess()); | ||
|
||
RecordedRequest request = this.mockWebServer.takeRequest(10, TimeUnit.SECONDS); | ||
assertThat(request).isNotNull(); | ||
assertThat(request.getRequestLine()).contains("/v1/traces"); | ||
assertThat(request.getHeader("Content-Type")).isEqualTo("application/x-protobuf"); | ||
assertThat(request.getHeader("custom")).isEqualTo("42"); | ||
assertThat(request.getBodySize()).isPositive(); | ||
try (Buffer body = request.getBody()) { | ||
assertThat(body.readString(StandardCharsets.UTF_8)).contains("org.springframework.boot"); | ||
} | ||
}); | ||
} | ||
|
||
@Test | ||
void httpSpanExporterShouldUseProtoBufAndGzip() { | ||
this.mockWebServer.enqueue(new MockResponse()); | ||
this.contextRunner | ||
.withPropertyValues("management.otlp.tracing.compression=GZIP", | ||
"management.otlp.tracing.endpoint=http://localhost:%d/test".formatted(this.mockWebServer.getPort())) | ||
.run((context) -> { | ||
assertThat(context).hasSingleBean(OtlpHttpSpanExporter.class).hasSingleBean(SpanExporter.class); | ||
context.getBean(Tracer.class).nextSpan().name("test").end(); | ||
assertThat(context.getBean(OtlpHttpSpanExporter.class).flush()) | ||
.isSameAs(CompletableResultCode.ofSuccess()); | ||
|
||
RecordedRequest request = this.mockWebServer.takeRequest(10, TimeUnit.SECONDS); | ||
assertThat(request).isNotNull(); | ||
assertThat(request.getRequestLine()).contains("/test"); | ||
assertThat(request.getHeader("Content-Type")).isEqualTo("application/x-protobuf"); | ||
assertThat(request.getHeader("Content-Encoding")).isEqualTo("gzip"); | ||
assertThat(request.getBodySize()).isPositive(); | ||
try (Buffer unCompressed = new Buffer(); Buffer body = request.getBody()) { | ||
unCompressed.writeAll(new GzipSource(body)); | ||
assertThat(unCompressed.readString(StandardCharsets.UTF_8)).contains("org.springframework.boot"); | ||
} | ||
}); | ||
} | ||
|
||
} |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi @jonatan-ivanov , I see that spring is diverging from the configuration properties proposed by the otel-sdk. For instance, the properties managed by this class maps to otel-sdk "otel.exporter.otlp.endpoint", "otel.exporter.otlp.compression" and so on...
Are there any plans to converge the config property names to the same used by otel-sdk ?
If no, isnt there another way to make them at least seem easier to map. E.g, spring version would only add the management.otlp prefix ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Here's the rationale behind the current structure: #30381 This PR follows that pattern.
There are at least two tricky things here:
otel.exporter.otlp.endpoint
does not really make that distinction.I think right now this is the easiest thing you can do to set these:
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we should figure out what would be the best and your feedback is apprechiated, right now this is what we have:
OtlpProperties
for metricsmanagement.otlp.metrics.export
(this PR hasmanagement.otlp.tracing
)url
(this PR calls itendpoint
)ZipkinProperties
for tracing (Brave/OTel)management.zipkin.tracing
(this PR follows that convention)otel.exporter.otlp
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fair enough, makes sense as users could possibly use 2 different otel-compatible backends: one for metrics and other for tracing
Im doing the opposite: trying to support the SDK props as they are more widely known, so end-users would be able to configure it based on SDK docs, but it is really hard to map all the spring props to the sdk props. If it was just a matter of prefixes, life would be easier. But I understand the complexity and the other scenarios that you mention. There's even a more complex one such as
management.tracing.sampling.probability=${otel.traces.sampler.arg}
- theotel.traces.sampler.arg
(urgh!) is a beast on its own as it includes inner properties on it depending on the sampler implementation. Perhaps somewhere in the docs we could have a section mapping the spring->otel-sdk properties.I personally like
management.otlp.tracing.export.http.endpoint
more.I also liked that you exposed all the OtlpHttpSpanExporterBuilder configurations, different from the
OpenTelemetryAutoConfiguration.otelSpanProcessor
which is using the BatchSpanProcessor internally and not offering the possibility to configure its propertyes such as batch/queue size, timeouts and so on...