You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
As a followup from #2663, I found there is a buffer overrun issue in the Jaeger exporter causing intermittent drops of trace batches. Traces appear randomly in Jaeger as incomplete or missing.
I apply the workaround in #2663 that configures MaxPacketSize in the Jaeger exporter to 1472 so to remain compatible with network interfaces with likely default 1500 MTU. While my application runs, errors randomly appear like:
2022/03/08 16:39:15 data does not fit within one UDP packet; size 1482, max 1472, spans 6
I've been able to "fix" with the following workaround patch:
diff --git a/exporters/jaeger/agent.go b/exporters/jaeger/agent.go
index d18d891a..b64522e5 100644
--- a/exporters/jaeger/agent.go
+++ b/exporters/jaeger/agent.go
@@ -242,7 +242,7 @@ func (a *agentClientUDP) flush(ctx context.Context, batch *gen.Batch) error {
func (a *agentClientUDP) calcSizeOfSerializedThrift(ctx context.Context, thriftStruct thrift.TStruct) (int, error) {
a.thriftBuffer.Reset()
err := thriftStruct.Write(ctx, a.thriftProtocol)
- return a.thriftBuffer.Len(), err
+ return a.thriftBuffer.Len() + 19, err
}
// Close implements Close() of io.Closer and closes the underlying UDP connection.
This is because I found that the actual packet size was always 19 bytes larger than the value returned by calcSizeOfSerializedThrift().
Build a trace with many spans with sufficient complexity that the spans must be sent in multiple batches. This increases the probability that the packet will be oversized. Consider this unit test:
Please disregard, I was not actually running latest versions of all the otel packages like I thought and so I did not have the recent fix from #2489. This resolved my issue.
Description
As a followup from #2663, I found there is a buffer overrun issue in the Jaeger exporter causing intermittent drops of trace batches. Traces appear randomly in Jaeger as incomplete or missing.
I apply the workaround in #2663 that configures
MaxPacketSize
in the Jaeger exporter to 1472 so to remain compatible with network interfaces with likely default 1500 MTU. While my application runs, errors randomly appear like:I've been able to "fix" with the following workaround patch:
This is because I found that the actual packet size was always 19 bytes larger than the value returned by
calcSizeOfSerializedThrift()
.Environment
Steps To Reproduce
Expected behavior
The text was updated successfully, but these errors were encountered: