Skip to content

Commit

Permalink
fix(sinks): use uncompressed body size for bytes sent (#19060)
Browse files Browse the repository at this point in the history
* fix(sinks): use uncompressed body size for bytes sent

* also http service

* add upgrade guide entry
  • Loading branch information
dsmith3197 committed Jan 8, 2024
1 parent 54c49ba commit a1e39ad
Show file tree
Hide file tree
Showing 6 changed files with 11 additions and 6 deletions.
2 changes: 1 addition & 1 deletion src/sinks/appsignal/service.rs
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@ impl Service<AppsignalRequest> for AppsignalService {
Box::pin(async move {
let metadata = std::mem::take(request.metadata_mut());
http_service.ready().await?;
let bytes_sent = metadata.request_wire_size();
let bytes_sent = metadata.request_encoded_size();
let event_byte_size = metadata.into_events_estimated_json_encoded_byte_size();
let http_response = http_service.call(request).await?;
let event_status = if http_response.is_successful() {
Expand Down
2 changes: 1 addition & 1 deletion src/sinks/datadog/metrics/service.rs
Original file line number Diff line number Diff line change
Expand Up @@ -142,7 +142,7 @@ impl DriverResponse for DatadogMetricsResponse {
}

fn bytes_sent(&self) -> Option<usize> {
Some(self.request_metadata.request_wire_size())
Some(self.request_metadata.request_encoded_size())
}
}

Expand Down
2 changes: 1 addition & 1 deletion src/sinks/greptimedb/service.rs
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,7 @@ impl DriverResponse for GreptimeDBBatchOutput {
}

fn bytes_sent(&self) -> Option<usize> {
Some(self.metadata.request_wire_size())
Some(self.metadata.request_encoded_size())
}
}

Expand Down
2 changes: 1 addition & 1 deletion src/sinks/statsd/service.rs
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ impl DriverResponse for StatsdResponse {
}

fn bytes_sent(&self) -> Option<usize> {
Some(self.metadata.request_wire_size())
Some(self.metadata.request_encoded_size())
}
}

Expand Down
3 changes: 1 addition & 2 deletions src/sinks/util/http.rs
Original file line number Diff line number Diff line change
Expand Up @@ -745,11 +745,10 @@ where
fn call(&mut self, mut request: HttpRequest) -> Self::Future {
let mut http_service = self.batch_service.clone();

let raw_byte_size = request.payload.len();

// NOTE: By taking the metadata here, when passing the request to `call()` below,
// that function does not have access to the metadata anymore.
let metadata = std::mem::take(request.metadata_mut());
let raw_byte_size = metadata.request_encoded_size();
let events_byte_size = metadata.into_events_estimated_json_encoded_byte_size();

Box::pin(async move {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ Vector's 0.35.0 release includes **deprecations**:

and **potentially impactful changes**:

1. [Update `component_sent_bytes_total` to correctly report uncompressed bytes for all sinks](#component-sent-bytes-total)
1. [Update `component_received_bytes_total` to correctly report decompressed bytes for all sources](#component-received-bytes-total)

We cover them below to help you upgrade quickly:
Expand All @@ -33,6 +34,11 @@ existing behavior. In the next release, the config option will be updated to def

### Potentially impactful changes

#### Update `component_sent_bytes_total` to correctly report uncompressed bytes for all sinks {#component-sent-bytes-total}

The AppSignal, Datadog Metrics, GreptimeDB, GCP Cloud Monitoring, Honeycomb, and HTTP sinks now correctly
report uncompressed bytes, rather than compressed bytes, for the `component_sent_bytes_total` internal metric.

#### Update `component_received_bytes_total` to correctly report decompressed bytes for all sources {#component-received-bytes-total}

The Heroku Logs, HTTP Server, Prometheus Remote Write, and Splunk HEC sources now correctly report decompressed bytes,
Expand Down

0 comments on commit a1e39ad

Please sign in to comment.