fix(datadog_metrics sink): improve aggregation performance #18759

dsmith3197 · 2023-10-03T21:15:40Z

Description

The worst case time complexity for sort_and_collapse_counters_by_series_and_timestamp() was previously O(n^2). It is now O(n log n). This also adds concurrency to this step in the sink's pipeline, as it is a computationally expensive operation.

Note that this behavior is induced specifically when the event stream is counter heavy.

Regression Results

Changing the Algorithm

syslog_regex_logs2metric_ddmetrics | ingress throughput | +218.43 | [+217.02, +219.85] | 100.00%
-- | -- | -- | -- | --

Adding Concurrency

syslog_regex_logs2metric_ddmetrics | ingress throughput | +456.52 | [+455.41, +457.63] | 100.00%
-- | -- | -- | -- | --

netlify · 2023-10-03T21:15:44Z

✅ Deploy Preview for vrl-playground canceled.

Name	Link
🔨 Latest commit	`a1eb365`
🔍 Latest deploy log	https://app.netlify.com/sites/vrl-playground/deploys/651eff5a5f69fd000814cc54

netlify · 2023-10-03T21:15:44Z

✅ Deploy Preview for vector-project canceled.

Name	Link
🔨 Latest commit	`a1eb365`
🔍 Latest deploy log	https://app.netlify.com/sites/vector-project/deploys/651eff5ad5bb9600084846e7

dsmith3197 · 2023-10-03T21:15:54Z

/ci-run-regression

src/sinks/datadog/metrics/sink.rs

aws-amplify-us-east-1 · 2023-10-03T21:20:11Z

This pull request is automatically being deployed by Amplify Hosting (learn more).

Access this pull request here: https://pr-18759.d1a7j77663uxsc.amplifyapp.com

aws-amplify-us-east-1 · 2023-10-03T21:25:23Z

This pull request is automatically being deployed by Amplify Hosting (learn more).

Access this pull request here: https://pr-18759.d2lr4eds605rpz.amplifyapp.com

aws-amplify-us-east-1 · 2023-10-03T21:28:41Z

This pull request is automatically being deployed by Amplify Hosting (learn more).

Access this pull request here: https://pr-18759.d1hoyoksbulg25.amplifyapp.com

datadog-vectordotdev · 2023-10-03T21:41:34Z

Datadog Report

Branch report: dougsmith/datadog-metrics-aggregation
Commit report: e2626f4

✅ vector: 0 Failed, 0 New Flaky, 2200 Passed, 0 Skipped, 25m 59.4s Wall Time

github-actions · 2023-10-03T22:07:08Z

Regression Detector Results

Run ID: 4df2d344-34ae-4719-874c-b8563aa25cd7
Baseline: 7a55e54
Comparison: d6edd12
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine quickly if vector performance is changed and to what degree by a pull request.

Because a target's optimization goal performance in each experiment will vary somewhat each time it is run, we can only estimate mean differences in optimization goal relative to the baseline target. We express these differences as a percentage change relative to the baseline target, denoted "Δ mean %". These estimates are made to a precision that balances accuracy and cost control. We represent this precision as a 90.00% confidence interval denoted "Δ mean % CI": there is a 90.00% chance that the true value of "Δ mean %" is in that interval.

We decide whether a change in performance is a "regression" -- a change worth investigating further -- if both of the following two criteria are true:

The estimated |Δ mean %| ≥ 5.00%. This criterion intends to answer the question "Does the estimated change in mean optimization goal performance have a meaningful impact on your customers?". We assume that when |Δ mean %| < 5.00%, the impact on your customers is not meaningful. We also assume that a performance change in optimization goal is worth investigating whether it is an increase or decrease, so long as the magnitude of the change is sufficiently large.
Zero is not in the 90.00% confidence interval "Δ mean % CI" about "Δ mean %". This statement is equivalent to saying that there is at least a 90.00% chance that the mean difference in optimization goal is not zero. This criterion intends to answer the question, "Is there a statistically significant difference in mean optimization goal performance?". It also means there is no more than a 10.00% chance this criterion reports a statistically significant difference when the true difference in mean optimization goal is zero -- a "false positive". We assume you are willing to accept a 10.00% chance of inaccurately detecting a change in performance when no true difference exists.

The table below, if present, lists those experiments that have experienced a statistically significant change in mean optimization goal performance between baseline and comparison SHAs with 90.00% confidence OR have been detected as newly erratic. Negative values of "Δ mean %" mean that baseline is faster, whereas positive values of "Δ mean %" mean that comparison is faster. Results that do not exhibit more than a ±5.00% change in their mean optimization goal are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting change is observed.

No interesting changes in experiment optimization goals with confidence ≥ 90.00% and |Δ mean %| ≥ 5.00%.

Fine details of change detection per experiment.

experiment	goal	Δ mean %	Δ mean % CI	confidence
syslog_regex_logs2metric_ddmetrics	ingress throughput	+218.43	[+217.02, +219.85]	100.00%
syslog_log2metric_splunk_hec_metrics	ingress throughput	+3.65	[+3.45, +3.84]	100.00%
http_to_http_noack	ingress throughput	+2.67	[+2.48, +2.86]	100.00%
http_to_http_acks	ingress throughput	+2.19	[+2.04, +2.34]	100.00%
http_text_to_http_json	ingress throughput	+1.71	[+1.58, +1.84]	100.00%
datadog_agent_remap_blackhole_acks	ingress throughput	+1.57	[+1.43, +1.71]	100.00%
syslog_humio_logs	ingress throughput	+1.56	[+1.42, +1.69]	100.00%
otlp_http_to_blackhole	ingress throughput	+1.00	[+0.82, +1.19]	100.00%
splunk_hec_route_s3	ingress throughput	+0.99	[+0.46, +1.52]	99.80%
datadog_agent_remap_datadog_logs	ingress throughput	+0.90	[+0.76, +1.04]	100.00%
socket_to_socket_blackhole	ingress throughput	+0.69	[+0.59, +0.79]	100.00%
enterprise_http_to_http	ingress throughput	+0.48	[+0.28, +0.69]	99.99%
syslog_splunk_hec_logs	ingress throughput	+0.47	[+0.36, +0.57]	100.00%
syslog_log2metric_humio_metrics	ingress throughput	+0.45	[+0.30, +0.61]	100.00%
file_to_blackhole	egress throughput	+0.39	[-0.77, +1.56]	42.37%
datadog_agent_remap_datadog_logs_acks	ingress throughput	+0.20	[+0.07, +0.33]	98.70%
otlp_grpc_to_blackhole	ingress throughput	+0.11	[-0.02, +0.23]	84.44%
http_to_s3	ingress throughput	+0.03	[-0.26, +0.32]	13.13%
splunk_hec_to_splunk_hec_logs_acks	ingress throughput	+0.02	[-0.14, +0.18]	14.44%
http_to_http_json	ingress throughput	-0.00	[-0.05, +0.05]	0.88%
splunk_hec_indexer_ack_blackhole	ingress throughput	-0.00	[-0.16, +0.16]	1.05%
splunk_hec_to_splunk_hec_logs_noack	ingress throughput	-0.01	[-0.15, +0.13]	9.91%
datadog_agent_remap_blackhole	ingress throughput	-0.43	[-0.54, -0.32]	100.00%
syslog_loki	ingress throughput	-1.71	[-1.79, -1.62]	100.00%
fluent_elasticsearch	ingress throughput	-1.79	[-2.10, -1.48]	100.00%

dsmith3197 · 2023-10-04T14:42:15Z

/ci-run-regression

dsmith3197 · 2023-10-04T14:46:24Z

regression/cases/syslog_regex_logs2metric_ddmetrics/vector/vector.toml

@@ -27,7 +27,7 @@ type = "log_to_metric"
 inputs = ["remap"]

  [[transforms.log2metric.metrics]]
-  type = "gauge"
+  type = "counter"


Changed this from a gauge to a counter to induce the worst case behavior in terms of performance.

github-actions · 2023-10-04T15:38:18Z

Regression Detector Results

Run ID: 672fe15-22d5-48ea-aec0-6bad62555c31
Baseline: 7a55e54
Comparison: 8a9f9bf
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine quickly if vector performance is changed and to what degree by a pull request.

Because a target's optimization goal performance in each experiment will vary somewhat each time it is run, we can only estimate mean differences in optimization goal relative to the baseline target. We express these differences as a percentage change relative to the baseline target, denoted "Δ mean %". These estimates are made to a precision that balances accuracy and cost control. We represent this precision as a 90.00% confidence interval denoted "Δ mean % CI": there is a 90.00% chance that the true value of "Δ mean %" is in that interval.

We decide whether a change in performance is a "regression" -- a change worth investigating further -- if both of the following two criteria are true:

The estimated |Δ mean %| ≥ 5.00%. This criterion intends to answer the question "Does the estimated change in mean optimization goal performance have a meaningful impact on your customers?". We assume that when |Δ mean %| < 5.00%, the impact on your customers is not meaningful. We also assume that a performance change in optimization goal is worth investigating whether it is an increase or decrease, so long as the magnitude of the change is sufficiently large.
Zero is not in the 90.00% confidence interval "Δ mean % CI" about "Δ mean %". This statement is equivalent to saying that there is at least a 90.00% chance that the mean difference in optimization goal is not zero. This criterion intends to answer the question, "Is there a statistically significant difference in mean optimization goal performance?". It also means there is no more than a 10.00% chance this criterion reports a statistically significant difference when the true difference in mean optimization goal is zero -- a "false positive". We assume you are willing to accept a 10.00% chance of inaccurately detecting a change in performance when no true difference exists.

The table below, if present, lists those experiments that have experienced a statistically significant change in mean optimization goal performance between baseline and comparison SHAs with 90.00% confidence OR have been detected as newly erratic. Negative values of "Δ mean %" mean that baseline is faster, whereas positive values of "Δ mean %" mean that comparison is faster. Results that do not exhibit more than a ±5.00% change in their mean optimization goal are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting change is observed.

No interesting changes in experiment optimization goals with confidence ≥ 90.00% and |Δ mean %| ≥ 5.00%.

Fine details of change detection per experiment.

experiment	goal	Δ mean %	Δ mean % CI	confidence
syslog_regex_logs2metric_ddmetrics	ingress throughput	+456.52	[+455.41, +457.63]	100.00%
file_to_blackhole	egress throughput	+1.09	[-0.09, +2.27]	86.98%
http_to_http_acks	ingress throughput	+0.91	[+0.73, +1.09]	100.00%
syslog_splunk_hec_logs	ingress throughput	+0.70	[+0.61, +0.79]	100.00%
http_text_to_http_json	ingress throughput	+0.69	[+0.60, +0.78]	100.00%
syslog_humio_logs	ingress throughput	+0.39	[+0.27, +0.51]	100.00%
datadog_agent_remap_blackhole	ingress throughput	+0.35	[+0.24, +0.46]	100.00%
fluent_elasticsearch	ingress throughput	+0.20	[-0.11, +0.51]	71.97%
splunk_hec_to_splunk_hec_logs_acks	ingress throughput	+0.01	[-0.16, +0.18]	8.03%
splunk_hec_to_splunk_hec_logs_noack	ingress throughput	+0.01	[-0.14, +0.15]	5.10%
splunk_hec_indexer_ack_blackhole	ingress throughput	+0.00	[-0.16, +0.17]	0.76%
http_to_http_json	ingress throughput	-0.00	[-0.05, +0.05]	3.45%
datadog_agent_remap_datadog_logs	ingress throughput	-0.00	[-0.14, +0.14]	1.51%
socket_to_socket_blackhole	ingress throughput	-0.03	[-0.08, +0.01]	82.83%
datadog_agent_remap_datadog_logs_acks	ingress throughput	-0.21	[-0.36, -0.05]	97.45%
http_to_http_noack	ingress throughput	-0.34	[-0.47, -0.21]	100.00%
http_to_s3	ingress throughput	-0.39	[-0.68, -0.09]	96.92%
syslog_log2metric_humio_metrics	ingress throughput	-0.46	[-0.59, -0.33]	100.00%
syslog_log2metric_splunk_hec_metrics	ingress throughput	-0.60	[-0.75, -0.44]	100.00%
syslog_loki	ingress throughput	-0.80	[-0.89, -0.72]	100.00%
datadog_agent_remap_blackhole_acks	ingress throughput	-0.82	[-0.92, -0.72]	100.00%
splunk_hec_route_s3	ingress throughput	-0.83	[-1.33, -0.32]	99.26%
otlp_http_to_blackhole	ingress throughput	-0.89	[-1.06, -0.72]	100.00%
otlp_grpc_to_blackhole	ingress throughput	-1.11	[-1.24, -0.98]	100.00%
enterprise_http_to_http	ingress throughput	-1.37	[-1.52, -1.22]	100.00%

lukesteensen · 2023-10-04T15:24:20Z

src/sinks/datadog/metrics/sink.rs

+    metrics.sort_unstable_by(|a, b| {
+        (
+            a.series(),
+            a.timestamp().map(|dt| dt.timestamp()).unwrap_or(now_ts),
+        )
+            .cmp(&(
+                b.series(),
+                b.timestamp().map(|dt| dt.timestamp()).unwrap_or(now_ts),
+            ))
+    });


This might be simpler as a sort_unstable_by_key.

I tried changing this but ran into a lifetime error when using sort_unstable_by_key without cloning. I think we should keep it as-is to avoid extra clones.

lukesteensen · 2023-10-04T16:08:00Z

src/sinks/datadog/metrics/sink.rs

+            ))
+    });
+
+    metrics.dedup_by(|left, right| {


Semantically, it might make a little more sense to use coalesce for this instead of dedupe_by. Hopefully they're about equivalent in terms of allocations, etc.

Another option could be group_by/into_grouping_map_by with the same key as the sort above, followed by a merge of each group.

I agree. However, the dedupe_by implementation is slightly more optimized as it does the operation in place rather than creating an iterator and then collecting into a new vector. Are you okay keeping it as dedupe_by? I'm happy to create a benchmark and/or change it if desired.

I am curious now what the difference would be, since in this case we could theoretically coalesce in place, but I'm fine keeping it as dedupe_by 😄 . A small explanatory comment might be a good addition though, just to emphasize that our intent here is not actually to "dedupe".

src/sinks/datadog/metrics/sink.rs

src/sinks/util/buffer/metrics/sort.rs

dsmith3197 · 2023-10-04T20:58:35Z

src/sinks/datadog/metrics/sink.rs

+    // Sort by series and timestamp which is required for the below dedupe to behave as desired.
+    // This also tends to compress better than a random ordering by 2-3x (JSON encoded, deflate algorithm).
+    // Note that `sort_unstable_by_key` would be simpler but results in lifetime errors without cloning.
+    metrics.sort_unstable_by(|a, b| {


Note that we were already doing this sort previously but doing it after the aggregation step. Moving the sort before the aggregation means that we now may be sorting a larger array but the subsequent aggregation step is now O(n) instead of O(n^2).

github-actions · 2023-10-05T16:14:18Z

Regression Detector Results

Run ID: 4ab42299-01be-4e39-af91-72f1e4086754
Baseline: afc166f
Comparison: 5acda1d
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine quickly if vector performance is changed and to what degree by a pull request.

Because a target's optimization goal performance in each experiment will vary somewhat each time it is run, we can only estimate mean differences in optimization goal relative to the baseline target. We express these differences as a percentage change relative to the baseline target, denoted "Δ mean %". These estimates are made to a precision that balances accuracy and cost control. We represent this precision as a 90.00% confidence interval denoted "Δ mean % CI": there is a 90.00% chance that the true value of "Δ mean %" is in that interval.

We decide whether a change in performance is a "regression" -- a change worth investigating further -- if both of the following two criteria are true:

The estimated |Δ mean %| ≥ 5.00%. This criterion intends to answer the question "Does the estimated change in mean optimization goal performance have a meaningful impact on your customers?". We assume that when |Δ mean %| < 5.00%, the impact on your customers is not meaningful. We also assume that a performance change in optimization goal is worth investigating whether it is an increase or decrease, so long as the magnitude of the change is sufficiently large.
Zero is not in the 90.00% confidence interval "Δ mean % CI" about "Δ mean %". This statement is equivalent to saying that there is at least a 90.00% chance that the mean difference in optimization goal is not zero. This criterion intends to answer the question, "Is there a statistically significant difference in mean optimization goal performance?". It also means there is no more than a 10.00% chance this criterion reports a statistically significant difference when the true difference in mean optimization goal is zero -- a "false positive". We assume you are willing to accept a 10.00% chance of inaccurately detecting a change in performance when no true difference exists.

The table below, if present, lists those experiments that have experienced a statistically significant change in mean optimization goal performance between baseline and comparison SHAs with 90.00% confidence OR have been detected as newly erratic. Negative values of "Δ mean %" mean that baseline is faster, whereas positive values of "Δ mean %" mean that comparison is faster. Results that do not exhibit more than a ±5.00% change in their mean optimization goal are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting change is observed.

Changes in experiment optimization goals with confidence ≥ 90.00% and |Δ mean %| ≥ 5.00%:

experiment	goal	Δ mean %	confidence
otlp_http_to_blackhole	ingress throughput	-10.42	100.00%

Fine details of change detection per experiment.

experiment	goal	Δ mean %	Δ mean % CI	confidence
syslog_regex_logs2metric_ddmetrics	ingress throughput	+439.75	[+438.51, +440.99]	100.00%
syslog_log2metric_humio_metrics	ingress throughput	+2.28	[+2.12, +2.44]	100.00%
datadog_agent_remap_blackhole	ingress throughput	+1.55	[+1.45, +1.65]	100.00%
enterprise_http_to_http	ingress throughput	+0.96	[+0.79, +1.13]	100.00%
fluent_elasticsearch	ingress throughput	+0.78	[+0.47, +1.10]	100.00%
datadog_agent_remap_blackhole_acks	ingress throughput	+0.65	[+0.55, +0.75]	100.00%
syslog_splunk_hec_logs	ingress throughput	+0.62	[+0.54, +0.71]	100.00%
http_to_s3	ingress throughput	+0.44	[+0.15, +0.73]	98.72%
syslog_humio_logs	ingress throughput	+0.23	[+0.10, +0.35]	99.76%
http_to_http_acks	ingress throughput	+0.09	[-0.06, +0.23]	66.40%
datadog_agent_remap_datadog_logs_acks	ingress throughput	+0.03	[-0.13, +0.20]	26.00%
splunk_hec_to_splunk_hec_logs_noack	ingress throughput	+0.00	[-0.14, +0.15]	4.19%
splunk_hec_indexer_ack_blackhole	ingress throughput	+0.00	[-0.17, +0.17]	0.90%
http_to_http_json	ingress throughput	-0.00	[-0.05, +0.05]	3.68%
splunk_hec_to_splunk_hec_logs_acks	ingress throughput	-0.01	[-0.17, +0.15]	10.51%
syslog_log2metric_splunk_hec_metrics	ingress throughput	-0.50	[-0.65, -0.34]	100.00%
http_to_http_noack	ingress throughput	-0.72	[-0.87, -0.57]	100.00%
datadog_agent_remap_datadog_logs	ingress throughput	-0.92	[-1.07, -0.78]	100.00%
syslog_loki	ingress throughput	-1.06	[-1.13, -0.99]	100.00%
otlp_grpc_to_blackhole	ingress throughput	-1.14	[-1.27, -1.02]	100.00%
splunk_hec_route_s3	ingress throughput	-1.68	[-2.20, -1.16]	100.00%
http_text_to_http_json	ingress throughput	-1.88	[-2.00, -1.76]	100.00%
file_to_blackhole	egress throughput	-2.03	[-3.08, -0.98]	99.85%
socket_to_socket_blackhole	ingress throughput	-3.48	[-3.51, -3.44]	100.00%
otlp_http_to_blackhole	ingress throughput	-10.42	[-10.98, -9.85]	100.00%

In light of regression detector results in #18650, #18740, and #18759 that report repeated suspicious regressions in the `otlp_http_to_blackhole` experiment, this commit / PR marks that experiment "erratic" so that its results do not prevent Vector PRs from merging. Signed-off-by: Geoffrey M. Oxberry <geoffrey.oxberry@datadoghq.com>

github-actions · 2023-10-05T20:38:10Z

Regression Detector Results

Run ID: 2e3f0430-16f0-47ff-8ea5-9f514d0466e5
Baseline: f300c85
Comparison: 6a40169
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine quickly if vector performance is changed and to what degree by a pull request.

Because a target's optimization goal performance in each experiment will vary somewhat each time it is run, we can only estimate mean differences in optimization goal relative to the baseline target. We express these differences as a percentage change relative to the baseline target, denoted "Δ mean %". These estimates are made to a precision that balances accuracy and cost control. We represent this precision as a 90.00% confidence interval denoted "Δ mean % CI": there is a 90.00% chance that the true value of "Δ mean %" is in that interval.

We decide whether a change in performance is a "regression" -- a change worth investigating further -- if both of the following two criteria are true:

The estimated |Δ mean %| ≥ 5.00%. This criterion intends to answer the question "Does the estimated change in mean optimization goal performance have a meaningful impact on your customers?". We assume that when |Δ mean %| < 5.00%, the impact on your customers is not meaningful. We also assume that a performance change in optimization goal is worth investigating whether it is an increase or decrease, so long as the magnitude of the change is sufficiently large.
Zero is not in the 90.00% confidence interval "Δ mean % CI" about "Δ mean %". This statement is equivalent to saying that there is at least a 90.00% chance that the mean difference in optimization goal is not zero. This criterion intends to answer the question, "Is there a statistically significant difference in mean optimization goal performance?". It also means there is no more than a 10.00% chance this criterion reports a statistically significant difference when the true difference in mean optimization goal is zero -- a "false positive". We assume you are willing to accept a 10.00% chance of inaccurately detecting a change in performance when no true difference exists.

The table below, if present, lists those experiments that have experienced a statistically significant change in mean optimization goal performance between baseline and comparison SHAs with 90.00% confidence OR have been detected as newly erratic. Negative values of "Δ mean %" mean that baseline is faster, whereas positive values of "Δ mean %" mean that comparison is faster. Results that do not exhibit more than a ±5.00% change in their mean optimization goal are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting change is observed.

No interesting changes in experiment optimization goals with confidence ≥ 90.00% and |Δ mean %| ≥ 5.00%.

Fine details of change detection per experiment.

experiment	goal	Δ mean %	Δ mean % CI	confidence
syslog_regex_logs2metric_ddmetrics	ingress throughput	+434.25	[+432.99, +435.51]	100.00%
datadog_agent_remap_blackhole	ingress throughput	+2.85	[+2.74, +2.97]	100.00%
http_to_http_noack	ingress throughput	+2.67	[+2.48, +2.86]	100.00%
splunk_hec_route_s3	ingress throughput	+1.51	[+0.97, +2.06]	100.00%
syslog_log2metric_humio_metrics	ingress throughput	+1.51	[+1.34, +1.69]	100.00%
http_to_http_acks	ingress throughput	+0.92	[+0.79, +1.06]	100.00%
http_to_s3	ingress throughput	+0.54	[+0.25, +0.82]	99.81%
datadog_agent_remap_datadog_logs	ingress throughput	+0.08	[-0.12, +0.28]	50.36%
splunk_hec_to_splunk_hec_logs_acks	ingress throughput	+0.03	[-0.13, +0.19]	24.93%
http_to_http_json	ingress throughput	+0.00	[-0.05, +0.06]	7.11%
splunk_hec_to_splunk_hec_logs_noack	ingress throughput	-0.00	[-0.14, +0.14]	0.17%
splunk_hec_indexer_ack_blackhole	ingress throughput	-0.00	[-0.16, +0.16]	1.36%
enterprise_http_to_http	ingress throughput	-0.04	[-0.25, +0.17]	21.93%
datadog_agent_remap_datadog_logs_acks	ingress throughput	-0.21	[-0.33, -0.08]	99.16%
syslog_loki	ingress throughput	-0.94	[-1.02, -0.86]	100.00%
otlp_grpc_to_blackhole	ingress throughput	-0.98	[-1.11, -0.86]	100.00%
fluent_elasticsearch	ingress throughput	-1.31	[-1.62, -1.00]	100.00%
datadog_agent_remap_blackhole_acks	ingress throughput	-1.33	[-1.44, -1.22]	100.00%
syslog_splunk_hec_logs	ingress throughput	-1.36	[-1.45, -1.28]	100.00%
syslog_humio_logs	ingress throughput	-1.55	[-1.68, -1.42]	100.00%
syslog_log2metric_splunk_hec_metrics	ingress throughput	-1.65	[-1.80, -1.50]	100.00%
file_to_blackhole	egress throughput	-1.83	[-2.96, -0.71]	99.25%
otlp_http_to_blackhole	ingress throughput	-1.85	[-2.00, -1.70]	100.00%
http_text_to_http_json	ingress throughput	-3.03	[-3.15, -2.90]	100.00%
socket_to_socket_blackhole	ingress throughput	-3.38	[-3.42, -3.33]	100.00%

In light of regression detector results in #18650, #18740, and #18759 that report repeated suspicious regressions in the `otlp_http_to_blackhole` experiment, this commit / PR marks that experiment "erratic" so that its results do not prevent Vector PRs from merging. Signed-off-by: Geoffrey M. Oxberry <geoffrey.oxberry@datadoghq.com>

* fix(datadog_metrics sink): improve aggregation performance * fix tests and add concurrency * feedback * add comment to dedupe_by * sort by metric type as well * fix flakey proptest * only dedupe non-counters in prop test

# [2.0.0](answerbook/vector@v1.36.2...v2.0.0) (2024-01-09) ### Bug Fixes * **aws provider**: Don't unwap external_id (vectordotdev#18452) [fd0ccd5](answerbook/vector@fd0ccd5) - GitHub * **aws_s3 source**: Allow region to be optional (vectordotdev#18258) [23a1a2d](answerbook/vector@23a1a2d) - GitHub * **aws_s3 source**: Use the decoder to calculate type defs (vectordotdev#18274) [40f525c](answerbook/vector@40f525c) - GitHub * **ci**: add comment trigger filter for regression workflow concurrency group (vectordotdev#18408) [ff6e888](answerbook/vector@ff6e888) - GitHub * **ci**: don't continue on errors in unit-mac test (vectordotdev#18496) [712a210](answerbook/vector@712a210) - GitHub * **ci**: Drop docker-compose from bootstrap install (vectordotdev#18407) [41c5567](answerbook/vector@41c5567) - GitHub * **ci**: Unlink python before `brew install` (vectordotdev#18402) [e39d9b3](answerbook/vector@e39d9b3) - GitHub * **config**: fix concurrency default & docs (vectordotdev#18651) [b35527d](answerbook/vector@b35527d) - GitHub * **config**: Only try default paths if the path is not specified (vectordotdev#18681) [b050a65](answerbook/vector@b050a65) - Jesse Szwedko * **core**: don't show warning about max allocation groups if tracing not enabled (vectordotdev#18589) [3afda3c](answerbook/vector@3afda3c) - GitHub * **datadog_metrics sink**: improve aggregation performance (vectordotdev#18759) [4f97d48](answerbook/vector@4f97d48) - Jesse Szwedko * **debian platform**: Re-add `conf-files` directive for `cargo-deb` (vectordotdev#18726) [aa39f5e](answerbook/vector@aa39f5e) - Jesse Szwedko * **debian platform**: Remove `conf-files` directive for `cargo-deb` (vectordotdev#18455) [40ef7c4](answerbook/vector@40ef7c4) - GitHub * default to nullbyte delimiter for GELF vectordotdev#18008 (vectordotdev#18419) [a112704](answerbook/vector@a112704) - GitHub * **deps, security**: temporarily ignore `ed25519-dalek` security vulnerability (vectordotdev#18245) [1b90398](answerbook/vector@1b90398) - GitHub * **deps**: fix [dev-dependencies] for some libs (vectordotdev#18328) [8086e19](answerbook/vector@8086e19) - GitHub * **deps**: load default and legacy openssl providers (vectordotdev#18276) [fc17fba](answerbook/vector@fc17fba) - GitHub * **distribution**: Use yaml instead of toml file (vectordotdev#18606) [24701cf](answerbook/vector@24701cf) - GitHub * **dnstap source**: support DNSSEC RRSIG record data (vectordotdev#18878) [3eaad37](answerbook/vector@3eaad37) - Jesse Szwedko * **docs**: add the 'http_client_requests_sent_total' (vectordotdev#18299) [69e8383](answerbook/vector@69e8383) - GitHub * **docs**: remove '---\n' prefix from toYaml config example generator (vectordotdev#18502) [8d07e18](answerbook/vector@8d07e18) - GitHub * **elasticsearch**: Ignore `pipeline` argument if it is an empty string (vectordotdev#18248) [d9dbed8](answerbook/vector@d9dbed8) - GitHub * **external docs**: Document intentional label on component_discarded_events_total (vectordotdev#18622) [fd58af9](answerbook/vector@fd58af9) - GitHub * **gcp service**: retry on unauthorized (vectordotdev#18586) [75d03b3](answerbook/vector@75d03b3) - GitHub * **json codec**: Fix deserializing non-object values with the `Vector` namespace (vectordotdev#18379) [f15144b](answerbook/vector@f15144b) - GitHub * **kafka sink**: Make KafkaService return `Poll::Pending` when producer queue is full (vectordotdev#18770) [fa09de3](answerbook/vector@fa09de3) - Jesse Szwedko * **kafka sink**: performance improvements and fix memory leak (vectordotdev#18634) [3c662f3](answerbook/vector@3c662f3) - GitHub * **kubernetes_logs source**: Fix events being empty when log namespacing is enabled (vectordotdev#18244) [8918c66](answerbook/vector@8918c66) - GitHub * **new_relic sink**: Multiple fixes related to metrics (vectordotdev#18151) [953e305](answerbook/vector@953e305) - GitHub * **observability**: add all events that are being encoded (vectordotdev#18289) [93baba2](answerbook/vector@93baba2) - GitHub * **observability**: don't increment component_errors_total for `HttpClient` warning (vectordotdev#18505) [a544e6e](answerbook/vector@a544e6e) - GitHub * **opentelemetry source**: Remove the 4MB default for gRPC request decoding (vectordotdev#18306) [a6262cd](answerbook/vector@a6262cd) - GitHub * **prometheus_remote_write source, prometheus_scrape source**: Fix feature check (vectordotdev#18440) [9c1abd6](answerbook/vector@9c1abd6) - GitHub * **releasing**: Update example YAML config data_dir (vectordotdev#18896) [ac86a8a](answerbook/vector@ac86a8a) - Jesse Szwedko * **remap transform**: log namespace should be used when splitting events from arrays (vectordotdev#18372) [15a63b4](answerbook/vector@15a63b4) - GitHub * **sample transform**: Use metadata when log namespacing is enabled (vectordotdev#18259) [69b4c1c](answerbook/vector@69b4c1c) - GitHub * **sinks**: resolve memory leak by always setting a request builder concurrency limit (vectordotdev#18637) [6ced6ca](answerbook/vector@6ced6ca) - GitHub * skip encoding empty sketches (vectordotdev#18530) [ae0aa11](answerbook/vector@ae0aa11) - GitHub * socket tcp port typedef (vectordotdev#18180) [b0c89ab](answerbook/vector@b0c89ab) - GitHub * **syslog source, docs**: Fix docs for `host` field for syslog source (vectordotdev#18453) [0ef902d](answerbook/vector@0ef902d) - GitHub * **tests**: fix tests for the generate command (vectordotdev#18383) [2a4235c](answerbook/vector@2a4235c) - GitHub * use `rstest` in `generate` command tests (vs wrong usage of `proptest`) (vectordotdev#18365) [4359c9a](answerbook/vector@4359c9a) - GitHub * **website**: Fix installer list for MacOS (vectordotdev#18364) [13eec06](answerbook/vector@13eec06) - GitHub ### Chores * Add SHA256 checksums file to GH releases (vectordotdev#18701) [3411642](answerbook/vector@3411642) - Jesse Szwedko * **appsignal sink**: Refactor to use StreamSink (vectordotdev#18209) [c495939](answerbook/vector@c495939) - GitHub * **aws provider, external_docs**: Update the AWS authentication documentation (vectordotdev#18492) [cd8c5fe](answerbook/vector@cd8c5fe) - GitHub * **ci**: Add a test to assert conf files aren't overwritten (vectordotdev#18728) [ab272f6](answerbook/vector@ab272f6) - Jesse Szwedko * **ci**: Bump actions/checkout from 3 to 4 (vectordotdev#18476) [59dfd67](answerbook/vector@59dfd67) - GitHub * **ci**: Bump aws-actions/configure-aws-credentials from 2.2.0 to 3.0.1 (vectordotdev#18386) [f33aff1](answerbook/vector@f33aff1) - GitHub * **ci**: Bump aws-actions/configure-aws-credentials from 3.0.1 to 3.0.2 (vectordotdev#18511) [737d2f1](answerbook/vector@737d2f1) - GitHub * **ci**: Bump aws-actions/configure-aws-credentials from 3.0.2 to 4.0.0 (vectordotdev#18544) [3018864](answerbook/vector@3018864) - GitHub * **ci**: Bump docker/build-push-action from 4.1.1 to 4.2.1 (vectordotdev#18512) [f19d166](answerbook/vector@f19d166) - GitHub * **ci**: Bump docker/build-push-action from 4.2.1 to 5.0.0 (vectordotdev#18546) [c30a4b2](answerbook/vector@c30a4b2) - GitHub * **ci**: Bump docker/login-action from 2 to 3 (vectordotdev#18556) [d976c6e](answerbook/vector@d976c6e) - GitHub * **ci**: Bump docker/metadata-action from 4.6.0 to 5.0.0 (vectordotdev#18543) [8aae235](answerbook/vector@8aae235) - GitHub * **ci**: Bump docker/setup-buildx-action from 2.10.0 to 3.0.0 (vectordotdev#18545) [899e3c0](answerbook/vector@899e3c0) - GitHub * **ci**: Bump docker/setup-buildx-action from 2.9.1 to 2.10.0 (vectordotdev#18406) [dd8a0ef](answerbook/vector@dd8a0ef) - GitHub * **ci**: Bump docker/setup-qemu-action from 2.2.0 to 3.0.0 (vectordotdev#18547) [dcda1de](answerbook/vector@dcda1de) - GitHub * **ci**: Bump MacOS unit test runners to 13 (vectordotdev#18823) [c499427](answerbook/vector@c499427) - Jesse Szwedko * **ci**: Bump myrotvorets/set-commit-status-action from 1.1.7 to 2.0.0 (vectordotdev#18510) [8d5003a](answerbook/vector@8d5003a) - GitHub * **ci**: Bump tibdex/github-app-token from 1.8.0 to 1.8.2 (vectordotdev#18454) [4e4ece6](answerbook/vector@4e4ece6) - GitHub * **ci**: Bump tibdex/github-app-token from 1.8.2 to 2.0.0 (vectordotdev#18528) [7295f22](answerbook/vector@7295f22) - GitHub * **ci**: Bump tibdex/github-app-token from 2.0.0 to 2.1.0 (vectordotdev#18608) [addc46e](answerbook/vector@addc46e) - GitHub * **ci**: Fix cookie banner style issues (vectordotdev#18745) [792a1b5](answerbook/vector@792a1b5) - Jesse Szwedko * **ci**: group azure and prost crates for dependabot (vectordotdev#18525) [7b6ad62](answerbook/vector@7b6ad62) - GitHub * **ci**: Re-add docker-compose installation (vectordotdev#18415) [c1ed017](answerbook/vector@c1ed017) - GitHub * **ci**: remove kinetic as it's no longer supported (vectordotdev#18540) [c74a469](answerbook/vector@c74a469) - GitHub * **ci**: revert bump actions/checkout from 3 to 4 (vectordotdev#18490) [3fd6486](answerbook/vector@3fd6486) - GitHub * **config, docs**: Replace 'vector.toml' with 'vector.yaml' (vectordotdev#18388) [9d29563](answerbook/vector@9d29563) - GitHub * **datadog_logs sink**: Use `endpoint` config setting consistent with the other datadog_ sinks. (vectordotdev#18497) [1ac19dd](answerbook/vector@1ac19dd) - GitHub * **deps**: Bump `nkeys` to 0.3.2 (vectordotdev#18264) [fc533a1](answerbook/vector@fc533a1) - GitHub * **deps**: Bump anyhow from 1.0.72 to 1.0.74 (vectordotdev#18255) [a77b652](answerbook/vector@a77b652) - GitHub * **deps**: Bump anyhow from 1.0.74 to 1.0.75 (vectordotdev#18284) [fb5f099](answerbook/vector@fb5f099) - GitHub * **deps**: Bump async-compression from 0.4.1 to 0.4.2 (vectordotdev#18417) [8639655](answerbook/vector@8639655) - GitHub * **deps**: Bump async-compression from 0.4.2 to 0.4.3 (vectordotdev#18539) [3e3c7ad](answerbook/vector@3e3c7ad) - GitHub * **deps**: Bump async-nats from 0.31.0 to 0.32.0 (vectordotdev#18640) [7ecb06a](answerbook/vector@7ecb06a) - GitHub * **deps**: Bump async-recursion from 1.0.4 to 1.0.5 (vectordotdev#18466) [8b017b6](answerbook/vector@8b017b6) - GitHub * **deps**: Bump azure_core from 0.13.0 to 0.14.0 (vectordotdev#18361) [0bbb152](answerbook/vector@0bbb152) - GitHub * **deps**: Bump base64 from 0.21.2 to 0.21.3 (vectordotdev#18398) [5dca377](answerbook/vector@5dca377) - GitHub * **deps**: Bump base64 from 0.21.3 to 0.21.4 (vectordotdev#18522) [44a87dc](answerbook/vector@44a87dc) - GitHub * **deps**: Bump bollard from 0.14.0 to 0.15.0 (vectordotdev#18581) [996372f](answerbook/vector@996372f) - GitHub * **deps**: Bump bstr from 1.6.0 to 1.6.1 (vectordotdev#18422) [fa6f2cd](answerbook/vector@fa6f2cd) - GitHub * **deps**: Bump bstr from 1.6.1 to 1.6.2 (vectordotdev#18433) [8b6a307](answerbook/vector@8b6a307) - GitHub * **deps**: Bump bytes from 1.4.0 to 1.5.0 (vectordotdev#18508) [bfdb5b0](answerbook/vector@bfdb5b0) - GitHub * **deps**: Bump bytesize from 1.2.0 to 1.3.0 (vectordotdev#18367) [836a31e](answerbook/vector@836a31e) - GitHub * **deps**: Bump cached from 0.44.0 to 0.45.0 (vectordotdev#18478) [325fbea](answerbook/vector@325fbea) - GitHub * **deps**: Bump cached from 0.45.0 to 0.45.1 (vectordotdev#18485) [f8981e1](answerbook/vector@f8981e1) - GitHub * **deps**: Bump cargo_toml from 0.15.3 to 0.16.0 (vectordotdev#18571) [0e60001](answerbook/vector@0e60001) - GitHub * **deps**: Bump cargo_toml from 0.16.0 to 0.16.1 (vectordotdev#18605) [c47e65f](answerbook/vector@c47e65f) - GitHub * **deps**: Bump cargo_toml from 0.16.1 to 0.16.2 (vectordotdev#18619) [fa526a8](answerbook/vector@fa526a8) - GitHub * **deps**: Bump cargo_toml from 0.16.2 to 0.16.3 (vectordotdev#18674) [a4cd8b7](answerbook/vector@a4cd8b7) - GitHub * **deps**: Bump chrono to 0.4.30 (vectordotdev#18527) [a6305de](answerbook/vector@a6305de) - GitHub * **deps**: Bump cidr-utils from 0.5.10 to 0.5.11 (vectordotdev#18516) [f5cd27a](answerbook/vector@f5cd27a) - GitHub * **deps**: Bump clap from 4.3.21 to 4.3.22 (vectordotdev#18293) [47f25d3](answerbook/vector@47f25d3) - GitHub * **deps**: Bump clap from 4.3.22 to 4.3.23 (vectordotdev#18311) [05765d8](answerbook/vector@05765d8) - GitHub * **deps**: Bump clap from 4.3.23 to 4.3.24 (vectordotdev#18362) [5f4a6d8](answerbook/vector@5f4a6d8) - GitHub * **deps**: Bump clap from 4.3.24 to 4.4.1 (vectordotdev#18411) [d3b9eda](answerbook/vector@d3b9eda) - GitHub * **deps**: Bump clap from 4.4.1 to 4.4.2 (vectordotdev#18447) [6998518](answerbook/vector@6998518) - GitHub * **deps**: Bump clap from 4.4.2 to 4.4.3 (vectordotdev#18550) [9e8407e](answerbook/vector@9e8407e) - GitHub * **deps**: Bump clap_complete from 4.3.2 to 4.4.0 (vectordotdev#18374) [76ffdab](answerbook/vector@76ffdab) - GitHub * **deps**: Bump clap_complete from 4.4.0 to 4.4.1 (vectordotdev#18509) [af444ea](answerbook/vector@af444ea) - GitHub * **deps**: Bump clap_complete from 4.4.1 to 4.4.2 (vectordotdev#18673) [3edafef](answerbook/vector@3edafef) - GitHub * **deps**: Bump crossterm from 0.26.1 to 0.27.0 (vectordotdev#18168) [d9c75bd](answerbook/vector@d9c75bd) - GitHub * **deps**: Bump dashmap from 5.5.0 to 5.5.1 (vectordotdev#18338) [754bee0](answerbook/vector@754bee0) - GitHub * **deps**: Bump dashmap from 5.5.1 to 5.5.3 (vectordotdev#18427) [adf134b](answerbook/vector@adf134b) - GitHub * **deps**: Bump dyn-clone from 1.0.12 to 1.0.13 (vectordotdev#18285) [1fbbdcb](answerbook/vector@1fbbdcb) - GitHub * **deps**: Bump dyn-clone from 1.0.13 to 1.0.14 (vectordotdev#18607) [d5f4caa](answerbook/vector@d5f4caa) - GitHub * **deps**: Bump encoding_rs from 0.8.32 to 0.8.33 (vectordotdev#18360) [2493288](answerbook/vector@2493288) - GitHub * **deps**: Bump enumflags2 from 0.7.7 to 0.7.8 (vectordotdev#18560) [0f90c39](answerbook/vector@0f90c39) - GitHub * **deps**: Bump flate2 from 1.0.26 to 1.0.27 (vectordotdev#18254) [3a6af99](answerbook/vector@3a6af99) - GitHub * **deps**: Bump h2 from 0.3.20 to 0.3.21 (vectordotdev#18330) [d41f9f6](answerbook/vector@d41f9f6) - GitHub * **deps**: Bump headers from 0.3.8 to 0.3.9 (vectordotdev#18448) [749594c](answerbook/vector@749594c) - GitHub * **deps**: Bump http-serde from 1.1.2 to 1.1.3 (vectordotdev#18310) [31ec4b3](answerbook/vector@31ec4b3) - GitHub * **deps**: Bump indicatif from 0.17.6 to 0.17.7 (vectordotdev#18647) [6bab5be](answerbook/vector@6bab5be) - GitHub * **deps**: Bump indoc from 2.0.3 to 2.0.4 (vectordotdev#18582) [b3f76b5](answerbook/vector@b3f76b5) - GitHub * **deps**: Bump inventory from 0.3.11 to 0.3.12 (vectordotdev#18429) [9f95026](answerbook/vector@9f95026) - GitHub * **deps**: Bump libc from 0.2.147 to 0.2.148 (vectordotdev#18563) [59e22fc](answerbook/vector@59e22fc) - GitHub * **deps**: Bump lru from 0.11.0 to 0.11.1 (vectordotdev#18484) [7ec5b97](answerbook/vector@7ec5b97) - GitHub * **deps**: Bump md-5 from 0.10.5 to 0.10.6 (vectordotdev#18648) [e8d946f](answerbook/vector@e8d946f) - GitHub * **deps**: Bump memchr from 2.5.0 to 2.6.0 (vectordotdev#18410) [7bc1942](answerbook/vector@7bc1942) - GitHub * **deps**: Bump memchr from 2.6.0 to 2.6.1 (vectordotdev#18421) [3f4603c](answerbook/vector@3f4603c) - GitHub * **deps**: Bump memchr from 2.6.1 to 2.6.2 (vectordotdev#18434) [5e3eaa9](answerbook/vector@5e3eaa9) - GitHub * **deps**: Bump memchr from 2.6.2 to 2.6.3 (vectordotdev#18470) [69a1ca0](answerbook/vector@69a1ca0) - GitHub * **deps**: Bump mlua from 0.8.9 to 0.8.10 (vectordotdev#18292) [03fe2fe](answerbook/vector@03fe2fe) - GitHub * **deps**: Bump mongodb from 2.6.0 to 2.6.1 (vectordotdev#18268) [28de959](answerbook/vector@28de959) - GitHub * **deps**: Bump MSRV to 1.70.0 (vectordotdev#18394) [fd21b19](answerbook/vector@fd21b19) - GitHub * **deps**: Bump no-proxy from 0.3.3 to 0.3.4 (vectordotdev#18277) [7a1c49c](answerbook/vector@7a1c49c) - GitHub * **deps**: Bump notify from 6.0.1 to 6.1.0 (vectordotdev#18317) [e963766](answerbook/vector@e963766) - GitHub * **deps**: Bump notify from 6.1.0 to 6.1.1 (vectordotdev#18332) [ae5de9c](answerbook/vector@ae5de9c) - GitHub * **deps**: Bump num_enum from 0.6.1 to 0.7.0 (vectordotdev#18238) [771f476](answerbook/vector@771f476) - GitHub * **deps**: Bump openssl from 0.10.56 to 0.10.57 (vectordotdev#18400) [15792f6](answerbook/vector@15792f6) - GitHub * **deps**: Bump ordered-float from 3.7.0 to 3.8.0 (vectordotdev#18302) [833ac19](answerbook/vector@833ac19) - GitHub * **deps**: Bump ordered-float from 3.8.0 to 3.9.0 (vectordotdev#18307) [ca30b6d](answerbook/vector@ca30b6d) - GitHub * **deps**: Bump ordered-float from 3.9.0 to 3.9.1 (vectordotdev#18350) [ab41edc](answerbook/vector@ab41edc) - GitHub * **deps**: Bump proc-macro2 from 1.0.66 to 1.0.67 (vectordotdev#18561) [6f091e1](answerbook/vector@6f091e1) - GitHub * **deps**: Bump prost from 0.11.9 to 0.12.0 (vectordotdev#18460) [4fba377](answerbook/vector@4fba377) - GitHub * **deps**: Bump prost-reflect from 0.11.4 to 0.11.5 (vectordotdev#18426) [221e0a1](answerbook/vector@221e0a1) - GitHub * **deps**: Bump quote from 1.0.32 to 1.0.33 (vectordotdev#18283) [f2a6887](answerbook/vector@f2a6887) - GitHub * **deps**: Bump ratatui from 0.22.0 to 0.23.0 (vectordotdev#18412) [82be883](answerbook/vector@82be883) - GitHub * **deps**: Bump rdkafka from 0.33.2 to 0.34.0 (vectordotdev#18393) [016890e](answerbook/vector@016890e) - GitHub * **deps**: Bump redis from 0.23.2 to 0.23.3 (vectordotdev#18464) [2e2692e](answerbook/vector@2e2692e) - GitHub * **deps**: Bump regex from 1.9.3 to 1.9.4 (vectordotdev#18399) [21f7679](answerbook/vector@21f7679) - GitHub * **deps**: Bump regex from 1.9.4 to 1.9.5 (vectordotdev#18472) [1aeb42e](answerbook/vector@1aeb42e) - GitHub * **deps**: Bump reqwest from 0.11.18 to 0.11.19 (vectordotdev#18329) [9b4625c](answerbook/vector@9b4625c) - GitHub * **deps**: Bump reqwest from 0.11.19 to 0.11.20 (vectordotdev#18366) [2aaea89](answerbook/vector@2aaea89) - GitHub * **deps**: Bump rmpv from 1.0.0 to 1.0.1 (vectordotdev#18049) [0cda906](answerbook/vector@0cda906) - GitHub * **deps**: Bump Rust to 1.72.1 (vectordotdev#18638) [466ef84](answerbook/vector@466ef84) - GitHub * **deps**: Bump serde from 1.0.183 to 1.0.185 (vectordotdev#18319) [ea2d576](answerbook/vector@ea2d576) - GitHub * **deps**: Bump serde from 1.0.185 to 1.0.186 (vectordotdev#18370) [83ec3cf](answerbook/vector@83ec3cf) - GitHub * **deps**: Bump serde from 1.0.186 to 1.0.188 (vectordotdev#18395) [4a2805d](answerbook/vector@4a2805d) - GitHub * **deps**: Bump serde_derive_internals from 0.28.0 to 0.29.0 (vectordotdev#18499) [f7c3c96](answerbook/vector@f7c3c96) - GitHub * **deps**: Bump serde_json from 1.0.104 to 1.0.105 (vectordotdev#18267) [d05bc3e](answerbook/vector@d05bc3e) - GitHub * **deps**: Bump serde_json from 1.0.105 to 1.0.106 (vectordotdev#18523) [2687ed1](answerbook/vector@2687ed1) - GitHub * **deps**: Bump serde_json from 1.0.106 to 1.0.107 (vectordotdev#18562) [3c3e251](answerbook/vector@3c3e251) - GitHub * **deps**: Bump serde_with from 3.2.0 to 3.3.0 (vectordotdev#18315) [54d48d7](answerbook/vector@54d48d7) - GitHub * **deps**: Bump serde-wasm-bindgen from 0.5.0 to 0.6.0 (vectordotdev#18565) [95297b2](answerbook/vector@95297b2) - GitHub * **deps**: Bump similar-asserts from 1.4.2 to 1.5.0 (vectordotdev#18318) [b3a1d53](answerbook/vector@b3a1d53) - GitHub * **deps**: Bump smallvec from 1.11.0 to 1.11.1 (vectordotdev#18620) [feca4c8](answerbook/vector@feca4c8) - GitHub * **deps**: Bump socket2 from 0.5.3 to 0.5.4 (vectordotdev#18531) [8eb5256](answerbook/vector@8eb5256) - GitHub * **deps**: Bump syn from 2.0.28 to 2.0.29 (vectordotdev#18282) [d10e373](answerbook/vector@d10e373) - GitHub * **deps**: Bump syn from 2.0.29 to 2.0.31 (vectordotdev#18471) [f2b46d6](answerbook/vector@f2b46d6) - GitHub * **deps**: Bump syn from 2.0.31 to 2.0.32 (vectordotdev#18524) [f30537e](answerbook/vector@f30537e) - GitHub * **deps**: Bump syn from 2.0.32 to 2.0.33 (vectordotdev#18559) [f750cf7](answerbook/vector@f750cf7) - GitHub * **deps**: Bump syn from 2.0.33 to 2.0.37 (vectordotdev#18580) [e80d7b7](answerbook/vector@e80d7b7) - GitHub * **deps**: Bump the azure group with 4 updates (vectordotdev#18529) [3d7199e](answerbook/vector@3d7199e) - GitHub * **deps**: Bump the prost group with 3 updates (vectordotdev#18579) [16edc22](answerbook/vector@16edc22) - GitHub * **deps**: Bump thiserror from 1.0.44 to 1.0.46 (vectordotdev#18253) [2a8d974](answerbook/vector@2a8d974) - GitHub * **deps**: Bump thiserror from 1.0.46 to 1.0.47 (vectordotdev#18286) [378926d](answerbook/vector@378926d) - GitHub * **deps**: Bump thiserror from 1.0.47 to 1.0.48 (vectordotdev#18473) [7ad6313](answerbook/vector@7ad6313) - GitHub * **deps**: Bump to Rust 1.72.0 (vectordotdev#18389) [7849d80](answerbook/vector@7849d80) - GitHub * **deps**: Bump tokio from 1.30.0 to 1.32.0 (vectordotdev#18279) [47051a5](answerbook/vector@47051a5) - GitHub * **deps**: Bump tokio-postgres from 0.7.7 to 0.7.9 (vectordotdev#18316) [a7800f7](answerbook/vector@a7800f7) - GitHub * **deps**: Bump tokio-postgres from 0.7.9 to 0.7.10 (vectordotdev#18391) [725b9bd](answerbook/vector@725b9bd) - GitHub * **deps**: Bump tokio-test from 0.4.2 to 0.4.3 (vectordotdev#18357) [6716959](answerbook/vector@6716959) - GitHub * **deps**: Bump tokio-tungstenite from 0.20.0 to 0.20.1 (vectordotdev#18661) [42fea39](answerbook/vector@42fea39) - Jesse Szwedko * **deps**: Bump toml from 0.7.6 to 0.7.7 (vectordotdev#18507) [688e2d9](answerbook/vector@688e2d9) - GitHub * **deps**: Bump toml from 0.7.7 to 0.7.8 (vectordotdev#18520) [0476bb5](answerbook/vector@0476bb5) - GitHub * **deps**: Bump toml from 0.7.8 to 0.8.0 (vectordotdev#18549) [0382bc5](answerbook/vector@0382bc5) - GitHub * **deps**: Bump tonic from 0.10.0 to 0.10.1 (vectordotdev#18639) [9523987](answerbook/vector@9523987) - GitHub * **deps**: Bump tonic-build from 0.10.0 to 0.10.1 (vectordotdev#18641) [ec9efb5](answerbook/vector@ec9efb5) - GitHub * **deps**: Bump tower-http from 0.4.3 to 0.4.4 (vectordotdev#18461) [d3d8a4c](answerbook/vector@d3d8a4c) - GitHub * **deps**: Bump trust-dns-proto from 0.22.0 to 0.23.0 (vectordotdev#18349) [a252eda](answerbook/vector@a252eda) - GitHub * **deps**: Bump typetag from 0.2.12 to 0.2.13 (vectordotdev#18287) [704cbfe](answerbook/vector@704cbfe) - GitHub * **deps**: Bump url from 2.4.0 to 2.4.1 (vectordotdev#18414) [314a37b](answerbook/vector@314a37b) - GitHub * **deps**: Bump warp from 0.3.5 to 0.3.6 (vectordotdev#18704) [0a5d29e](answerbook/vector@0a5d29e) - Jesse Szwedko * **deps**: Bump webpki 0.22.1 -> 0.22.4 [d9c4f2e](answerbook/vector@d9c4f2e) - Jesse Szwedko * **deps**: Bump webpki from 0.22.0 to 0.22.1 (vectordotdev#18494) [9859b9e](answerbook/vector@9859b9e) - GitHub * **deps**: Bump webpki from 0.22.1 to 0.22.2 (vectordotdev#18744) [2cbc16c](answerbook/vector@2cbc16c) - Jesse Szwedko * **deps**: Drop patch to use custom `chrono` repo (vectordotdev#18567) [6c34cd5](answerbook/vector@6c34cd5) - GitHub * **deps**: Drop use of `once_cell::{sync,unsync}::OnceCell` (vectordotdev#17621) [f2cd59a](answerbook/vector@f2cd59a) - GitHub * **deps**: Fix issue with `cargo` refetching refs on every run (vectordotdev#18331) [6397edb](answerbook/vector@6397edb) - GitHub * **deps**: group tonic crates in dependabot (vectordotdev#18645) [32be0d3](answerbook/vector@32be0d3) - GitHub * **deps**: Remove openssl-sys patch (vectordotdev#18495) [8ec87eb](answerbook/vector@8ec87eb) - GitHub * **deps**: Update fork of rust-openssl (vectordotdev#18404) [f8d073e](answerbook/vector@f8d073e) - GitHub * **deps**: Update VRL to 0.7.0 (vectordotdev#18672) [70e8b5f](answerbook/vector@70e8b5f) - GitHub * **dev**: Add CODEOWNERS for documentation updates (vectordotdev#18628) [918beac](answerbook/vector@918beac) - GitHub * **dev**: Add DEPRECATIONS.md file to track deprecations (vectordotdev#18613) [d8f36e4](answerbook/vector@d8f36e4) - GitHub * **distribution**: Fix PATH modification to allow for spaces (vectordotdev#18294) [aed8224](answerbook/vector@aed8224) - GitHub * **docs**: Add highlight announcing YAML as the new default config … (vectordotdev#18435) [c150b14](answerbook/vector@c150b14) - GitHub * **docs**: add more comparison examples (vectordotdev#18333) [2f458f6](answerbook/vector@2f458f6) - GitHub * **docs**: Change the default configuration tab and add comments to… (vectordotdev#18420) [ed1dedf](answerbook/vector@ed1dedf) - GitHub * **docs**: Convert a few more configs to YAML (vectordotdev#18632) [f2d60cb](answerbook/vector@f2d60cb) - GitHub * **docs**: discuss disk throughput configurations in sizing guidance (vectordotdev#18566) [5a52b61](answerbook/vector@5a52b61) - GitHub * **docs**: Emphasize the "may" bit of the backpressure docs (vectordotdev#18457) [1647964](answerbook/vector@1647964) - GitHub * **docs**: make YAML appear first in the example configurations (vectordotdev#18325) [85dd439](answerbook/vector@85dd439) - GitHub * **exec source**: remove obsolete codec file and import (vectordotdev#18257) [adbc06f](answerbook/vector@adbc06f) - GitHub * **exec source**: split tests into own module file (vectordotdev#18301) [ac90069](answerbook/vector@ac90069) - GitHub * **external docs**: Remove or replace mentions of vector in functions doc (vectordotdev#18679) [3910677](answerbook/vector@3910677) - Jesse Szwedko * feature gate aws-core features (vectordotdev#18482) [e19243f](answerbook/vector@e19243f) - GitHub * **gcp_stackdriver_logs sink**: refactor to new style (vectordotdev#18335) [0d8ab26](answerbook/vector@0d8ab26) - GitHub * **honeycomb sink**: refactor to new style (vectordotdev#18280) [e104dc4](answerbook/vector@e104dc4) - GitHub * **http sink**: refactor to new style (vectordotdev#18200) [d3a6235](answerbook/vector@d3a6235) - GitHub * Improve checksum script (vectordotdev#18487) [8981cba](answerbook/vector@8981cba) - GitHub * **internal docs**: Add workspace to new rust docs deployment (vectordotdev#18616) [180647b](answerbook/vector@180647b) - GitHub * **kubernetes**: Regenerate manifests from new Helm chart version (vectordotdev#18629) [99de28b](answerbook/vector@99de28b) - GitHub * **log_to_metric transform**: Add Cargo feature for the transform (vectordotdev#18337) [d044084](answerbook/vector@d044084) - GitHub * **log_to_metric transform**: add missing Cargo feature (vectordotdev#18308) [69621bd](answerbook/vector@69621bd) - GitHub * **log_to_metric transform**: Revert missing Cargo feature `transforms-log_to_metric` (vectordotdev#18327) [e6ec664](answerbook/vector@e6ec664) - GitHub * make pin-project a workspace dependency (vectordotdev#18176) [b755a46](answerbook/vector@b755a46) - GitHub * **nats sink**: Refactor to use StreamSink components (vectordotdev#18243) [294c1dd](answerbook/vector@294c1dd) - GitHub * **prometheus_remote_write source, prometheus_scrape source**: Split compilation features (vectordotdev#18431) [c07c99d](answerbook/vector@c07c99d) - GitHub * **redis sink**: Refactor to use StreamSink (vectordotdev#18220) [1e7e99c](answerbook/vector@1e7e99c) - GitHub * **releasing**: Add 0.32.0 highlight for legacy OpenSSL provider deprecation (vectordotdev#18263) [62bd340](answerbook/vector@62bd340) - GitHub * **releasing**: Add known issue for 0.33.0 debian packaging regression (vectordotdev#18727) [eae7b82](answerbook/vector@eae7b82) - Jesse Szwedko * **releasing**: Add known issues for v0.32.0 (vectordotdev#18298) [d09b2af](answerbook/vector@d09b2af) - GitHub * **releasing**: Add note about protobuf codec addition for 0.32.0 release (vectordotdev#18275) [87850ee](answerbook/vector@87850ee) - GitHub * **releasing**: Bump k8s manifests to v0.24.1 of the chart (vectordotdev#18334) [a1d05e4](answerbook/vector@a1d05e4) - GitHub * **releasing**: Bump Vector to 0.33.0 (vectordotdev#18250) [ef51e7a](answerbook/vector@ef51e7a) - GitHub * **releasing**: Prepare v0.32.0 release [50685f9](answerbook/vector@50685f9) - Jesse Szwedko * **releasing**: Prepare v0.32.1 release [bb42d52](answerbook/vector@bb42d52) - Jesse Szwedko * **releasing**: Prepare v0.32.2 release [d833296](answerbook/vector@d833296) - Jesse Szwedko * **releasing**: Prepare v0.33.0 release [89605fb](answerbook/vector@89605fb) - Jesse Szwedko * **releasing**: Prepare v0.33.1 release [3cc27b9](answerbook/vector@3cc27b9) - Jesse Szwedko * **releasing**: Regenerate k8s manifests for v0.24.0 of the chart (vectordotdev#18251) [d155a23](answerbook/vector@d155a23) - GitHub * **releasing**: Use large pages for better OS compatibility (vectordotdev#18481) [e23941c](answerbook/vector@e23941c) - GitHub * **rust-doc**: Add new Make command for CI builds (vectordotdev#18444) [4127b2c](answerbook/vector@4127b2c) - GitHub * **rust-doc**: Update amplify build to include workspace flag (vectordotdev#18630) [f01354e](answerbook/vector@f01354e) - GitHub * **security**: Make the warning for the deprecated OpenSSL provider more verbose (vectordotdev#18278) [7952bfb](answerbook/vector@7952bfb) - GitHub * tidy `encode_input` function (vectordotdev#18300) [2a45722](answerbook/vector@2a45722) - GitHub * update `rustls-webpki` due to security advisory (vectordotdev#18344) [b982a74](answerbook/vector@b982a74) - GitHub * Use `Ipv#Addr` constants (vectordotdev#17627) [7a6365e](answerbook/vector@7a6365e) - GitHub * **websites**: Add amplify build spec files to appropriate directories (vectordotdev#18668) [7cbb758](answerbook/vector@7cbb758) - GitHub * **website**: Set download page dropdown to latest version (vectordotdev#18758) [4b72f7e](answerbook/vector@4b72f7e) - Jesse Szwedko * **website**: Update chat.vector.dev redirect (vectordotdev#18635) [76efb4b](answerbook/vector@76efb4b) - GitHub ### Features * Add checksums for artifacts (vectordotdev#18483) [f11eeb3](answerbook/vector@f11eeb3) - GitHub * add convert config command (vectordotdev#18378) [9cb0ca3](answerbook/vector@9cb0ca3) - GitHub * Add git sha to the VRL playground header (vectordotdev#18500) [567de50](answerbook/vector@567de50) - GitHub * add support for YAML and JSON to the generate command (vectordotdev#18345) [e15aec7](answerbook/vector@e15aec7) - GitHub * **appsignal sink**: Normalize metrics (vectordotdev#18217) [61c0ae8](answerbook/vector@61c0ae8) - GitHub * Begin publishing armv7hl rpm packages (vectordotdev#18387) [4c901ed](answerbook/vector@4c901ed) - GitHub * change dedupe config paths to `ConfigTargetPath` (vectordotdev#18241) [4ec6c11](answerbook/vector@4ec6c11) - GitHub * **dev**: add `ENVIRONMENT_AUTOPULL` override to Makefile (vectordotdev#18446) [bc6c421](answerbook/vector@bc6c421) - GitHub * **dev**: add environment networking overrides to Makefile (vectordotdev#18654) [89697d1](answerbook/vector@89697d1) - GitHub * **dev**: add networking overrides to website Makefile (vectordotdev#18655) [75ebda0](answerbook/vector@75ebda0) - GitHub * **dev**: decouple syslog source and codec features (vectordotdev#18381) [aca3a29](answerbook/vector@aca3a29) - GitHub * disable vrl 'string_path' feature (vectordotdev#18188) [5ce5ff1](answerbook/vector@5ce5ff1) - GitHub * **exec source**: add support for customizing command environment (vectordotdev#18223) [e27684c](answerbook/vector@e27684c) - GitHub * **metrics**: support Datadog metric origin metadata (vectordotdev#18405) [587c2e7](answerbook/vector@587c2e7) - GitHub * **new sink**: add AWS Simple Notification Service `aws_sns` sink (vectordotdev#18141) [7b2bddc](answerbook/vector@7b2bddc) - GitHub * **playground**: Create built.rs with versions and expose versions to the UI (vectordotdev#18424) [ce7da4e](answerbook/vector@ce7da4e) - GitHub * **route transform**: Add option to enable/disable unmatched output (vectordotdev#18309) [71343bd](answerbook/vector@71343bd) - GitHub ### Miscellaneous * Merge pull request vectordotdev#390 from answerbook/feature/LOG-18931 [0ce8b8d](answerbook/vector@0ce8b8d) - GitHub [LOG-18931](https://logdna.atlassian.net/browse/LOG-18931) * Merge branch 'master' into feature/LOG-18931 [e8a7404](answerbook/vector@e8a7404) - Darin Spivey [LOG-18931](https://logdna.atlassian.net/browse/LOG-18931) * Merge tag 'v0.33.1' into feature/LOG-18931 [c5efad4](answerbook/vector@c5efad4) - Darin Spivey [LOG-18931](https://logdna.atlassian.net/browse/LOG-18931) [LOG-18931](https://logdna.atlassian.net/browse/LOG-18931) * updating the doc, 2 urls were 404 (vectordotdev#18949) [904cb67](answerbook/vector@904cb67) - Jesse Szwedko * [WEB-3464] Adds TrustArc cookie consent banner (vectordotdev#18741) [69b4623](answerbook/vector@69b4623) - Jesse Szwedko * enhancement (journald source): Add `extra_journalctl_args` option to specify arbitrary command line arguments to journalctl (vectordotdev#18568) [21aec0c](answerbook/vector@21aec0c) - GitHub * Bump `chrono` to 0.4.27 (vectordotdev#18436) [e9feabd](answerbook/vector@e9feabd) - GitHub * fix(codecs) csv encoding quoting bug (vectordotdev#18320) [93ad80d](answerbook/vector@93ad80d) - GitHub * 0.32.0.cue typo (vectordotdev#18270) [f124e70](answerbook/vector@f124e70) - GitHub * Add notes about `ingress_upstreaminfo` log format for `parse_nginx_log()` function (vectordotdev#18477) [5cfb3e4](answerbook/vector@5cfb3e4) - GitHub * add PGO information (vectordotdev#18369) [eac3cd0](answerbook/vector@eac3cd0) - GitHub * **aws provider**: Use FIPS endpoints when configured to do so (vectordotdev#18390) [02c1b4c](answerbook/vector@02c1b4c) - GitHub * **ci**: Add protobuf compatibility check (pt 1) (vectordotdev#18552) [ddb5195](answerbook/vector@ddb5195) - GitHub * **ci**: protobuf compatibility check (pt 2) (vectordotdev#18553) [1eb933d](answerbook/vector@1eb933d) - GitHub * **core**: Add CLI arg and env variable to control openssl probing (vectordotdev#18229) [a4d73ca](answerbook/vector@a4d73ca) - GitHub * **core**: default tokio worker threads to `std::thread::available_parallelism()` (vectordotdev#18541) [730bb15](answerbook/vector@730bb15) - GitHub * **deps**: remove openssl legacy provider flag and update docs (vectordotdev#18609) [a6b1bed](answerbook/vector@a6b1bed) - GitHub * Editorial edits for updated component descriptions (vectordotdev#18590) [d179c57](answerbook/vector@d179c57) - GitHub * **enterprise**: configurable app name (vectordotdev#18554) [2e7e7e7](answerbook/vector@2e7e7e7) - GitHub * **es sink**: separate aws support in es & prometheus sink (vectordotdev#18288) [e652ea4](answerbook/vector@e652ea4) - GitHub * **file source**: fix some typos (vectordotdev#18401) [a3a1ef0](answerbook/vector@a3a1ef0) - GitHub * Fixed NixOS page (vectordotdev#18396) [89f0f08](answerbook/vector@89f0f08) - GitHub * **http_server source**: Configurable http response code (vectordotdev#18208) [afdc66e](answerbook/vector@afdc66e) - GitHub * **kubernetes_logs source**: Expose `oldest_first` (vectordotdev#18376) [dc66566](answerbook/vector@dc66566) - GitHub * **tls**: add new dedicated page for TLS configuration (vectordotdev#18844) [9ca6c7b](answerbook/vector@9ca6c7b) - Jesse Szwedko * **websocket sink**: Allow any data type depending on configured codec (vectordotdev#18295) [1c303e8](answerbook/vector@1c303e8) - GitHub ### **BREAKING CHANGES** * **config:** fix concurrency default & docs (vectordotdev#18651) * **deps:** remove openssl legacy provider flag and update docs (vectordotdev#18609) * **datadog_logs sink:** Use `endpoint` config setting consistent with the other datadog_ sinks. (vectordotdev#18497)

@spencergilbert

# [3.0.0](answerbook/vector@v2.1.2...v3.0.0) (2024-01-17) ### Bug Fixes * **amqp sink**: remove duplicate events (vectordotdev#18932) [a916605](answerbook/vector@a916605) - GitHub * **amqp sink**: remove unnecessary unwrap & emit event dropped errors (vectordotdev#18923) [26f430c](answerbook/vector@26f430c) - GitHub * **amqp sink**: remove unused feature flag (vectordotdev#18948) [cb53588](answerbook/vector@cb53588) - GitHub * **buffers**: apply stricter file permissions to buffer data files when possible (vectordotdev#18895) [cf7298f](answerbook/vector@cf7298f) - GitHub * **clickhouse sink**: fix healthcheck uri (vectordotdev#19067) [7a7b53b](answerbook/vector@7a7b53b) - GitHub * **codecs**: fix 'ProtobufSerializerConfig' input type (vectordotdev#19264) [d2fea65](answerbook/vector@d2fea65) - Jesse Szwedko * **codecs**: native JSON serialization/deserialization for special f64 values (vectordotdev#18650) [39b9298](answerbook/vector@39b9298) - GitHub * **config**: Only try default paths if the path is not specified (vectordotdev#18681) [37fc9db](answerbook/vector@37fc9db) - GitHub * **datadog_agent source, datadog_metrics sink**: handle interval for non-rate series metrics (vectordotdev#18889) [dc9d966](answerbook/vector@dc9d966) - GitHub * **datadog_agent source**: return 200 on empty object payload (vectordotdev#19093) [ab7983a](answerbook/vector@ab7983a) - Jesse Szwedko * **datadog_metrics sink**: evaluate series v1 env var at runtime (vectordotdev#19148) [7b292ce](answerbook/vector@7b292ce) - Jesse Szwedko * **datadog_metrics sink**: improve aggregation performance (vectordotdev#18759) [6a40169](answerbook/vector@6a40169) - GitHub * **datadog_metrics sink**: Revert to using v1 endpoint by default (vectordotdev#19138) [3158f46](answerbook/vector@3158f46) - Jesse Szwedko * **datadog_metrics sink**: the integration tests weren't actually validating anything (vectordotdev#18754) [afc166f](answerbook/vector@afc166f) - GitHub * **datadog_traces sink**: improve request size limiting (vectordotdev#18903) [a477d72](answerbook/vector@a477d72) - GitHub * **debian platform**: Re-add `conf-files` directive for `cargo-deb` (vectordotdev#18726) [e445721](answerbook/vector@e445721) - GitHub * **dev**: Gate config conversion tests (vectordotdev#18698) [c35ae64](answerbook/vector@c35ae64) - GitHub * **dev**: update environment for website development (vectordotdev#18657) [4327776](answerbook/vector@4327776) - GitHub * **dnstap source**: support DNSSEC RRSIG record data (vectordotdev#18878) [ed97f0d](answerbook/vector@ed97f0d) - GitHub * **docker source**: do not emit component error for out of order logs (vectordotdev#18649) [93d7af4](answerbook/vector@93d7af4) - GitHub * **http_server source**: panic when http server receives metric events (vectordotdev#18781) [b107ff7](answerbook/vector@b107ff7) - GitHub * **kafka sink**: Make KafkaService return `Poll::Pending` when producer queue is full (vectordotdev#18770) [a1863e6](answerbook/vector@a1863e6) - GitHub * **kafka source**: fix acknowledgement handling during shutdown and rebalance events (vectordotdev#17497) [f2efb1a](answerbook/vector@f2efb1a) - GitHub * **loki sink**: update to use the global list of compression algorithms (vectordotdev#19099) [218963a](answerbook/vector@218963a) - Jesse Szwedko * **releasing**: Update cargo-deb (vectordotdev#19009) [88194e7](answerbook/vector@88194e7) - GitHub * **releasing**: Update example YAML config data_dir (vectordotdev#18896) [0b27019](answerbook/vector@0b27019) - GitHub * remove gh token call (vectordotdev#19047) [1c864aa](answerbook/vector@1c864aa) - GitHub * **sources**: emit `ComponentEventsDropped` when source send is cancelled (vectordotdev#18859) [a0e2769](answerbook/vector@a0e2769) - GitHub * **tls**: for incoming connection alpn negotiation should be done using set_alpn_select_callback (vectordotdev#18843) [8a5b67e](answerbook/vector@8a5b67e) - GitHub ### Chores * Add SHA256 checksums file to GH releases (vectordotdev#18701) [18f07a0](answerbook/vector@18f07a0) - GitHub * **ci**: Add a summary if the regression workflow is skipped (vectordotdev#18724) [17bd2b1](answerbook/vector@17bd2b1) - GitHub * **ci**: Add a test to assert conf files aren't overwritten (vectordotdev#18728) [3ade682](answerbook/vector@3ade682) - GitHub * **ci**: add dependabot group for futures (vectordotdev#18954) [c23efce](answerbook/vector@c23efce) - GitHub * **ci**: Bump aws-actions/amazon-ecr-login from 1 to 2 (vectordotdev#18752) [8e2032c](answerbook/vector@8e2032c) - GitHub * **ci**: Bump aws-actions/configure-aws-credentials from 4.0.0 to 4.0.1 (vectordotdev#18771) [c9804f0](answerbook/vector@c9804f0) - GitHub * **ci**: Bump bufbuild/buf-setup-action from 1.26.1 to 1.27.0 (vectordotdev#18783) [a784018](answerbook/vector@a784018) - GitHub * **ci**: Bump bufbuild/buf-setup-action from 1.27.0 to 1.27.1 (vectordotdev#18866) [811b7f7](answerbook/vector@811b7f7) - GitHub * **ci**: Bump bufbuild/buf-setup-action from 1.27.1 to 1.27.2 (vectordotdev#18981) [72560b1](answerbook/vector@72560b1) - GitHub * **ci**: Bump check-spelling/check-spelling from 0.0.21 to 0.0.22 (vectordotdev#18723) [f98cd5d](answerbook/vector@f98cd5d) - GitHub * **ci**: Bump MacOS unit test runners to 13 (vectordotdev#18823) [5bf18df](answerbook/vector@5bf18df) - GitHub * **ci**: Bump tspascoal/get-user-teams-membership from 2 to 3 (vectordotdev#18808) [4701bb9](answerbook/vector@4701bb9) - GitHub * **ci**: filter team members from gardener issue comment workflow (vectordotdev#18915) [bf56ac5](answerbook/vector@bf56ac5) - GitHub * **ci**: Fix cookie banner style issues (vectordotdev#18745) [7a55e54](answerbook/vector@7a55e54) - GitHub * **ci**: Remove unusued Dockerfile (vectordotdev#18824) [9d1a676](answerbook/vector@9d1a676) - GitHub * **ci**: Revet bump check-spelling/check-spelling from 0.0.21 to 0.0.22 (vectordotdev#18742) [7dce292](answerbook/vector@7dce292) - GitHub * **ci**: Run deny check nightly instead of on every PR (vectordotdev#18799) [ae117dc](answerbook/vector@ae117dc) - GitHub * **ci**: temporarily peg greptimedb to `v0.4.0` to unblock CI (vectordotdev#18838) [0776cc0](answerbook/vector@0776cc0) - GitHub * convert test config to yaml (vectordotdev#18856) [8b00214](answerbook/vector@8b00214) - GitHub * **core**: Add a CLI flag to allow for empty configs (vectordotdev#19021) [df4921b](answerbook/vector@df4921b) - GitHub * **core**: add more event metadata to proto (vectordotdev#18816) [2deeba1](answerbook/vector@2deeba1) - GitHub * **core**: Refactor `vector-core::stream` into its own package (vectordotdev#18900) [96f4d73](answerbook/vector@96f4d73) - GitHub * **core**: Set up internal topology API (vectordotdev#18919) [c9c184e](answerbook/vector@c9c184e) - GitHub * **datadog_metrics sink**: Set partial Origin Metrics in edge cases (vectordotdev#18677) [3dab239](answerbook/vector@3dab239) - GitHub * **datadog_metrics sink**: support and migrate to the `v2` series API endpoint (vectordotdev#18761) [3485f2c](answerbook/vector@3485f2c) - GitHub * **datadog**: remove deprecated config options (vectordotdev#18940) [f42751d](answerbook/vector@f42751d) - GitHub * **deps**: Add more `dependabot` groups (vectordotdev#18719) [d0e605e](answerbook/vector@d0e605e) - GitHub * **deps**: Bump @babel/traverse from 7.17.0 to 7.23.2 in /website (vectordotdev#18852) [85d2f17](answerbook/vector@85d2f17) - GitHub * **deps**: Bump apache-avro from 0.15.0 to 0.16.0 (vectordotdev#18685) [b37ce3c](answerbook/vector@b37ce3c) - GitHub * **deps**: Bump async-compression from 0.4.3 to 0.4.4 (vectordotdev#18848) [1a8a8cc](answerbook/vector@1a8a8cc) - GitHub * **deps**: Bump async-graphql from 5.0.10 to 6.0.9 (vectordotdev#18988) [4a4eb61](answerbook/vector@4a4eb61) - GitHub * **deps**: Bump async-nats from 0.32.0 to 0.32.1 (vectordotdev#18735) [047c772](answerbook/vector@047c772) - GitHub * **deps**: Bump async-trait from 0.1.73 to 0.1.74 (vectordotdev#18849) [8d82257](answerbook/vector@8d82257) - GitHub * **deps**: Bump base64 from 0.21.4 to 0.21.5 (vectordotdev#18907) [e754dee](answerbook/vector@e754dee) - GitHub * **deps**: Bump bitmask-enum from 2.2.2 to 2.2.3 (vectordotdev#19057) [7a16ee2](answerbook/vector@7a16ee2) - GitHub * **deps**: Bump bstr from 1.6.2 to 1.7.0 (vectordotdev#18810) [91221c6](answerbook/vector@91221c6) - GitHub * **deps**: Bump cached from 0.45.1 to 0.46.0 (vectordotdev#18660) [331c5a0](answerbook/vector@331c5a0) - GitHub * **deps**: Bump cached from 0.46.0 to 0.46.1 (vectordotdev#19058) [51c6b57](answerbook/vector@51c6b57) - GitHub * **deps**: Bump cargo_toml from 0.16.3 to 0.17.0 (vectordotdev#18978) [b55e436](answerbook/vector@b55e436) - GitHub * **deps**: Bump chrono from 0.4.30 to 0.4.31 (vectordotdev#18583) [052ed98](answerbook/vector@052ed98) - GitHub * **deps**: Bump chrono-tz from 0.8.3 to 0.8.4 (vectordotdev#18979) [7f44b4c](answerbook/vector@7f44b4c) - GitHub * **deps**: Bump clap from 4.4.5 to 4.4.6 (vectordotdev#18715) [4d98fdf](answerbook/vector@4d98fdf) - GitHub * **deps**: Bump clap_complete from 4.4.2 to 4.4.3 (vectordotdev#18716) [27b2c93](answerbook/vector@27b2c93) - GitHub * **deps**: Bump console-subscriber from 0.1.10 to 0.2.0 (vectordotdev#18732) [eda0378](answerbook/vector@eda0378) - GitHub * **deps**: Bump csv from 1.2.2 to 1.3.0 (vectordotdev#18768) [7cb8b52](answerbook/vector@7cb8b52) - GitHub * **deps**: Bump dd-rust-license-tool to 1.0.2 (vectordotdev#18711) [570bd52](answerbook/vector@570bd52) - GitHub * **deps**: Bump dyn-clone from 1.0.14 to 1.0.16 (vectordotdev#19040) [8ba28e0](answerbook/vector@8ba28e0) - GitHub * **deps**: Bump fakedata_generator from 0.2.4 to 0.4.0 (vectordotdev#18910) [7debc60](answerbook/vector@7debc60) - GitHub * **deps**: Bump flate2 from 1.0.27 to 1.0.28 (vectordotdev#18850) [b3889bc](answerbook/vector@b3889bc) - GitHub * **deps**: Bump futures-util from 0.3.28 to 0.3.29 (vectordotdev#18951) [741aec3](answerbook/vector@741aec3) - GitHub * **deps**: Bump goauth from 0.13.1 to 0.14.0 (vectordotdev#18872) [1913ee5](answerbook/vector@1913ee5) - GitHub * **deps**: Bump hashbrown from 0.14.0 to 0.14.1 (vectordotdev#18731) [31d92c2](answerbook/vector@31d92c2) - GitHub * **deps**: Bump hashbrown from 0.14.1 to 0.14.2 (vectordotdev#18893) [aebe8db](answerbook/vector@aebe8db) - GitHub * **deps**: Bump indexmap from 2.0.0 to 2.0.1 (vectordotdev#18705) [65643da](answerbook/vector@65643da) - GitHub * **deps**: Bump indexmap from 2.0.1 to 2.0.2 (vectordotdev#18737) [f47df40](answerbook/vector@f47df40) - GitHub * **deps**: Bump inventory from 0.3.12 to 0.3.13 (vectordotdev#19024) [72eacf5](answerbook/vector@72eacf5) - GitHub * **deps**: Bump lading to 0.19.1 (vectordotdev#18869) [2d7c1bb](answerbook/vector@2d7c1bb) - GitHub * **deps**: Bump libc from 0.2.148 to 0.2.149 (vectordotdev#18800) [4002ef0](answerbook/vector@4002ef0) - GitHub * **deps**: Bump libc from 0.2.149 to 0.2.150 (vectordotdev#19059) [611a652](answerbook/vector@611a652) - GitHub * **deps**: Bump lru from 0.11.1 to 0.12.0 (vectordotdev#18767) [c531a3b](answerbook/vector@c531a3b) - GitHub * **deps**: Bump memchr from 2.6.3 to 2.6.4 (vectordotdev#18736) [c7482d0](answerbook/vector@c7482d0) - GitHub * **deps**: Bump memmap2 from 0.7.1 to 0.8.0 (vectordotdev#18659) [ca9e5b4](answerbook/vector@ca9e5b4) - GitHub * **deps**: Bump memmap2 from 0.8.0 to 0.9.0 (vectordotdev#18765) [dcbbb9b](answerbook/vector@dcbbb9b) - GitHub * **deps**: Bump mongodb from 2.6.1 to 2.7.0 (vectordotdev#18703) [539a40f](answerbook/vector@539a40f) - GitHub * **deps**: Bump mongodb from 2.7.0 to 2.7.1 (vectordotdev#19023) [602f630](answerbook/vector@602f630) - GitHub * **deps**: Bump num_enum from 0.7.0 to 0.7.1 (vectordotdev#18975) [c9b6d45](answerbook/vector@c9b6d45) - GitHub * **deps**: Bump num-traits from 0.2.16 to 0.2.17 (vectordotdev#18802) [c4fbc25](answerbook/vector@c4fbc25) - GitHub * **deps**: Bump OpenDAL to v0.41 (vectordotdev#19039) [5655f76](answerbook/vector@5655f76) - GitHub * **deps**: Bump openssl from 0.10.57 to 0.10.58 (vectordotdev#19025) [c4f2d0e](answerbook/vector@c4f2d0e) - GitHub * **deps**: Bump openssl from 0.10.58 to 0.10.59 (vectordotdev#19054) [ee232f8](answerbook/vector@ee232f8) - GitHub * **deps**: Bump openssl-src from 300.1.5+3.1.3 to 300.1.6+3.1.4 (vectordotdev#18936) [a75a043](answerbook/vector@a75a043) - GitHub * **deps**: Bump ordered-float from 4.1.0 to 4.1.1 (vectordotdev#18818) [99643ca](answerbook/vector@99643ca) - GitHub * **deps**: Bump postcss from 8.4.6 to 8.4.31 in /website (vectordotdev#18750) [92d2be9](answerbook/vector@92d2be9) - GitHub * **deps**: Bump proc-macro2 from 1.0.67 to 1.0.69 (vectordotdev#18803) [bc3b3a2](answerbook/vector@bc3b3a2) - GitHub * **deps**: Bump proptest from 1.2.0 to 1.3.1 (vectordotdev#18738) [2b15c63](answerbook/vector@2b15c63) - GitHub * **deps**: Bump pulsar from 6.0.1 to 6.1.0 (vectordotdev#19004) [43f5913](answerbook/vector@43f5913) - GitHub * **deps**: Bump quanta from 0.11.1 to 0.12.0 (vectordotdev#18774) [e9d2dae](answerbook/vector@e9d2dae) - GitHub * **deps**: Bump quanta from 0.12.0 to 0.12.1 (vectordotdev#19005) [051de5a](answerbook/vector@051de5a) - GitHub * **deps**: Bump ratatui from 0.23.0 to 0.24.0 (vectordotdev#18908) [8b56a93](answerbook/vector@8b56a93) - GitHub * **deps**: Bump regex from 1.10.0 to 1.10.2 (vectordotdev#18858) [434849d](answerbook/vector@434849d) - GitHub * **deps**: Bump regex from 1.9.5 to 1.9.6 (vectordotdev#18739) [8e98321](answerbook/vector@8e98321) - GitHub * **deps**: Bump regex from 1.9.6 to 1.10.0 (vectordotdev#18812) [4d02abf](answerbook/vector@4d02abf) - GitHub * **deps**: Bump reqwest from 0.11.20 to 0.11.22 (vectordotdev#18760) [bac60ad](answerbook/vector@bac60ad) - GitHub * **deps**: Bump rustix from 0.37.19 to 0.37.25 (vectordotdev#18879) [3ca32b8](answerbook/vector@3ca32b8) - GitHub * **deps**: Bump semver from 1.0.18 to 1.0.19 (vectordotdev#18662) [aca7753](answerbook/vector@aca7753) - GitHub * **deps**: Bump semver from 1.0.19 to 1.0.20 (vectordotdev#18811) [d9aca80](answerbook/vector@d9aca80) - GitHub * **deps**: Bump serde from 1.0.188 to 1.0.189 (vectordotdev#18834) [0d09898](answerbook/vector@0d09898) - GitHub * **deps**: Bump serde from 1.0.189 to 1.0.190 (vectordotdev#18945) [73afddb](answerbook/vector@73afddb) - GitHub * **deps**: Bump serde_with from 3.3.0 to 3.4.0 (vectordotdev#18874) [33243ac](answerbook/vector@33243ac) - GitHub * **deps**: Bump serde_yaml from 0.9.25 to 0.9.27 (vectordotdev#18956) [2ee96b1](answerbook/vector@2ee96b1) - GitHub * **deps**: Bump serde-toml-merge from 0.3.2 to 0.3.3 (vectordotdev#18804) [abb9101](answerbook/vector@abb9101) - GitHub * **deps**: Bump serde-wasm-bindgen from 0.6.0 to 0.6.1 (vectordotdev#18935) [ffed6f7](answerbook/vector@ffed6f7) - GitHub * **deps**: Bump sha2 from 0.10.7 to 0.10.8 (vectordotdev#18684) [87af0bd](answerbook/vector@87af0bd) - GitHub * **deps**: Bump socket2 from 0.5.4 to 0.5.5 (vectordotdev#18902) [691fdca](answerbook/vector@691fdca) - GitHub * **deps**: Bump syn from 2.0.37 to 2.0.38 (vectordotdev#18789) [ec5238e](answerbook/vector@ec5238e) - GitHub * **deps**: Bump syn from 2.0.38 to 2.0.39 (vectordotdev#19056) [0f0a0b4](answerbook/vector@0f0a0b4) - GitHub * **deps**: Bump tempfile from 3.6.0 to 3.8.0 (vectordotdev#18686) [c0d24b9](answerbook/vector@c0d24b9) - GitHub * **deps**: Bump tempfile from 3.8.0 to 3.8.1 (vectordotdev#18957) [40961ed](answerbook/vector@40961ed) - GitHub * **deps**: Bump the azure group with 4 updates (vectordotdev#18773) [0e61acc](answerbook/vector@0e61acc) - GitHub * **deps**: Bump the azure group with 4 updates (vectordotdev#19052) [53ffbc3](answerbook/vector@53ffbc3) - GitHub * **deps**: Bump the clap group with 1 update (vectordotdev#18906) [cddb835](answerbook/vector@cddb835) - GitHub * **deps**: Bump the clap group with 2 updates (vectordotdev#18925) [78934c2](answerbook/vector@78934c2) - GitHub * **deps**: Bump the futures group with 1 update (vectordotdev#18961) [0a5e3db](answerbook/vector@0a5e3db) - GitHub * **deps**: Bump the tonic group with 2 updates (vectordotdev#18714) [c95df7c](answerbook/vector@c95df7c) - GitHub * **deps**: Bump the zstd group with 1 update (vectordotdev#18826) [96ef9ee](answerbook/vector@96ef9ee) - GitHub * **deps**: Bump thiserror from 1.0.48 to 1.0.49 (vectordotdev#18683) [6b92a83](answerbook/vector@6b92a83) - GitHub * **deps**: Bump thiserror from 1.0.49 to 1.0.50 (vectordotdev#18892) [16df7ea](answerbook/vector@16df7ea) - GitHub * **deps**: Bump tokio from 1.32.0 to 1.33.0 (vectordotdev#18809) [76971bd](answerbook/vector@76971bd) - GitHub * **deps**: Bump tokio-tungstenite from 0.20.0 to 0.20.1 (vectordotdev#18661) [8abed12](answerbook/vector@8abed12) - GitHub * **deps**: Bump toml from 0.8.0 to 0.8.1 (vectordotdev#18687) [efbe673](answerbook/vector@efbe673) - GitHub * **deps**: Bump toml from 0.8.1 to 0.8.2 (vectordotdev#18747) [f9e51e1](answerbook/vector@f9e51e1) - GitHub * **deps**: Bump toml from 0.8.2 to 0.8.3 (vectordotdev#18909) [22402ca](answerbook/vector@22402ca) - GitHub * **deps**: Bump toml from 0.8.3 to 0.8.4 (vectordotdev#18913) [249330a](answerbook/vector@249330a) - GitHub * **deps**: Bump toml from 0.8.4 to 0.8.5 (vectordotdev#18950) [4a525a8](answerbook/vector@4a525a8) - GitHub * **deps**: Bump toml from 0.8.5 to 0.8.6 (vectordotdev#18962) [5e7ae83](answerbook/vector@5e7ae83) - GitHub * **deps**: Bump tracing-log from 0.1.3 to 0.1.4 (vectordotdev#18914) [e4fd78c](answerbook/vector@e4fd78c) - GitHub [log-0](https://logdna.atlassian.net/browse/log-0) [log-0](https://logdna.atlassian.net/browse/log-0) * **deps**: Bump tracing-log from 0.1.4 to 0.2.0 (vectordotdev#18941) [30a1e26](answerbook/vector@30a1e26) - GitHub [log-0](https://logdna.atlassian.net/browse/log-0) [log-0](https://logdna.atlassian.net/browse/log-0) * **deps**: Bump trust-dns-proto from 0.23.0 to 0.23.1 (vectordotdev#18846) [0568d7a](answerbook/vector@0568d7a) - GitHub * **deps**: Bump trust-dns-proto from 0.23.1 to 0.23.2 (vectordotdev#18911) [1eaf8b1](answerbook/vector@1eaf8b1) - GitHub * **deps**: Bump uuid from 1.4.1 to 1.5.0 (vectordotdev#18880) [a025caa](answerbook/vector@a025caa) - GitHub * **deps**: Bump warp from 0.3.5 to 0.3.6 (vectordotdev#18704) [decaaeb](answerbook/vector@decaaeb) - GitHub * **deps**: Bump wasm-bindgen from 0.2.87 to 0.2.88 (vectordotdev#19026) [63bb9e4](answerbook/vector@63bb9e4) - GitHub * **deps**: Bump webpki from 0.22.1 to 0.22.2 (vectordotdev#18744) [4295985](answerbook/vector@4295985) - GitHub * **deps**: Bump wiremock from 0.5.19 to 0.5.21 (vectordotdev#19055) [4622ef6](answerbook/vector@4622ef6) - GitHub * **deps**: Bump zerocopy from 0.7.21 to 0.7.31 (vectordotdev#19394) [74d6cb1](answerbook/vector@74d6cb1) - Jesse Szwedko * **deps**: clean up VRL crate features (vectordotdev#18740) [1452d54](answerbook/vector@1452d54) - GitHub * **deps**: Group csv crate updates (vectordotdev#18797) [98aa157](answerbook/vector@98aa157) - GitHub * **deps**: Remove usages of atty (vectordotdev#18985) [371580c](answerbook/vector@371580c) - GitHub * **deps**: Update dependencies (vectordotdev#18971) [3ead10f](answerbook/vector@3ead10f) - GitHub * **deps**: Update lading to 0.19.0 (vectordotdev#18861) [dc729f5](answerbook/vector@dc729f5) - GitHub * **deps**: Update license-tool.toml webpki version (vectordotdev#18986) [0051ec0](answerbook/vector@0051ec0) - GitHub * **deps**: Update VRL to 0.8.1 (vectordotdev#19011) [f5ea285](answerbook/vector@f5ea285) - GitHub * **dev**: Add `vector-lib` wrapper for three more libs (vectordotdev#18992) [1eb418b](answerbook/vector@1eb418b) - GitHub * **dev**: Add wrapper for `codecs` to `vector-lib` (vectordotdev#18959) [5f30f74](answerbook/vector@5f30f74) - GitHub * **dev**: Add wrapper for `enrichment` to `vector-lib` (vectordotdev#18977) [e61f308](answerbook/vector@e61f308) - GitHub * **dev**: Add wrapper for `file-source` in `vector-lib` (vectordotdev#18984) [f44da16](answerbook/vector@f44da16) - GitHub * **dev**: Add wrapper for `lookup` in `vector-lib` (vectordotdev#18995) [164f1e9](answerbook/vector@164f1e9) - GitHub * **dev**: Add wrapper for `vector-buffers` to `vector-lib` (vectordotdev#18964) [2cef62c](answerbook/vector@2cef62c) - GitHub * **dev**: Detail the format of DEPRECATIONS.md file (vectordotdev#19016) [223dd7b](answerbook/vector@223dd7b) - GitHub * **dev**: Move some macros into `vector-core` (vectordotdev#19002) [4f613ce](answerbook/vector@4f613ce) - GitHub * **dev**: Remove deprecation action item for armv7 RPMs (vectordotdev#19018) [ff7b95f](answerbook/vector@ff7b95f) - GitHub * **dev**: Set up `vector-lib` wrapper crate with `vector-common` (vectordotdev#18927) [00c40d7](answerbook/vector@00c40d7) - GitHub * **dev**: Wrap `vector-config` in `vector-lib` as `configurable` (vectordotdev#18944) [42beb3f](answerbook/vector@42beb3f) - GitHub * **dev**: Wrap `vector-core` in `vector-lib` (vectordotdev#18934) [270fdfd](answerbook/vector@270fdfd) - GitHub * **dev**: Wrap `vector-stream` in `vector-lib` (vectordotdev#18953) [8a02b16](answerbook/vector@8a02b16) - GitHub * **docs**: Add alpha to traces and beta to metrics in descriptions (vectordotdev#19139) [1b9fb9b](answerbook/vector@1b9fb9b) - Jesse Szwedko * **docs**: add highlight post for secrets in disk buffers (vectordotdev#18994) [0cc9389](answerbook/vector@0cc9389) - GitHub * **docs**: Add spec for `listen` option (vectordotdev#18080) [29e5e22](answerbook/vector@29e5e22) - GitHub * **docs**: Replace setup.vector.dev references (vectordotdev#19080) [def235e](answerbook/vector@def235e) - Jesse Szwedko * **docs**: update a few more examples to YAML (vectordotdev#19103) [a59329a](answerbook/vector@a59329a) - Jesse Szwedko * **docs**: update quickstart.md to use YAML (vectordotdev#18796) [0e76fe0](answerbook/vector@0e76fe0) - GitHub * **external docs**: First batch of editorial edits for the Functions doc (vectordotdev#18780) [1da9005](answerbook/vector@1da9005) - GitHub * **external docs**: Remove or replace mentions of vector in functions doc (vectordotdev#18679) [7ad4112](answerbook/vector@7ad4112) - GitHub * Follow redirects for `sh.vector.dev` (vectordotdev#19000) [9893b86](answerbook/vector@9893b86) - GitHub * **gcp_stackdriver_metrics sink**: rewrite to stream based sink (vectordotdev#18749) [92268e4](answerbook/vector@92268e4) - GitHub * **kubernetes**: Regenerate manifests from 0.27.0 chart (vectordotdev#19001) [5e9dd1d](answerbook/vector@5e9dd1d) - GitHub * **metrics**: improve creation of Origin metadata structures (vectordotdev#18788) [f0adce7](answerbook/vector@f0adce7) - GitHub * Note the version to remove the v1 metrics support from the Datadog Metrics sink (vectordotdev#19017) [be9f229](answerbook/vector@be9f229) - GitHub * **observability, blackhole sink**: Don't report by default (vectordotdev#18963) [3b85b48](answerbook/vector@3b85b48) - GitHub * **observability**: deprecate obsolete http metrics (vectordotdev#18972) [f33dce2](answerbook/vector@f33dce2) - GitHub * **observability**: fix tokio unstable (vectordotdev#18776) [67c4beb](answerbook/vector@67c4beb) - GitHub * **observability**: remove `peer_addr` internal metric tag (vectordotdev#18982) [b9447f6](answerbook/vector@b9447f6) - GitHub * **observability**: remove deprecated `component_name` metric tag (vectordotdev#18942) [c6f5d2b](answerbook/vector@c6f5d2b) - GitHub * **observability**: remove metrics replaced by component_errors_total (vectordotdev#18965) [17f4ed2](answerbook/vector@17f4ed2) - GitHub * **prometheus_remote_write sink**: remote write sink rewrite (vectordotdev#18676) [5c1707f](answerbook/vector@5c1707f) - GitHub * **regression**: Unmark regression tests as erratic now (vectordotdev#19020) [2cdf654](answerbook/vector@2cdf654) - GitHub * **releasing, kubernetes**: Update manifests to v0.26.0 of the chart (vectordotdev#18694) [5f4c3ba](answerbook/vector@5f4c3ba) - GitHub * **releasing**: Add deprecation note about respositories.timber.io deprecation (vectordotdev#19078) [09df599](answerbook/vector@09df599) - Jesse Szwedko * **releasing**: Add known issue for 0.33.0 debian packaging regression (vectordotdev#18727) [ff745ab](answerbook/vector@ff745ab) - GitHub * **releasing**: Add known issue for Datadog Metrics sink in v0.34.0 (vectordotdev#19122) [cee9d07](answerbook/vector@cee9d07) - Jesse Szwedko * **releasing**: Add known issue for protobuf encoder in v0.34.0 (vectordotdev#19244) [f7c3824](answerbook/vector@f7c3824) - Jesse Szwedko * **releasing**: Add upgrade note about TOML breaking change to v0.34.0 (vectordotdev#19120) [dba0ba1](answerbook/vector@dba0ba1) - Jesse Szwedko * **releasing**: Bump Vector version to v0.34.0 (vectordotdev#18693) [1c2f970](answerbook/vector@1c2f970) - GitHub * **releasing**: Fix changelog not for kafka fix in 0.33.1 (vectordotdev#19032) [2501049](answerbook/vector@2501049) - GitHub * **releasing**: Fix formatting for v0.34.0 release note (vectordotdev#19085) [3569271](answerbook/vector@3569271) - Jesse Szwedko * **releasing**: Prepare v0.33.0 release [682f0e0](answerbook/vector@682f0e0) - Jesse Szwedko * **releasing**: Prepare v0.33.1 release [409b69d](answerbook/vector@409b69d) - Jesse Szwedko * **releasing**: Prepare v0.34.0 release [c909b66](answerbook/vector@c909b66) - Jesse Szwedko * **releasing**: Prepare v0.34.1 release [86f1c22](answerbook/vector@86f1c22) - Jesse Szwedko * **releasing**: Prepare v0.34.2 release [d685a16](answerbook/vector@d685a16) - Jesse Szwedko * **releasing**: Typo in v0.33.1 release docs (vectordotdev#18987) [36974a0](answerbook/vector@36974a0) - GitHub * Remove @spencergilbert from CODEOWNERS (vectordotdev#18778) [f300c85](answerbook/vector@f300c85) - GitHub * Remove armv7 RPM package (vectordotdev#18837) [eca8c76](answerbook/vector@eca8c76) - GitHub * remove config/vector.toml (vectordotdev#18833) [efb0d1a](answerbook/vector@efb0d1a) - GitHub * **security**: Ignore RUSTSEC-2023-0071 for now (vectordotdev#19263) [e27b7bd](answerbook/vector@e27b7bd) - Jesse Szwedko * **security**: Remove legacy OpenSSL provider flags (vectordotdev#19015) [2bba40a](answerbook/vector@2bba40a) - GitHub * **sinks**: Update `PartitionBatcher` to use `BatchConfig` (vectordotdev#18792) [4a7d0c3](answerbook/vector@4a7d0c3) - GitHub * Update *-release.md issue templates for vector.dev package release (vectordotdev#18814) [ab8f8d2](answerbook/vector@ab8f8d2) - GitHub * **vrl**: Revive old remap tests (vectordotdev#18678) [53cad38](answerbook/vector@53cad38) - GitHub * **website**: Set download page dropdown to latest version (vectordotdev#18758) [23745f2](answerbook/vector@23745f2) - GitHub * **websites**: Setup preview site workflows (vectordotdev#18924) [9d006c7](answerbook/vector@9d006c7) - GitHub * **websites**: Workflow updates (vectordotdev#19036) [b4ca866](answerbook/vector@b4ca866) - GitHub * **website**: WEB-4247 | Update references from s3 to setup.vector.dev (vectordotdev#19149) [9e1ad37](answerbook/vector@9e1ad37) - Jesse Szwedko * **website**: WEB-4275 | Update Navigation (vectordotdev#19186) [73a668c](answerbook/vector@73a668c) - Doug Smith * **website**: Workflow fixes (vectordotdev#19046) [fb63f8e](answerbook/vector@fb63f8e) - GitHub ### Continuous Integration * mark otlp_http_to_blackhole experiment erratic (vectordotdev#18786) [54b54a5](answerbook/vector@54b54a5) - GitHub ### Features * add PR comment trigger for the workload checks workflow (vectordotdev#18839) [11bc5d9](answerbook/vector@11bc5d9) - GitHub * **amqp**: added integration test for TLS (vectordotdev#18813) [3c4ae86](answerbook/vector@3c4ae86) - GitHub [LOG-16435](https://logdna.atlassian.net/browse/LOG-16435) * **ci**: Add Vector workload checks (vectordotdev#18569) [e2b7de0](answerbook/vector@e2b7de0) - GitHub * **codecs**: add support for protobuf encoding (vectordotdev#18598) [737f5c3](answerbook/vector@737f5c3) - GitHub * **docs**: add fallibility examples (vectordotdev#18931) [08b45a5](answerbook/vector@08b45a5) - GitHub * **journald source**: Add emit_cursor option (vectordotdev#18882) [74051dc](answerbook/vector@74051dc) - GitHub * **regression**: convert all regression cases configs to YAML (vectordotdev#18825) [1fb0f0d](answerbook/vector@1fb0f0d) - GitHub * **timestamp encoding**: add unixtime formats (vectordotdev#18817) [53039e7](answerbook/vector@53039e7) - GitHub ### Miscellaneous * Merge pull request vectordotdev#396 from answerbook/feature/LOG-18978 [2b046bf](answerbook/vector@2b046bf) - GitHub [LOG-18978](https://logdna.atlassian.net/browse/LOG-18978) * Merge tag 'v0.34.2' into feature/LOG-18978 [a88d5ed](answerbook/vector@a88d5ed) - Darin Spivey [LOG-18978](https://logdna.atlassian.net/browse/LOG-18978) [LOG-18978](https://logdna.atlassian.net/browse/LOG-18978) * Update RUM domain (vectordotdev#19367) [4acb9de](answerbook/vector@4acb9de) - Jesse Szwedko * chore(docs):Add Obs Pipelines to docs (vectordotdev#19201) [8f3f160](answerbook/vector@8f3f160) - Jesse Szwedko * updating the doc, 2 urls were 404 (vectordotdev#18949) [bf58b06](answerbook/vector@bf58b06) - GitHub * chore!(config, docs): delete deprecated vector.toml code (vectordotdev#18795) [c8557d0](answerbook/vector@c8557d0) - GitHub * [WEB-3464] Adds TrustArc cookie consent banner (vectordotdev#18741) [43428d3](answerbook/vector@43428d3) - GitHub * Add announcement for new repository URLs (vectordotdev#18798) [239cf94](answerbook/vector@239cf94) - GitHub * **examples**: Convert config/examples from TOML to YAML (vectordotdev#18832) [6ffb072](answerbook/vector@6ffb072) - GitHub * **external docs**: Fix metrics test example (vectordotdev#18725) [96def01](answerbook/vector@96def01) - GitHub * fix truncate arguments (vectordotdev#19068) [9a5cb51](answerbook/vector@9a5cb51) - Jesse Szwedko * **gcp_pubsub source**: Add required fields to documentation examples (vectordotdev#18998) [21f741d](answerbook/vector@21f741d) - GitHub * **nats source**: add subscriber_capacity option (vectordotdev#18899) [e7b563d](answerbook/vector@e7b563d) - GitHub * **sources, sinks**: add telemetry to http and grpc servers (vectordotdev#18887) [e779019](answerbook/vector@e779019) - GitHub * **tls**: add new dedicated page for TLS configuration (vectordotdev#18844) [625e4bd](answerbook/vector@625e4bd) - GitHub * **vrl**: add an example of parsing upstreaminfo with parse_nginx_log (vectordotdev#18815) [774094e](answerbook/vector@774094e) - GitHub ### **BREAKING CHANGES** * **security:** Remove legacy OpenSSL provider flags (vectordotdev#19015) * **observability:** remove metrics replaced by component_errors_total (vectordotdev#18965) * **observability:** remove `peer_addr` internal metric tag (vectordotdev#18982) * **observability, blackhole sink:** Don't report by default (vectordotdev#18963) * **observability:** remove deprecated `component_name` metric tag (vectordotdev#18942) * **datadog:** remove deprecated config options (vectordotdev#18940)

fix(datadog_metrics sink): improve aggregation performance

d6edd12

dsmith3197 requested a review from a team October 3, 2023 21:15

dsmith3197 marked this pull request as draft October 3, 2023 21:15

github-actions bot added the domain: sinks Anything related to the Vector's sinks label Oct 3, 2023

github-advanced-security bot found potential problems Oct 3, 2023

View reviewed changes

src/sinks/datadog/metrics/sink.rs Fixed Show fixed Hide fixed

fix tests and add concurrency

8a9f9bf

dsmith3197 commented Oct 4, 2023

View reviewed changes

dsmith3197 marked this pull request as ready for review October 4, 2023 15:41

lukesteensen reviewed Oct 4, 2023

View reviewed changes

bruceg reviewed Oct 4, 2023

View reviewed changes

src/sinks/datadog/metrics/sink.rs Outdated Show resolved Hide resolved

src/sinks/datadog/metrics/sink.rs Outdated Show resolved Hide resolved

src/sinks/util/buffer/metrics/sort.rs Outdated Show resolved Hide resolved

feedback

aca6278

dsmith3197 requested review from lukesteensen and bruceg October 4, 2023 20:48

dsmith3197 added this to the Vector v0.33.1 milestone Oct 4, 2023

dsmith3197 commented Oct 4, 2023

View reviewed changes

bruceg approved these changes Oct 4, 2023

View reviewed changes

neuronull approved these changes Oct 4, 2023

View reviewed changes

lukesteensen approved these changes Oct 4, 2023

View reviewed changes

add comment to dedupe_by

273a065

dsmith3197 enabled auto-merge October 5, 2023 13:10

sort by metric type as well

472fdb4

dsmith3197 added this pull request to the merge queue Oct 5, 2023

github-merge-queue bot removed this pull request from the merge queue due to failed status checks Oct 5, 2023

fix flakey proptest

e76e08c

dsmith3197 enabled auto-merge October 5, 2023 17:57

only dedupe non-counters in prop test

a1eb365

goxberry mentioned this pull request Oct 5, 2023

ci: mark otlp_http_to_blackhole experiment erratic #18786

Merged

dsmith3197 added this pull request to the merge queue Oct 5, 2023

Merged via the queue into master with commit 6a40169 Oct 5, 2023
43 checks passed

dsmith3197 deleted the dougsmith/datadog-metrics-aggregation branch October 5, 2023 21:01

dsmith3197 mentioned this pull request Nov 7, 2023

Internal Metrics Hangs Forever #16561

Open

dsmith3197 mentioned this pull request Nov 27, 2023

Vector Crashes during Loadtests when allowing backpressure #19184

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix(datadog_metrics sink): improve aggregation performance #18759

fix(datadog_metrics sink): improve aggregation performance #18759

dsmith3197 commented Oct 3, 2023 •

edited

Loading

netlify bot commented Oct 3, 2023 •

edited

Loading

netlify bot commented Oct 3, 2023 •

edited

Loading

dsmith3197 commented Oct 3, 2023

aws-amplify-us-east-1 bot commented Oct 3, 2023

aws-amplify-us-east-1 bot commented Oct 3, 2023

aws-amplify-us-east-1 bot commented Oct 3, 2023

datadog-vectordotdev bot commented Oct 3, 2023 •

edited

Loading

github-actions bot commented Oct 3, 2023

dsmith3197 commented Oct 4, 2023

dsmith3197 Oct 4, 2023

github-actions bot commented Oct 4, 2023

lukesteensen Oct 4, 2023

dsmith3197 Oct 4, 2023

lukesteensen Oct 4, 2023

dsmith3197 Oct 4, 2023 •

edited

Loading

lukesteensen Oct 4, 2023

dsmith3197 Oct 4, 2023

github-actions bot commented Oct 5, 2023

github-actions bot commented Oct 5, 2023

fix(datadog_metrics sink): improve aggregation performance #18759

fix(datadog_metrics sink): improve aggregation performance #18759

Conversation

dsmith3197 commented Oct 3, 2023 • edited Loading

Description

Regression Results

Changing the Algorithm

Adding Concurrency

netlify bot commented Oct 3, 2023 • edited Loading

✅ Deploy Preview for vrl-playground canceled.

netlify bot commented Oct 3, 2023 • edited Loading

✅ Deploy Preview for vector-project canceled.

dsmith3197 commented Oct 3, 2023

aws-amplify-us-east-1 bot commented Oct 3, 2023

aws-amplify-us-east-1 bot commented Oct 3, 2023

aws-amplify-us-east-1 bot commented Oct 3, 2023

datadog-vectordotdev bot commented Oct 3, 2023 • edited Loading

Datadog Report

github-actions bot commented Oct 3, 2023

Regression Detector Results

dsmith3197 commented Oct 4, 2023

dsmith3197 Oct 4, 2023

Choose a reason for hiding this comment

github-actions bot commented Oct 4, 2023

Regression Detector Results

lukesteensen Oct 4, 2023

Choose a reason for hiding this comment

dsmith3197 Oct 4, 2023

Choose a reason for hiding this comment

lukesteensen Oct 4, 2023

Choose a reason for hiding this comment

dsmith3197 Oct 4, 2023 • edited Loading

Choose a reason for hiding this comment

lukesteensen Oct 4, 2023

Choose a reason for hiding this comment

dsmith3197 Oct 4, 2023

Choose a reason for hiding this comment

github-actions bot commented Oct 5, 2023

Regression Detector Results

github-actions bot commented Oct 5, 2023

Regression Detector Results

dsmith3197 commented Oct 3, 2023 •

edited

Loading

netlify bot commented Oct 3, 2023 •

edited

Loading

netlify bot commented Oct 3, 2023 •

edited

Loading

datadog-vectordotdev bot commented Oct 3, 2023 •

edited

Loading

dsmith3197 Oct 4, 2023 •

edited

Loading