chore(prometheus_remote_write sink): remote write sink rewrite #18676

StephenWakely · 2023-09-26T16:00:14Z

This rewrites the Prometheus Remote Write sink to a stream based sink.

After discussing on Slack, I would like for us to merge this PR as is and address the following question in a followup PR.

Questions have arisen over whether this sink should be aggregating metrics within a batch. This was the old behaviour, but it is not clear that this is correct.

I have added a new option which allows this aggregation to be switched off batch.aggregate = false.

The default is currently to aggregate to keep it inline with the current behaviour. We will likely change the default to not aggregate in a future version, and potentially deprecate and remove the option altogether in time.

Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>

…e_sink_rewrite

Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>

netlify · 2023-09-26T16:00:20Z

✅ Deploy Preview for vrl-playground canceled.

Name	Link
🔨 Latest commit	`e014b29`
🔍 Latest deploy log	https://app.netlify.com/sites/vrl-playground/deploys/6537bf0d550c250008f8e76e

netlify · 2023-09-26T16:00:20Z

✅ Deploy Preview for vector-project canceled.

Name	Link
🔨 Latest commit	`e014b29`
🔍 Latest deploy log	https://app.netlify.com/sites/vector-project/deploys/6537bf0d07277f000713ebc2

Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>

…rewrite

Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>

datadog-vectordotdev · 2023-09-27T09:23:54Z

Datadog Report

Branch report: stephen/remote_write_sink_rewrite
Commit report: b4880e4

✅ vector: 0 Failed, 0 New Flaky, 2180 Passed, 0 Skipped, 24m 6.97s Wall Time

neuronull

LGTM

neuronull · 2023-09-27T16:16:28Z

src/sinks/prometheus/remote_write/mod.rs

+impl From<Compression> for crate::sinks::util::Compression {
+    fn from(value: Compression) -> Self {
+        match value {
+            Compression::Snappy => crate::sinks::util::Compression::Snappy,
+            Compression::Gzip => crate::sinks::util::Compression::Gzip(Default::default()),
+            Compression::Zstd => crate::sinks::util::Compression::Zstd(Default::default()),


🥜 nitpick: ‏might be slightly cleaner looking to have the local Compression enum renamed to something like PrometheusRWCompression and then have a use statement for the util one.

src/sinks/prometheus/remote_write/service.rs

src/sinks/util/snappy.rs

Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>

…nk_rewrite Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>

Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>

datadog-vectordotdev · 2023-10-19T10:02:29Z

Datadog Report

Branch report: stephen/remote_write_sink_rewrite
Commit report: c5d59af

✅ vector: 0 Failed, 0 New Flaky, 2220 Passed, 0 Skipped, 28m 20.71s Wall Time

Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>

dsmith3197 · 2023-10-19T14:05:42Z

website/cue/reference/components/sinks/base/prometheus_remote_write.cue

+				gzip: """
+					[Gzip][gzip] compression.
+
+					[gzip]: https://www.gzip.org/
+					"""
+				none: "No compression."
+				zlib: """
+					[Zlib][zlib] compression.
+
+					[zlib]: https://zlib.net/
+					"""
+				zstd: """
+					[Zstandard][zstd] compression.
+
+					[zstd]: https://facebook.github.io/zstd/
+					"""


It looks like the docs are missing the snappy option. It should be automatically added to all components using the Compression enum, no?

Ah, good catch.

Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>

…rewrite Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>

github-actions · 2023-10-24T12:00:33Z

Regression Detector Results

Run ID: 38ac708e-ae59-4b48-889a-48e8b5fffa42
Baseline: 1eaf8b1
Comparison: 64f73f0
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine quickly if vector performance is changed and to what degree by a pull request.

Because a target's optimization goal performance in each experiment will vary somewhat each time it is run, we can only estimate mean differences in optimization goal relative to the baseline target. We express these differences as a percentage change relative to the baseline target, denoted "Δ mean %". These estimates are made to a precision that balances accuracy and cost control. We represent this precision as a 90.00% confidence interval denoted "Δ mean % CI": there is a 90.00% chance that the true value of "Δ mean %" is in that interval.

We decide whether a change in performance is a "regression" -- a change worth investigating further -- if both of the following two criteria are true:

The estimated |Δ mean %| ≥ 5.00%. This criterion intends to answer the question "Does the estimated change in mean optimization goal performance have a meaningful impact on your customers?". We assume that when |Δ mean %| < 5.00%, the impact on your customers is not meaningful. We also assume that a performance change in optimization goal is worth investigating whether it is an increase or decrease, so long as the magnitude of the change is sufficiently large.
Zero is not in the 90.00% confidence interval "Δ mean % CI" about "Δ mean %". This statement is equivalent to saying that there is at least a 90.00% chance that the mean difference in optimization goal is not zero. This criterion intends to answer the question, "Is there a statistically significant difference in mean optimization goal performance?". It also means there is no more than a 10.00% chance this criterion reports a statistically significant difference when the true difference in mean optimization goal is zero -- a "false positive". We assume you are willing to accept a 10.00% chance of inaccurately detecting a change in performance when no true difference exists.

The table below, if present, lists those experiments that have experienced a statistically significant change in mean optimization goal performance between baseline and comparison SHAs with 90.00% confidence OR have been detected as newly erratic. Negative values of "Δ mean %" mean that baseline is faster, whereas positive values of "Δ mean %" mean that comparison is faster. Results that do not exhibit more than a ±5.00% change in their mean optimization goal are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting change is observed.

No interesting changes in experiment optimization goals with confidence ≥ 90.00% and |Δ mean %| ≥ 5.00%.

Fine details of change detection per experiment.

experiment	goal	Δ mean %	Δ mean % CI	confidence
otlp_http_to_blackhole	ingress throughput	+3.43	[+3.27, +3.59]	100.00%
syslog_loki	ingress throughput	+1.33	[+1.22, +1.43]	100.00%
splunk_hec_route_s3	ingress throughput	+0.90	[+0.35, +1.44]	99.34%
syslog_log2metric_splunk_hec_metrics	ingress throughput	+0.87	[+0.74, +1.01]	100.00%
socket_to_socket_blackhole	ingress throughput	+0.87	[+0.80, +0.94]	100.00%
datadog_agent_remap_blackhole_acks	ingress throughput	+0.77	[+0.67, +0.87]	100.00%
otlp_grpc_to_blackhole	ingress throughput	+0.69	[+0.60, +0.78]	100.00%
fluent_elasticsearch	ingress throughput	+0.64	[+0.20, +1.08]	98.22%
http_to_http_noack	ingress throughput	+0.12	[+0.04, +0.21]	98.11%
http_to_http_json	ingress throughput	+0.07	[-0.02, +0.16]	78.83%
splunk_hec_indexer_ack_blackhole	ingress throughput	-0.01	[-0.17, +0.16]	4.30%
splunk_hec_to_splunk_hec_logs_noack	ingress throughput	-0.04	[-0.16, +0.08]	39.60%
http_to_http_acks	ingress throughput	-0.04	[-1.34, +1.26]	4.09%
splunk_hec_to_splunk_hec_logs_acks	ingress throughput	-0.07	[-0.22, +0.08]	55.68%
enterprise_http_to_http	ingress throughput	-0.12	[-0.21, -0.02]	96.17%
http_to_s3	ingress throughput	-0.33	[-0.62, -0.04]	94.26%
http_text_to_http_json	ingress throughput	-0.37	[-0.50, -0.23]	100.00%
datadog_agent_remap_datadog_logs_acks	ingress throughput	-0.44	[-0.59, -0.28]	100.00%
syslog_regex_logs2metric_ddmetrics	ingress throughput	-0.50	[-0.63, -0.37]	100.00%
syslog_humio_logs	ingress throughput	-1.13	[-1.27, -0.99]	100.00%
datadog_agent_remap_blackhole	ingress throughput	-1.45	[-1.55, -1.34]	100.00%
file_to_blackhole	egress throughput	-1.51	[-3.91, +0.89]	69.94%
datadog_agent_remap_datadog_logs	ingress throughput	-1.62	[-1.77, -1.46]	100.00%
syslog_log2metric_humio_metrics	ingress throughput	-2.53	[-2.67, -2.38]	100.00%
syslog_splunk_hec_logs	ingress throughput	-3.55	[-3.63, -3.48]	100.00%

Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>

github-actions · 2023-10-24T14:47:19Z

Regression Detector Results

Run ID: 4d052dc7-4be6-4cad-a639-11a286704b88
Baseline: 1eaf8b1
Comparison: b5c474f
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine quickly if vector performance is changed and to what degree by a pull request.

Because a target's optimization goal performance in each experiment will vary somewhat each time it is run, we can only estimate mean differences in optimization goal relative to the baseline target. We express these differences as a percentage change relative to the baseline target, denoted "Δ mean %". These estimates are made to a precision that balances accuracy and cost control. We represent this precision as a 90.00% confidence interval denoted "Δ mean % CI": there is a 90.00% chance that the true value of "Δ mean %" is in that interval.

We decide whether a change in performance is a "regression" -- a change worth investigating further -- if both of the following two criteria are true:

The estimated |Δ mean %| ≥ 5.00%. This criterion intends to answer the question "Does the estimated change in mean optimization goal performance have a meaningful impact on your customers?". We assume that when |Δ mean %| < 5.00%, the impact on your customers is not meaningful. We also assume that a performance change in optimization goal is worth investigating whether it is an increase or decrease, so long as the magnitude of the change is sufficiently large.
Zero is not in the 90.00% confidence interval "Δ mean % CI" about "Δ mean %". This statement is equivalent to saying that there is at least a 90.00% chance that the mean difference in optimization goal is not zero. This criterion intends to answer the question, "Is there a statistically significant difference in mean optimization goal performance?". It also means there is no more than a 10.00% chance this criterion reports a statistically significant difference when the true difference in mean optimization goal is zero -- a "false positive". We assume you are willing to accept a 10.00% chance of inaccurately detecting a change in performance when no true difference exists.

The table below, if present, lists those experiments that have experienced a statistically significant change in mean optimization goal performance between baseline and comparison SHAs with 90.00% confidence OR have been detected as newly erratic. Negative values of "Δ mean %" mean that baseline is faster, whereas positive values of "Δ mean %" mean that comparison is faster. Results that do not exhibit more than a ±5.00% change in their mean optimization goal are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting change is observed.

Changes in experiment optimization goals with confidence ≥ 90.00% and |Δ mean %| ≥ 5.00%:

experiment	goal	Δ mean %	confidence
syslog_log2metric_humio_metrics	ingress throughput	-6.06	100.00%

Fine details of change detection per experiment.

experiment	goal	Δ mean %	Δ mean % CI	confidence
datadog_agent_remap_datadog_logs	ingress throughput	+2.75	[+2.59, +2.90]	100.00%
datadog_agent_remap_blackhole_acks	ingress throughput	+2.42	[+2.27, +2.56]	100.00%
datadog_agent_remap_datadog_logs_acks	ingress throughput	+2.08	[+1.91, +2.26]	100.00%
http_text_to_http_json	ingress throughput	+1.79	[+1.65, +1.93]	100.00%
otlp_http_to_blackhole	ingress throughput	+1.18	[+1.04, +1.33]	100.00%
datadog_agent_remap_blackhole	ingress throughput	+1.16	[+1.06, +1.26]	100.00%
syslog_loki	ingress throughput	+1.03	[+0.97, +1.10]	100.00%
socket_to_socket_blackhole	ingress throughput	+0.73	[+0.66, +0.80]	100.00%
fluent_elasticsearch	ingress throughput	+0.64	[+0.19, +1.08]	98.14%
otlp_grpc_to_blackhole	ingress throughput	+0.58	[+0.49, +0.68]	100.00%
http_to_http_noack	ingress throughput	+0.15	[+0.05, +0.25]	98.43%
http_to_http_json	ingress throughput	+0.03	[-0.05, +0.12]	45.79%
splunk_hec_indexer_ack_blackhole	ingress throughput	+0.01	[-0.15, +0.17]	7.56%
splunk_hec_to_splunk_hec_logs_acks	ingress throughput	-0.00	[-0.15, +0.14]	2.99%
splunk_hec_to_splunk_hec_logs_noack	ingress throughput	-0.08	[-0.20, +0.05]	70.19%
enterprise_http_to_http	ingress throughput	-0.17	[-0.26, -0.08]	99.87%
http_to_http_acks	ingress throughput	-0.39	[-1.68, +0.91]	37.70%
syslog_splunk_hec_logs	ingress throughput	-0.54	[-0.61, -0.47]	100.00%
http_to_s3	ingress throughput	-0.57	[-0.86, -0.28]	99.89%
splunk_hec_route_s3	ingress throughput	-0.78	[-1.33, -0.24]	98.13%
syslog_log2metric_splunk_hec_metrics	ingress throughput	-1.02	[-1.19, -0.85]	100.00%
syslog_regex_logs2metric_ddmetrics	ingress throughput	-1.80	[-1.96, -1.64]	100.00%
file_to_blackhole	egress throughput	-3.17	[-5.66, -0.68]	96.39%
syslog_humio_logs	ingress throughput	-4.75	[-5.16, -4.34]	100.00%
syslog_log2metric_humio_metrics	ingress throughput	-6.06	[-6.25, -5.86]	100.00%

github-actions · 2023-10-25T10:17:43Z

Regression Detector Results

Run ID: b9363325-870b-412e-b902-2e4d5024856c
Baseline: a1863e6
Comparison: 5c1707f
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine quickly if vector performance is changed and to what degree by a pull request.

Because a target's optimization goal performance in each experiment will vary somewhat each time it is run, we can only estimate mean differences in optimization goal relative to the baseline target. We express these differences as a percentage change relative to the baseline target, denoted "Δ mean %". These estimates are made to a precision that balances accuracy and cost control. We represent this precision as a 90.00% confidence interval denoted "Δ mean % CI": there is a 90.00% chance that the true value of "Δ mean %" is in that interval.

We decide whether a change in performance is a "regression" -- a change worth investigating further -- if both of the following two criteria are true:

The estimated |Δ mean %| ≥ 5.00%. This criterion intends to answer the question "Does the estimated change in mean optimization goal performance have a meaningful impact on your customers?". We assume that when |Δ mean %| < 5.00%, the impact on your customers is not meaningful. We also assume that a performance change in optimization goal is worth investigating whether it is an increase or decrease, so long as the magnitude of the change is sufficiently large.
Zero is not in the 90.00% confidence interval "Δ mean % CI" about "Δ mean %". This statement is equivalent to saying that there is at least a 90.00% chance that the mean difference in optimization goal is not zero. This criterion intends to answer the question, "Is there a statistically significant difference in mean optimization goal performance?". It also means there is no more than a 10.00% chance this criterion reports a statistically significant difference when the true difference in mean optimization goal is zero -- a "false positive". We assume you are willing to accept a 10.00% chance of inaccurately detecting a change in performance when no true difference exists.

The table below, if present, lists those experiments that have experienced a statistically significant change in mean optimization goal performance between baseline and comparison SHAs with 90.00% confidence OR have been detected as newly erratic. Negative values of "Δ mean %" mean that baseline is faster, whereas positive values of "Δ mean %" mean that comparison is faster. Results that do not exhibit more than a ±5.00% change in their mean optimization goal are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting change is observed.

No interesting changes in experiment optimization goals with confidence ≥ 90.00% and |Δ mean %| ≥ 5.00%.

Fine details of change detection per experiment.

experiment	goal	Δ mean %	Δ mean % CI	confidence
file_to_blackhole	egress throughput	+2.80	[+0.39, +5.21]	94.40%
syslog_log2metric_humio_metrics	ingress throughput	+2.14	[+2.02, +2.26]	100.00%
syslog_splunk_hec_logs	ingress throughput	+0.96	[+0.87, +1.05]	100.00%
datadog_agent_remap_blackhole_acks	ingress throughput	+0.87	[+0.76, +0.98]	100.00%
otlp_grpc_to_blackhole	ingress throughput	+0.70	[+0.61, +0.79]	100.00%
syslog_log2metric_splunk_hec_metrics	ingress throughput	+0.60	[+0.45, +0.75]	100.00%
syslog_humio_logs	ingress throughput	+0.25	[+0.15, +0.36]	99.99%
http_text_to_http_json	ingress throughput	+0.18	[+0.06, +0.30]	98.22%
syslog_loki	ingress throughput	+0.11	[+0.04, +0.17]	99.01%
http_to_http_noack	ingress throughput	+0.10	[+0.01, +0.19]	93.10%
datadog_agent_remap_blackhole	ingress throughput	+0.08	[-0.01, +0.18]	84.45%
splunk_hec_to_splunk_hec_logs_acks	ingress throughput	+0.01	[-0.13, +0.16]	11.11%
http_to_http_json	ingress throughput	+0.01	[-0.08, +0.09]	12.32%
splunk_hec_indexer_ack_blackhole	ingress throughput	-0.01	[-0.17, +0.15]	7.46%
splunk_hec_to_splunk_hec_logs_noack	ingress throughput	-0.01	[-0.13, +0.11]	10.43%
enterprise_http_to_http	ingress throughput	-0.07	[-0.16, +0.02]	82.87%
datadog_agent_remap_datadog_logs	ingress throughput	-0.09	[-0.25, +0.07]	65.72%
http_to_http_acks	ingress throughput	-0.27	[-1.57, +1.03]	26.86%
http_to_s3	ingress throughput	-0.34	[-0.62, -0.05]	94.88%
syslog_regex_logs2metric_ddmetrics	ingress throughput	-0.65	[-0.75, -0.56]	100.00%
splunk_hec_route_s3	ingress throughput	-0.82	[-1.35, -0.29]	98.86%
socket_to_socket_blackhole	ingress throughput	-0.99	[-1.06, -0.91]	100.00%
fluent_elasticsearch	ingress throughput	-1.38	[-1.83, -0.93]	100.00%
datadog_agent_remap_datadog_logs_acks	ingress throughput	-1.50	[-1.66, -1.34]	100.00%
otlp_http_to_blackhole	ingress throughput	-1.81	[-1.96, -1.66]	100.00%

@spencergilbert

# [3.0.0](answerbook/vector@v2.1.2...v3.0.0) (2024-01-17) ### Bug Fixes * **amqp sink**: remove duplicate events (vectordotdev#18932) [a916605](answerbook/vector@a916605) - GitHub * **amqp sink**: remove unnecessary unwrap & emit event dropped errors (vectordotdev#18923) [26f430c](answerbook/vector@26f430c) - GitHub * **amqp sink**: remove unused feature flag (vectordotdev#18948) [cb53588](answerbook/vector@cb53588) - GitHub * **buffers**: apply stricter file permissions to buffer data files when possible (vectordotdev#18895) [cf7298f](answerbook/vector@cf7298f) - GitHub * **clickhouse sink**: fix healthcheck uri (vectordotdev#19067) [7a7b53b](answerbook/vector@7a7b53b) - GitHub * **codecs**: fix 'ProtobufSerializerConfig' input type (vectordotdev#19264) [d2fea65](answerbook/vector@d2fea65) - Jesse Szwedko * **codecs**: native JSON serialization/deserialization for special f64 values (vectordotdev#18650) [39b9298](answerbook/vector@39b9298) - GitHub * **config**: Only try default paths if the path is not specified (vectordotdev#18681) [37fc9db](answerbook/vector@37fc9db) - GitHub * **datadog_agent source, datadog_metrics sink**: handle interval for non-rate series metrics (vectordotdev#18889) [dc9d966](answerbook/vector@dc9d966) - GitHub * **datadog_agent source**: return 200 on empty object payload (vectordotdev#19093) [ab7983a](answerbook/vector@ab7983a) - Jesse Szwedko * **datadog_metrics sink**: evaluate series v1 env var at runtime (vectordotdev#19148) [7b292ce](answerbook/vector@7b292ce) - Jesse Szwedko * **datadog_metrics sink**: improve aggregation performance (vectordotdev#18759) [6a40169](answerbook/vector@6a40169) - GitHub * **datadog_metrics sink**: Revert to using v1 endpoint by default (vectordotdev#19138) [3158f46](answerbook/vector@3158f46) - Jesse Szwedko * **datadog_metrics sink**: the integration tests weren't actually validating anything (vectordotdev#18754) [afc166f](answerbook/vector@afc166f) - GitHub * **datadog_traces sink**: improve request size limiting (vectordotdev#18903) [a477d72](answerbook/vector@a477d72) - GitHub * **debian platform**: Re-add `conf-files` directive for `cargo-deb` (vectordotdev#18726) [e445721](answerbook/vector@e445721) - GitHub * **dev**: Gate config conversion tests (vectordotdev#18698) [c35ae64](answerbook/vector@c35ae64) - GitHub * **dev**: update environment for website development (vectordotdev#18657) [4327776](answerbook/vector@4327776) - GitHub * **dnstap source**: support DNSSEC RRSIG record data (vectordotdev#18878) [ed97f0d](answerbook/vector@ed97f0d) - GitHub * **docker source**: do not emit component error for out of order logs (vectordotdev#18649) [93d7af4](answerbook/vector@93d7af4) - GitHub * **http_server source**: panic when http server receives metric events (vectordotdev#18781) [b107ff7](answerbook/vector@b107ff7) - GitHub * **kafka sink**: Make KafkaService return `Poll::Pending` when producer queue is full (vectordotdev#18770) [a1863e6](answerbook/vector@a1863e6) - GitHub * **kafka source**: fix acknowledgement handling during shutdown and rebalance events (vectordotdev#17497) [f2efb1a](answerbook/vector@f2efb1a) - GitHub * **loki sink**: update to use the global list of compression algorithms (vectordotdev#19099) [218963a](answerbook/vector@218963a) - Jesse Szwedko * **releasing**: Update cargo-deb (vectordotdev#19009) [88194e7](answerbook/vector@88194e7) - GitHub * **releasing**: Update example YAML config data_dir (vectordotdev#18896) [0b27019](answerbook/vector@0b27019) - GitHub * remove gh token call (vectordotdev#19047) [1c864aa](answerbook/vector@1c864aa) - GitHub * **sources**: emit `ComponentEventsDropped` when source send is cancelled (vectordotdev#18859) [a0e2769](answerbook/vector@a0e2769) - GitHub * **tls**: for incoming connection alpn negotiation should be done using set_alpn_select_callback (vectordotdev#18843) [8a5b67e](answerbook/vector@8a5b67e) - GitHub ### Chores * Add SHA256 checksums file to GH releases (vectordotdev#18701) [18f07a0](answerbook/vector@18f07a0) - GitHub * **ci**: Add a summary if the regression workflow is skipped (vectordotdev#18724) [17bd2b1](answerbook/vector@17bd2b1) - GitHub * **ci**: Add a test to assert conf files aren't overwritten (vectordotdev#18728) [3ade682](answerbook/vector@3ade682) - GitHub * **ci**: add dependabot group for futures (vectordotdev#18954) [c23efce](answerbook/vector@c23efce) - GitHub * **ci**: Bump aws-actions/amazon-ecr-login from 1 to 2 (vectordotdev#18752) [8e2032c](answerbook/vector@8e2032c) - GitHub * **ci**: Bump aws-actions/configure-aws-credentials from 4.0.0 to 4.0.1 (vectordotdev#18771) [c9804f0](answerbook/vector@c9804f0) - GitHub * **ci**: Bump bufbuild/buf-setup-action from 1.26.1 to 1.27.0 (vectordotdev#18783) [a784018](answerbook/vector@a784018) - GitHub * **ci**: Bump bufbuild/buf-setup-action from 1.27.0 to 1.27.1 (vectordotdev#18866) [811b7f7](answerbook/vector@811b7f7) - GitHub * **ci**: Bump bufbuild/buf-setup-action from 1.27.1 to 1.27.2 (vectordotdev#18981) [72560b1](answerbook/vector@72560b1) - GitHub * **ci**: Bump check-spelling/check-spelling from 0.0.21 to 0.0.22 (vectordotdev#18723) [f98cd5d](answerbook/vector@f98cd5d) - GitHub * **ci**: Bump MacOS unit test runners to 13 (vectordotdev#18823) [5bf18df](answerbook/vector@5bf18df) - GitHub * **ci**: Bump tspascoal/get-user-teams-membership from 2 to 3 (vectordotdev#18808) [4701bb9](answerbook/vector@4701bb9) - GitHub * **ci**: filter team members from gardener issue comment workflow (vectordotdev#18915) [bf56ac5](answerbook/vector@bf56ac5) - GitHub * **ci**: Fix cookie banner style issues (vectordotdev#18745) [7a55e54](answerbook/vector@7a55e54) - GitHub * **ci**: Remove unusued Dockerfile (vectordotdev#18824) [9d1a676](answerbook/vector@9d1a676) - GitHub * **ci**: Revet bump check-spelling/check-spelling from 0.0.21 to 0.0.22 (vectordotdev#18742) [7dce292](answerbook/vector@7dce292) - GitHub * **ci**: Run deny check nightly instead of on every PR (vectordotdev#18799) [ae117dc](answerbook/vector@ae117dc) - GitHub * **ci**: temporarily peg greptimedb to `v0.4.0` to unblock CI (vectordotdev#18838) [0776cc0](answerbook/vector@0776cc0) - GitHub * convert test config to yaml (vectordotdev#18856) [8b00214](answerbook/vector@8b00214) - GitHub * **core**: Add a CLI flag to allow for empty configs (vectordotdev#19021) [df4921b](answerbook/vector@df4921b) - GitHub * **core**: add more event metadata to proto (vectordotdev#18816) [2deeba1](answerbook/vector@2deeba1) - GitHub * **core**: Refactor `vector-core::stream` into its own package (vectordotdev#18900) [96f4d73](answerbook/vector@96f4d73) - GitHub * **core**: Set up internal topology API (vectordotdev#18919) [c9c184e](answerbook/vector@c9c184e) - GitHub * **datadog_metrics sink**: Set partial Origin Metrics in edge cases (vectordotdev#18677) [3dab239](answerbook/vector@3dab239) - GitHub * **datadog_metrics sink**: support and migrate to the `v2` series API endpoint (vectordotdev#18761) [3485f2c](answerbook/vector@3485f2c) - GitHub * **datadog**: remove deprecated config options (vectordotdev#18940) [f42751d](answerbook/vector@f42751d) - GitHub * **deps**: Add more `dependabot` groups (vectordotdev#18719) [d0e605e](answerbook/vector@d0e605e) - GitHub * **deps**: Bump @babel/traverse from 7.17.0 to 7.23.2 in /website (vectordotdev#18852) [85d2f17](answerbook/vector@85d2f17) - GitHub * **deps**: Bump apache-avro from 0.15.0 to 0.16.0 (vectordotdev#18685) [b37ce3c](answerbook/vector@b37ce3c) - GitHub * **deps**: Bump async-compression from 0.4.3 to 0.4.4 (vectordotdev#18848) [1a8a8cc](answerbook/vector@1a8a8cc) - GitHub * **deps**: Bump async-graphql from 5.0.10 to 6.0.9 (vectordotdev#18988) [4a4eb61](answerbook/vector@4a4eb61) - GitHub * **deps**: Bump async-nats from 0.32.0 to 0.32.1 (vectordotdev#18735) [047c772](answerbook/vector@047c772) - GitHub * **deps**: Bump async-trait from 0.1.73 to 0.1.74 (vectordotdev#18849) [8d82257](answerbook/vector@8d82257) - GitHub * **deps**: Bump base64 from 0.21.4 to 0.21.5 (vectordotdev#18907) [e754dee](answerbook/vector@e754dee) - GitHub * **deps**: Bump bitmask-enum from 2.2.2 to 2.2.3 (vectordotdev#19057) [7a16ee2](answerbook/vector@7a16ee2) - GitHub * **deps**: Bump bstr from 1.6.2 to 1.7.0 (vectordotdev#18810) [91221c6](answerbook/vector@91221c6) - GitHub * **deps**: Bump cached from 0.45.1 to 0.46.0 (vectordotdev#18660) [331c5a0](answerbook/vector@331c5a0) - GitHub * **deps**: Bump cached from 0.46.0 to 0.46.1 (vectordotdev#19058) [51c6b57](answerbook/vector@51c6b57) - GitHub * **deps**: Bump cargo_toml from 0.16.3 to 0.17.0 (vectordotdev#18978) [b55e436](answerbook/vector@b55e436) - GitHub * **deps**: Bump chrono from 0.4.30 to 0.4.31 (vectordotdev#18583) [052ed98](answerbook/vector@052ed98) - GitHub * **deps**: Bump chrono-tz from 0.8.3 to 0.8.4 (vectordotdev#18979) [7f44b4c](answerbook/vector@7f44b4c) - GitHub * **deps**: Bump clap from 4.4.5 to 4.4.6 (vectordotdev#18715) [4d98fdf](answerbook/vector@4d98fdf) - GitHub * **deps**: Bump clap_complete from 4.4.2 to 4.4.3 (vectordotdev#18716) [27b2c93](answerbook/vector@27b2c93) - GitHub * **deps**: Bump console-subscriber from 0.1.10 to 0.2.0 (vectordotdev#18732) [eda0378](answerbook/vector@eda0378) - GitHub * **deps**: Bump csv from 1.2.2 to 1.3.0 (vectordotdev#18768) [7cb8b52](answerbook/vector@7cb8b52) - GitHub * **deps**: Bump dd-rust-license-tool to 1.0.2 (vectordotdev#18711) [570bd52](answerbook/vector@570bd52) - GitHub * **deps**: Bump dyn-clone from 1.0.14 to 1.0.16 (vectordotdev#19040) [8ba28e0](answerbook/vector@8ba28e0) - GitHub * **deps**: Bump fakedata_generator from 0.2.4 to 0.4.0 (vectordotdev#18910) [7debc60](answerbook/vector@7debc60) - GitHub * **deps**: Bump flate2 from 1.0.27 to 1.0.28 (vectordotdev#18850) [b3889bc](answerbook/vector@b3889bc) - GitHub * **deps**: Bump futures-util from 0.3.28 to 0.3.29 (vectordotdev#18951) [741aec3](answerbook/vector@741aec3) - GitHub * **deps**: Bump goauth from 0.13.1 to 0.14.0 (vectordotdev#18872) [1913ee5](answerbook/vector@1913ee5) - GitHub * **deps**: Bump hashbrown from 0.14.0 to 0.14.1 (vectordotdev#18731) [31d92c2](answerbook/vector@31d92c2) - GitHub * **deps**: Bump hashbrown from 0.14.1 to 0.14.2 (vectordotdev#18893) [aebe8db](answerbook/vector@aebe8db) - GitHub * **deps**: Bump indexmap from 2.0.0 to 2.0.1 (vectordotdev#18705) [65643da](answerbook/vector@65643da) - GitHub * **deps**: Bump indexmap from 2.0.1 to 2.0.2 (vectordotdev#18737) [f47df40](answerbook/vector@f47df40) - GitHub * **deps**: Bump inventory from 0.3.12 to 0.3.13 (vectordotdev#19024) [72eacf5](answerbook/vector@72eacf5) - GitHub * **deps**: Bump lading to 0.19.1 (vectordotdev#18869) [2d7c1bb](answerbook/vector@2d7c1bb) - GitHub * **deps**: Bump libc from 0.2.148 to 0.2.149 (vectordotdev#18800) [4002ef0](answerbook/vector@4002ef0) - GitHub * **deps**: Bump libc from 0.2.149 to 0.2.150 (vectordotdev#19059) [611a652](answerbook/vector@611a652) - GitHub * **deps**: Bump lru from 0.11.1 to 0.12.0 (vectordotdev#18767) [c531a3b](answerbook/vector@c531a3b) - GitHub * **deps**: Bump memchr from 2.6.3 to 2.6.4 (vectordotdev#18736) [c7482d0](answerbook/vector@c7482d0) - GitHub * **deps**: Bump memmap2 from 0.7.1 to 0.8.0 (vectordotdev#18659) [ca9e5b4](answerbook/vector@ca9e5b4) - GitHub * **deps**: Bump memmap2 from 0.8.0 to 0.9.0 (vectordotdev#18765) [dcbbb9b](answerbook/vector@dcbbb9b) - GitHub * **deps**: Bump mongodb from 2.6.1 to 2.7.0 (vectordotdev#18703) [539a40f](answerbook/vector@539a40f) - GitHub * **deps**: Bump mongodb from 2.7.0 to 2.7.1 (vectordotdev#19023) [602f630](answerbook/vector@602f630) - GitHub * **deps**: Bump num_enum from 0.7.0 to 0.7.1 (vectordotdev#18975) [c9b6d45](answerbook/vector@c9b6d45) - GitHub * **deps**: Bump num-traits from 0.2.16 to 0.2.17 (vectordotdev#18802) [c4fbc25](answerbook/vector@c4fbc25) - GitHub * **deps**: Bump OpenDAL to v0.41 (vectordotdev#19039) [5655f76](answerbook/vector@5655f76) - GitHub * **deps**: Bump openssl from 0.10.57 to 0.10.58 (vectordotdev#19025) [c4f2d0e](answerbook/vector@c4f2d0e) - GitHub * **deps**: Bump openssl from 0.10.58 to 0.10.59 (vectordotdev#19054) [ee232f8](answerbook/vector@ee232f8) - GitHub * **deps**: Bump openssl-src from 300.1.5+3.1.3 to 300.1.6+3.1.4 (vectordotdev#18936) [a75a043](answerbook/vector@a75a043) - GitHub * **deps**: Bump ordered-float from 4.1.0 to 4.1.1 (vectordotdev#18818) [99643ca](answerbook/vector@99643ca) - GitHub * **deps**: Bump postcss from 8.4.6 to 8.4.31 in /website (vectordotdev#18750) [92d2be9](answerbook/vector@92d2be9) - GitHub * **deps**: Bump proc-macro2 from 1.0.67 to 1.0.69 (vectordotdev#18803) [bc3b3a2](answerbook/vector@bc3b3a2) - GitHub * **deps**: Bump proptest from 1.2.0 to 1.3.1 (vectordotdev#18738) [2b15c63](answerbook/vector@2b15c63) - GitHub * **deps**: Bump pulsar from 6.0.1 to 6.1.0 (vectordotdev#19004) [43f5913](answerbook/vector@43f5913) - GitHub * **deps**: Bump quanta from 0.11.1 to 0.12.0 (vectordotdev#18774) [e9d2dae](answerbook/vector@e9d2dae) - GitHub * **deps**: Bump quanta from 0.12.0 to 0.12.1 (vectordotdev#19005) [051de5a](answerbook/vector@051de5a) - GitHub * **deps**: Bump ratatui from 0.23.0 to 0.24.0 (vectordotdev#18908) [8b56a93](answerbook/vector@8b56a93) - GitHub * **deps**: Bump regex from 1.10.0 to 1.10.2 (vectordotdev#18858) [434849d](answerbook/vector@434849d) - GitHub * **deps**: Bump regex from 1.9.5 to 1.9.6 (vectordotdev#18739) [8e98321](answerbook/vector@8e98321) - GitHub * **deps**: Bump regex from 1.9.6 to 1.10.0 (vectordotdev#18812) [4d02abf](answerbook/vector@4d02abf) - GitHub * **deps**: Bump reqwest from 0.11.20 to 0.11.22 (vectordotdev#18760) [bac60ad](answerbook/vector@bac60ad) - GitHub * **deps**: Bump rustix from 0.37.19 to 0.37.25 (vectordotdev#18879) [3ca32b8](answerbook/vector@3ca32b8) - GitHub * **deps**: Bump semver from 1.0.18 to 1.0.19 (vectordotdev#18662) [aca7753](answerbook/vector@aca7753) - GitHub * **deps**: Bump semver from 1.0.19 to 1.0.20 (vectordotdev#18811) [d9aca80](answerbook/vector@d9aca80) - GitHub * **deps**: Bump serde from 1.0.188 to 1.0.189 (vectordotdev#18834) [0d09898](answerbook/vector@0d09898) - GitHub * **deps**: Bump serde from 1.0.189 to 1.0.190 (vectordotdev#18945) [73afddb](answerbook/vector@73afddb) - GitHub * **deps**: Bump serde_with from 3.3.0 to 3.4.0 (vectordotdev#18874) [33243ac](answerbook/vector@33243ac) - GitHub * **deps**: Bump serde_yaml from 0.9.25 to 0.9.27 (vectordotdev#18956) [2ee96b1](answerbook/vector@2ee96b1) - GitHub * **deps**: Bump serde-toml-merge from 0.3.2 to 0.3.3 (vectordotdev#18804) [abb9101](answerbook/vector@abb9101) - GitHub * **deps**: Bump serde-wasm-bindgen from 0.6.0 to 0.6.1 (vectordotdev#18935) [ffed6f7](answerbook/vector@ffed6f7) - GitHub * **deps**: Bump sha2 from 0.10.7 to 0.10.8 (vectordotdev#18684) [87af0bd](answerbook/vector@87af0bd) - GitHub * **deps**: Bump socket2 from 0.5.4 to 0.5.5 (vectordotdev#18902) [691fdca](answerbook/vector@691fdca) - GitHub * **deps**: Bump syn from 2.0.37 to 2.0.38 (vectordotdev#18789) [ec5238e](answerbook/vector@ec5238e) - GitHub * **deps**: Bump syn from 2.0.38 to 2.0.39 (vectordotdev#19056) [0f0a0b4](answerbook/vector@0f0a0b4) - GitHub * **deps**: Bump tempfile from 3.6.0 to 3.8.0 (vectordotdev#18686) [c0d24b9](answerbook/vector@c0d24b9) - GitHub * **deps**: Bump tempfile from 3.8.0 to 3.8.1 (vectordotdev#18957) [40961ed](answerbook/vector@40961ed) - GitHub * **deps**: Bump the azure group with 4 updates (vectordotdev#18773) [0e61acc](answerbook/vector@0e61acc) - GitHub * **deps**: Bump the azure group with 4 updates (vectordotdev#19052) [53ffbc3](answerbook/vector@53ffbc3) - GitHub * **deps**: Bump the clap group with 1 update (vectordotdev#18906) [cddb835](answerbook/vector@cddb835) - GitHub * **deps**: Bump the clap group with 2 updates (vectordotdev#18925) [78934c2](answerbook/vector@78934c2) - GitHub * **deps**: Bump the futures group with 1 update (vectordotdev#18961) [0a5e3db](answerbook/vector@0a5e3db) - GitHub * **deps**: Bump the tonic group with 2 updates (vectordotdev#18714) [c95df7c](answerbook/vector@c95df7c) - GitHub * **deps**: Bump the zstd group with 1 update (vectordotdev#18826) [96ef9ee](answerbook/vector@96ef9ee) - GitHub * **deps**: Bump thiserror from 1.0.48 to 1.0.49 (vectordotdev#18683) [6b92a83](answerbook/vector@6b92a83) - GitHub * **deps**: Bump thiserror from 1.0.49 to 1.0.50 (vectordotdev#18892) [16df7ea](answerbook/vector@16df7ea) - GitHub * **deps**: Bump tokio from 1.32.0 to 1.33.0 (vectordotdev#18809) [76971bd](answerbook/vector@76971bd) - GitHub * **deps**: Bump tokio-tungstenite from 0.20.0 to 0.20.1 (vectordotdev#18661) [8abed12](answerbook/vector@8abed12) - GitHub * **deps**: Bump toml from 0.8.0 to 0.8.1 (vectordotdev#18687) [efbe673](answerbook/vector@efbe673) - GitHub * **deps**: Bump toml from 0.8.1 to 0.8.2 (vectordotdev#18747) [f9e51e1](answerbook/vector@f9e51e1) - GitHub * **deps**: Bump toml from 0.8.2 to 0.8.3 (vectordotdev#18909) [22402ca](answerbook/vector@22402ca) - GitHub * **deps**: Bump toml from 0.8.3 to 0.8.4 (vectordotdev#18913) [249330a](answerbook/vector@249330a) - GitHub * **deps**: Bump toml from 0.8.4 to 0.8.5 (vectordotdev#18950) [4a525a8](answerbook/vector@4a525a8) - GitHub * **deps**: Bump toml from 0.8.5 to 0.8.6 (vectordotdev#18962) [5e7ae83](answerbook/vector@5e7ae83) - GitHub * **deps**: Bump tracing-log from 0.1.3 to 0.1.4 (vectordotdev#18914) [e4fd78c](answerbook/vector@e4fd78c) - GitHub [log-0](https://logdna.atlassian.net/browse/log-0) [log-0](https://logdna.atlassian.net/browse/log-0) * **deps**: Bump tracing-log from 0.1.4 to 0.2.0 (vectordotdev#18941) [30a1e26](answerbook/vector@30a1e26) - GitHub [log-0](https://logdna.atlassian.net/browse/log-0) [log-0](https://logdna.atlassian.net/browse/log-0) * **deps**: Bump trust-dns-proto from 0.23.0 to 0.23.1 (vectordotdev#18846) [0568d7a](answerbook/vector@0568d7a) - GitHub * **deps**: Bump trust-dns-proto from 0.23.1 to 0.23.2 (vectordotdev#18911) [1eaf8b1](answerbook/vector@1eaf8b1) - GitHub * **deps**: Bump uuid from 1.4.1 to 1.5.0 (vectordotdev#18880) [a025caa](answerbook/vector@a025caa) - GitHub * **deps**: Bump warp from 0.3.5 to 0.3.6 (vectordotdev#18704) [decaaeb](answerbook/vector@decaaeb) - GitHub * **deps**: Bump wasm-bindgen from 0.2.87 to 0.2.88 (vectordotdev#19026) [63bb9e4](answerbook/vector@63bb9e4) - GitHub * **deps**: Bump webpki from 0.22.1 to 0.22.2 (vectordotdev#18744) [4295985](answerbook/vector@4295985) - GitHub * **deps**: Bump wiremock from 0.5.19 to 0.5.21 (vectordotdev#19055) [4622ef6](answerbook/vector@4622ef6) - GitHub * **deps**: Bump zerocopy from 0.7.21 to 0.7.31 (vectordotdev#19394) [74d6cb1](answerbook/vector@74d6cb1) - Jesse Szwedko * **deps**: clean up VRL crate features (vectordotdev#18740) [1452d54](answerbook/vector@1452d54) - GitHub * **deps**: Group csv crate updates (vectordotdev#18797) [98aa157](answerbook/vector@98aa157) - GitHub * **deps**: Remove usages of atty (vectordotdev#18985) [371580c](answerbook/vector@371580c) - GitHub * **deps**: Update dependencies (vectordotdev#18971) [3ead10f](answerbook/vector@3ead10f) - GitHub * **deps**: Update lading to 0.19.0 (vectordotdev#18861) [dc729f5](answerbook/vector@dc729f5) - GitHub * **deps**: Update license-tool.toml webpki version (vectordotdev#18986) [0051ec0](answerbook/vector@0051ec0) - GitHub * **deps**: Update VRL to 0.8.1 (vectordotdev#19011) [f5ea285](answerbook/vector@f5ea285) - GitHub * **dev**: Add `vector-lib` wrapper for three more libs (vectordotdev#18992) [1eb418b](answerbook/vector@1eb418b) - GitHub * **dev**: Add wrapper for `codecs` to `vector-lib` (vectordotdev#18959) [5f30f74](answerbook/vector@5f30f74) - GitHub * **dev**: Add wrapper for `enrichment` to `vector-lib` (vectordotdev#18977) [e61f308](answerbook/vector@e61f308) - GitHub * **dev**: Add wrapper for `file-source` in `vector-lib` (vectordotdev#18984) [f44da16](answerbook/vector@f44da16) - GitHub * **dev**: Add wrapper for `lookup` in `vector-lib` (vectordotdev#18995) [164f1e9](answerbook/vector@164f1e9) - GitHub * **dev**: Add wrapper for `vector-buffers` to `vector-lib` (vectordotdev#18964) [2cef62c](answerbook/vector@2cef62c) - GitHub * **dev**: Detail the format of DEPRECATIONS.md file (vectordotdev#19016) [223dd7b](answerbook/vector@223dd7b) - GitHub * **dev**: Move some macros into `vector-core` (vectordotdev#19002) [4f613ce](answerbook/vector@4f613ce) - GitHub * **dev**: Remove deprecation action item for armv7 RPMs (vectordotdev#19018) [ff7b95f](answerbook/vector@ff7b95f) - GitHub * **dev**: Set up `vector-lib` wrapper crate with `vector-common` (vectordotdev#18927) [00c40d7](answerbook/vector@00c40d7) - GitHub * **dev**: Wrap `vector-config` in `vector-lib` as `configurable` (vectordotdev#18944) [42beb3f](answerbook/vector@42beb3f) - GitHub * **dev**: Wrap `vector-core` in `vector-lib` (vectordotdev#18934) [270fdfd](answerbook/vector@270fdfd) - GitHub * **dev**: Wrap `vector-stream` in `vector-lib` (vectordotdev#18953) [8a02b16](answerbook/vector@8a02b16) - GitHub * **docs**: Add alpha to traces and beta to metrics in descriptions (vectordotdev#19139) [1b9fb9b](answerbook/vector@1b9fb9b) - Jesse Szwedko * **docs**: add highlight post for secrets in disk buffers (vectordotdev#18994) [0cc9389](answerbook/vector@0cc9389) - GitHub * **docs**: Add spec for `listen` option (vectordotdev#18080) [29e5e22](answerbook/vector@29e5e22) - GitHub * **docs**: Replace setup.vector.dev references (vectordotdev#19080) [def235e](answerbook/vector@def235e) - Jesse Szwedko * **docs**: update a few more examples to YAML (vectordotdev#19103) [a59329a](answerbook/vector@a59329a) - Jesse Szwedko * **docs**: update quickstart.md to use YAML (vectordotdev#18796) [0e76fe0](answerbook/vector@0e76fe0) - GitHub * **external docs**: First batch of editorial edits for the Functions doc (vectordotdev#18780) [1da9005](answerbook/vector@1da9005) - GitHub * **external docs**: Remove or replace mentions of vector in functions doc (vectordotdev#18679) [7ad4112](answerbook/vector@7ad4112) - GitHub * Follow redirects for `sh.vector.dev` (vectordotdev#19000) [9893b86](answerbook/vector@9893b86) - GitHub * **gcp_stackdriver_metrics sink**: rewrite to stream based sink (vectordotdev#18749) [92268e4](answerbook/vector@92268e4) - GitHub * **kubernetes**: Regenerate manifests from 0.27.0 chart (vectordotdev#19001) [5e9dd1d](answerbook/vector@5e9dd1d) - GitHub * **metrics**: improve creation of Origin metadata structures (vectordotdev#18788) [f0adce7](answerbook/vector@f0adce7) - GitHub * Note the version to remove the v1 metrics support from the Datadog Metrics sink (vectordotdev#19017) [be9f229](answerbook/vector@be9f229) - GitHub * **observability, blackhole sink**: Don't report by default (vectordotdev#18963) [3b85b48](answerbook/vector@3b85b48) - GitHub * **observability**: deprecate obsolete http metrics (vectordotdev#18972) [f33dce2](answerbook/vector@f33dce2) - GitHub * **observability**: fix tokio unstable (vectordotdev#18776) [67c4beb](answerbook/vector@67c4beb) - GitHub * **observability**: remove `peer_addr` internal metric tag (vectordotdev#18982) [b9447f6](answerbook/vector@b9447f6) - GitHub * **observability**: remove deprecated `component_name` metric tag (vectordotdev#18942) [c6f5d2b](answerbook/vector@c6f5d2b) - GitHub * **observability**: remove metrics replaced by component_errors_total (vectordotdev#18965) [17f4ed2](answerbook/vector@17f4ed2) - GitHub * **prometheus_remote_write sink**: remote write sink rewrite (vectordotdev#18676) [5c1707f](answerbook/vector@5c1707f) - GitHub * **regression**: Unmark regression tests as erratic now (vectordotdev#19020) [2cdf654](answerbook/vector@2cdf654) - GitHub * **releasing, kubernetes**: Update manifests to v0.26.0 of the chart (vectordotdev#18694) [5f4c3ba](answerbook/vector@5f4c3ba) - GitHub * **releasing**: Add deprecation note about respositories.timber.io deprecation (vectordotdev#19078) [09df599](answerbook/vector@09df599) - Jesse Szwedko * **releasing**: Add known issue for 0.33.0 debian packaging regression (vectordotdev#18727) [ff745ab](answerbook/vector@ff745ab) - GitHub * **releasing**: Add known issue for Datadog Metrics sink in v0.34.0 (vectordotdev#19122) [cee9d07](answerbook/vector@cee9d07) - Jesse Szwedko * **releasing**: Add known issue for protobuf encoder in v0.34.0 (vectordotdev#19244) [f7c3824](answerbook/vector@f7c3824) - Jesse Szwedko * **releasing**: Add upgrade note about TOML breaking change to v0.34.0 (vectordotdev#19120) [dba0ba1](answerbook/vector@dba0ba1) - Jesse Szwedko * **releasing**: Bump Vector version to v0.34.0 (vectordotdev#18693) [1c2f970](answerbook/vector@1c2f970) - GitHub * **releasing**: Fix changelog not for kafka fix in 0.33.1 (vectordotdev#19032) [2501049](answerbook/vector@2501049) - GitHub * **releasing**: Fix formatting for v0.34.0 release note (vectordotdev#19085) [3569271](answerbook/vector@3569271) - Jesse Szwedko * **releasing**: Prepare v0.33.0 release [682f0e0](answerbook/vector@682f0e0) - Jesse Szwedko * **releasing**: Prepare v0.33.1 release [409b69d](answerbook/vector@409b69d) - Jesse Szwedko * **releasing**: Prepare v0.34.0 release [c909b66](answerbook/vector@c909b66) - Jesse Szwedko * **releasing**: Prepare v0.34.1 release [86f1c22](answerbook/vector@86f1c22) - Jesse Szwedko * **releasing**: Prepare v0.34.2 release [d685a16](answerbook/vector@d685a16) - Jesse Szwedko * **releasing**: Typo in v0.33.1 release docs (vectordotdev#18987) [36974a0](answerbook/vector@36974a0) - GitHub * Remove @spencergilbert from CODEOWNERS (vectordotdev#18778) [f300c85](answerbook/vector@f300c85) - GitHub * Remove armv7 RPM package (vectordotdev#18837) [eca8c76](answerbook/vector@eca8c76) - GitHub * remove config/vector.toml (vectordotdev#18833) [efb0d1a](answerbook/vector@efb0d1a) - GitHub * **security**: Ignore RUSTSEC-2023-0071 for now (vectordotdev#19263) [e27b7bd](answerbook/vector@e27b7bd) - Jesse Szwedko * **security**: Remove legacy OpenSSL provider flags (vectordotdev#19015) [2bba40a](answerbook/vector@2bba40a) - GitHub * **sinks**: Update `PartitionBatcher` to use `BatchConfig` (vectordotdev#18792) [4a7d0c3](answerbook/vector@4a7d0c3) - GitHub * Update *-release.md issue templates for vector.dev package release (vectordotdev#18814) [ab8f8d2](answerbook/vector@ab8f8d2) - GitHub * **vrl**: Revive old remap tests (vectordotdev#18678) [53cad38](answerbook/vector@53cad38) - GitHub * **website**: Set download page dropdown to latest version (vectordotdev#18758) [23745f2](answerbook/vector@23745f2) - GitHub * **websites**: Setup preview site workflows (vectordotdev#18924) [9d006c7](answerbook/vector@9d006c7) - GitHub * **websites**: Workflow updates (vectordotdev#19036) [b4ca866](answerbook/vector@b4ca866) - GitHub * **website**: WEB-4247 | Update references from s3 to setup.vector.dev (vectordotdev#19149) [9e1ad37](answerbook/vector@9e1ad37) - Jesse Szwedko * **website**: WEB-4275 | Update Navigation (vectordotdev#19186) [73a668c](answerbook/vector@73a668c) - Doug Smith * **website**: Workflow fixes (vectordotdev#19046) [fb63f8e](answerbook/vector@fb63f8e) - GitHub ### Continuous Integration * mark otlp_http_to_blackhole experiment erratic (vectordotdev#18786) [54b54a5](answerbook/vector@54b54a5) - GitHub ### Features * add PR comment trigger for the workload checks workflow (vectordotdev#18839) [11bc5d9](answerbook/vector@11bc5d9) - GitHub * **amqp**: added integration test for TLS (vectordotdev#18813) [3c4ae86](answerbook/vector@3c4ae86) - GitHub [LOG-16435](https://logdna.atlassian.net/browse/LOG-16435) * **ci**: Add Vector workload checks (vectordotdev#18569) [e2b7de0](answerbook/vector@e2b7de0) - GitHub * **codecs**: add support for protobuf encoding (vectordotdev#18598) [737f5c3](answerbook/vector@737f5c3) - GitHub * **docs**: add fallibility examples (vectordotdev#18931) [08b45a5](answerbook/vector@08b45a5) - GitHub * **journald source**: Add emit_cursor option (vectordotdev#18882) [74051dc](answerbook/vector@74051dc) - GitHub * **regression**: convert all regression cases configs to YAML (vectordotdev#18825) [1fb0f0d](answerbook/vector@1fb0f0d) - GitHub * **timestamp encoding**: add unixtime formats (vectordotdev#18817) [53039e7](answerbook/vector@53039e7) - GitHub ### Miscellaneous * Merge pull request vectordotdev#396 from answerbook/feature/LOG-18978 [2b046bf](answerbook/vector@2b046bf) - GitHub [LOG-18978](https://logdna.atlassian.net/browse/LOG-18978) * Merge tag 'v0.34.2' into feature/LOG-18978 [a88d5ed](answerbook/vector@a88d5ed) - Darin Spivey [LOG-18978](https://logdna.atlassian.net/browse/LOG-18978) [LOG-18978](https://logdna.atlassian.net/browse/LOG-18978) * Update RUM domain (vectordotdev#19367) [4acb9de](answerbook/vector@4acb9de) - Jesse Szwedko * chore(docs):Add Obs Pipelines to docs (vectordotdev#19201) [8f3f160](answerbook/vector@8f3f160) - Jesse Szwedko * updating the doc, 2 urls were 404 (vectordotdev#18949) [bf58b06](answerbook/vector@bf58b06) - GitHub * chore!(config, docs): delete deprecated vector.toml code (vectordotdev#18795) [c8557d0](answerbook/vector@c8557d0) - GitHub * [WEB-3464] Adds TrustArc cookie consent banner (vectordotdev#18741) [43428d3](answerbook/vector@43428d3) - GitHub * Add announcement for new repository URLs (vectordotdev#18798) [239cf94](answerbook/vector@239cf94) - GitHub * **examples**: Convert config/examples from TOML to YAML (vectordotdev#18832) [6ffb072](answerbook/vector@6ffb072) - GitHub * **external docs**: Fix metrics test example (vectordotdev#18725) [96def01](answerbook/vector@96def01) - GitHub * fix truncate arguments (vectordotdev#19068) [9a5cb51](answerbook/vector@9a5cb51) - Jesse Szwedko * **gcp_pubsub source**: Add required fields to documentation examples (vectordotdev#18998) [21f741d](answerbook/vector@21f741d) - GitHub * **nats source**: add subscriber_capacity option (vectordotdev#18899) [e7b563d](answerbook/vector@e7b563d) - GitHub * **sources, sinks**: add telemetry to http and grpc servers (vectordotdev#18887) [e779019](answerbook/vector@e779019) - GitHub * **tls**: add new dedicated page for TLS configuration (vectordotdev#18844) [625e4bd](answerbook/vector@625e4bd) - GitHub * **vrl**: add an example of parsing upstreaminfo with parse_nginx_log (vectordotdev#18815) [774094e](answerbook/vector@774094e) - GitHub ### **BREAKING CHANGES** * **security:** Remove legacy OpenSSL provider flags (vectordotdev#19015) * **observability:** remove metrics replaced by component_errors_total (vectordotdev#18965) * **observability:** remove `peer_addr` internal metric tag (vectordotdev#18982) * **observability, blackhole sink:** Don't report by default (vectordotdev#18963) * **observability:** remove deprecated `component_name` metric tag (vectordotdev#18942) * **datadog:** remove deprecated config options (vectordotdev#18940)

…rdotdev#18676) * Refactor prometheus remote write sink Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com> * WIP Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com> * Made tests pass Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com> * Use shared compression Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com> * Clippy Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com> * Some comments Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com> * Use HttpResponse Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com> * Update request builder default Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com> * Update PartitionBatcher to use BatchConfig Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com> * Update PartitionBatcher to use BatchConfig Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com> * Remove zorkwonk Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com> * Make into fns as fns instead Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com> * Spelling Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com> * Don't box the closure Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com> * Clippy Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com> * Only insert timeout if we add the batch to the list Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com> * Allow the timer to remove an item Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com> * Allow partitions to be types other than Vec Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com> * Clippy Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com> * Added test for aggregation Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com> * Allow a custom object to be used for the reducer Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com> * Make aggregating optional Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com> * Adde test for non aggregation Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com> * Clippy Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com> * Component docs Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com> * Feedback from Kyle and Doug Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com> * Use generic compression options Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com> * Default compression to Snappy Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com> * Update docs Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com> * Add snappy to the compression docs Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com> * Remove proptest file Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com> * Snappy is no longer optional Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com> --------- Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>

StephenWakely added 6 commits August 17, 2023 17:00

Refactor prometheus remote write sink

33b7c17

Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>

Merge remote-tracking branch 'origin/master' into stephen/remote_writ…

1e83eec

…e_sink_rewrite

WIP

06a7ae3

Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>

Made tests pass

a3ef0f3

Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>

Use shared compression

31d7fb2

Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>

Clippy

4be6d00

Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>

StephenWakely requested a review from a team September 26, 2023 16:00

github-actions bot added domain: sources Anything related to the Vector's sources domain: sinks Anything related to the Vector's sinks labels Sep 26, 2023

StephenWakely added 4 commits September 26, 2023 17:08

Some comments

5970717

Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>

Use HttpResponse

844a2ee

Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>

Merge remote-tracking branch 'origin' into stephen/remote_write_sink_…

104642b

…rewrite

Update request builder default

e185761

Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>

neuronull approved these changes Sep 29, 2023

View reviewed changes

StephenWakely added 11 commits October 6, 2023 12:05

Update PartitionBatcher to use BatchConfig

ef48469

Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>

Update PartitionBatcher to use BatchConfig

b439288

Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>

Remove zorkwonk

cc81a68

Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>

Make into fns as fns instead

8302725

Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>

Spelling

e6b052d

Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>

Don't box the closure

179f351

Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>

Clippy

f8967b7

Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>

Only insert timeout if we add the batch to the list

896d76f

Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>

Allow the timer to remove an item

817bc46

Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>

Merge branch 'stephen/partition_batcher' into stephen/remote_write_si…

c664a1a

…nk_rewrite Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>

Allow partitions to be types other than Vec

eab7397

Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>

StephenWakely requested a review from dsmith3197 as a code owner October 11, 2023 16:21

StephenWakely requested a review from a team October 11, 2023 16:21

Default compression to Snappy

3f54b76

Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>

Update docs

17d46f2

Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>

dsmith3197 reviewed Oct 19, 2023

View reviewed changes

StephenWakely added 2 commits October 20, 2023 10:00

Add snappy to the compression docs

411eb34

Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>

Remove proptest file

7e51462

Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>

StephenWakely requested a review from dsmith3197 October 20, 2023 09:16

dsmith3197 approved these changes Oct 23, 2023

View reviewed changes

Merge remote-tracking branch 'origin' into stephen/remote_write_sink_…

47eefa5

…rewrite Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>

github-actions bot removed the domain: core Anything related to core crates i.e. vector-core, core-common, etc label Oct 24, 2023

StephenWakely enabled auto-merge October 24, 2023 10:31

StephenWakely added this pull request to the merge queue Oct 24, 2023

github-merge-queue bot removed this pull request from the merge queue due to failed status checks Oct 24, 2023

Snappy is no longer optional

e014b29

Signed-off-by: Stephen Wakely <fungus.humungus@gmail.com>

StephenWakely enabled auto-merge October 24, 2023 13:16

StephenWakely added this pull request to the merge queue Oct 24, 2023

github-merge-queue bot removed this pull request from the merge queue due to failed status checks Oct 24, 2023

StephenWakely added this pull request to the merge queue Oct 25, 2023

Merged via the queue into master with commit 5c1707f Oct 25, 2023
42 checks passed

StephenWakely deleted the stephen/remote_write_sink_rewrite branch October 25, 2023 10:39

StephenWakely mentioned this pull request Oct 25, 2023

Make metrics aggregation opt-in for batches where sink supports multiple timestamps #18917

Open

neuronull mentioned this pull request Oct 27, 2023

batch.max_bytes incompatible with Prometheus remote write sink #18960

Closed

This was referenced Nov 9, 2023

fix(loki sink): update to use the global list of compression algorithms #19099

Merged

fix(loki sink): update to use the global list of compression algorithms #19104

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

chore(prometheus_remote_write sink): remote write sink rewrite #18676

chore(prometheus_remote_write sink): remote write sink rewrite #18676

StephenWakely commented Sep 26, 2023 •

edited

Loading

netlify bot commented Sep 26, 2023 •

edited

Loading

netlify bot commented Sep 26, 2023 •

edited

Loading

datadog-vectordotdev bot commented Sep 27, 2023 •

edited

Loading

neuronull left a comment

neuronull Sep 27, 2023

datadog-vectordotdev bot commented Oct 19, 2023 •

edited

Loading

dsmith3197 Oct 19, 2023

StephenWakely Oct 19, 2023

github-actions bot commented Oct 24, 2023

github-actions bot commented Oct 24, 2023

github-actions bot commented Oct 25, 2023

chore(prometheus_remote_write sink): remote write sink rewrite #18676

chore(prometheus_remote_write sink): remote write sink rewrite #18676

Conversation

StephenWakely commented Sep 26, 2023 • edited Loading

netlify bot commented Sep 26, 2023 • edited Loading

✅ Deploy Preview for vrl-playground canceled.

netlify bot commented Sep 26, 2023 • edited Loading

✅ Deploy Preview for vector-project canceled.

datadog-vectordotdev bot commented Sep 27, 2023 • edited Loading

Datadog Report

neuronull left a comment

Choose a reason for hiding this comment

neuronull Sep 27, 2023

Choose a reason for hiding this comment

datadog-vectordotdev bot commented Oct 19, 2023 • edited Loading

Datadog Report

dsmith3197 Oct 19, 2023

Choose a reason for hiding this comment

StephenWakely Oct 19, 2023

Choose a reason for hiding this comment

github-actions bot commented Oct 24, 2023

Regression Detector Results

github-actions bot commented Oct 24, 2023

Regression Detector Results

github-actions bot commented Oct 25, 2023

Regression Detector Results

StephenWakely commented Sep 26, 2023 •

edited

Loading

netlify bot commented Sep 26, 2023 •

edited

Loading

netlify bot commented Sep 26, 2023 •

edited

Loading

datadog-vectordotdev bot commented Sep 27, 2023 •

edited

Loading

datadog-vectordotdev bot commented Oct 19, 2023 •

edited

Loading