Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(docs): editorial review of transforms and sources #16934

Merged

Conversation

maycmlee
Copy link
Contributor

Editorial review of transforms and sources descriptions to fit our style guide.

@maycmlee maycmlee requested a review from a team March 23, 2023 20:02
@maycmlee maycmlee requested a review from StephenWakely as a code owner March 23, 2023 20:02
@maycmlee maycmlee requested a review from a team March 23, 2023 20:02
@maycmlee maycmlee requested a review from a team March 23, 2023 20:02
@netlify
Copy link

netlify bot commented Mar 23, 2023

Deploy Preview for vrl-playground canceled.

Name Link
🔨 Latest commit 56e1e40
🔍 Latest deploy log https://app.netlify.com/sites/vrl-playground/deploys/6422f0d7cddb6000081721f8

@netlify
Copy link

netlify bot commented Mar 23, 2023

Deploy Preview for vector-project ready!

Name Link
🔨 Latest commit 56e1e40
🔍 Latest deploy log https://app.netlify.com/sites/vector-project/deploys/6422f0d70e12aa0008b152b0
😎 Deploy Preview https://deploy-preview-16934--vector-project.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site settings.

@github-actions github-actions bot added domain: core Anything related to core crates i.e. vector-core, core-common, etc domain: sinks Anything related to the Vector's sinks domain: sources Anything related to the Vector's sources domain: transforms Anything related to Vector's transform components labels Mar 23, 2023
src/sources/logstash.rs Outdated Show resolved Hide resolved
src/sources/socket/tcp.rs Outdated Show resolved Hide resolved
src/sources/socket/udp.rs Outdated Show resolved Hide resolved
src/sources/statsd/mod.rs Outdated Show resolved Hide resolved
src/sources/statsd/mod.rs Outdated Show resolved Hide resolved
@github-actions
Copy link

Regression Detector Results

Run ID: 84107f8b-4e0a-4887-a02e-8478e27e99c3
Baseline: 6f1d11e
Comparison: dd046e7
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine quickly if vector performance is changed and to what degree by a pull request.

The table below, if present, lists those experiments that have experienced a statistically significant change in mean optimization goal performance between baseline and comparison SHAs with 90.00% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±5.00% change in their mean optimization goal are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting change is observed.

No interesting changes in experiment optimization goals with confidence ≥ 90.00% and |Δ mean %| ≥ 5.00%.

Fine details of change detection per experiment.
experiment goal Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
socket_to_socket_blackhole ingress throughput 344.12KiB/CPU-s 2.59 100.00% 12.97MiB/CPU-s 500.83KiB/CPU-s 6.46KiB/CPU-s 0.0 0.037702 13.31MiB/CPU-s 337.62KiB/CPU-s 4.15KiB/CPU-s 0.0 0.024774 False False
datadog_agent_remap_datadog_logs ingress throughput 737.33KiB/CPU-s 2.22 100.00% 32.41MiB/CPU-s 1.16MiB/CPU-s 14.67KiB/CPU-s 0.0 0.035931 33.13MiB/CPU-s 866.97KiB/CPU-s 10.67KiB/CPU-s 0.0 0.025552 False False
otlp_http_to_blackhole ingress throughput 29.92KiB/CPU-s 1.92 100.00% 1.52MiB/CPU-s 124.56KiB/CPU-s 1.53KiB/CPU-s 0.0 0.07989 1.55MiB/CPU-s 112.17KiB/CPU-s 1.38KiB/CPU-s 0.0 0.070587 False False
datadog_agent_remap_datadog_logs_acks ingress throughput 379.31KiB/CPU-s 1.14 100.00% 32.46MiB/CPU-s 1.07MiB/CPU-s 13.53KiB/CPU-s 0.0 0.033087 32.83MiB/CPU-s 1.15MiB/CPU-s 14.46KiB/CPU-s 0.0 0.034971 False False
syslog_humio_logs ingress throughput 87.63KiB/CPU-s 0.97 100.00% 8.86MiB/CPU-s 337.23KiB/CPU-s 4.15KiB/CPU-s 0.0 0.037149 8.95MiB/CPU-s 223.2KiB/CPU-s 2.75KiB/CPU-s 0.0 0.024352 False False
syslog_regex_logs2metric_ddmetrics ingress throughput 34.13KiB/CPU-s 0.95 100.00% 3.51MiB/CPU-s 329.64KiB/CPU-s 4.06KiB/CPU-s 0.0 0.091597 3.55MiB/CPU-s 348.06KiB/CPU-s 4.28KiB/CPU-s 0.0 0.095804 False True
datadog_agent_remap_blackhole ingress throughput 305.77KiB/CPU-s 0.94 100.00% 31.69MiB/CPU-s 1.31MiB/CPU-s 16.54KiB/CPU-s 0.0 0.041437 31.98MiB/CPU-s 1.02MiB/CPU-s 12.9KiB/CPU-s 0.0 0.031999 False False
syslog_log2metric_humio_metrics ingress throughput 42.16KiB/CPU-s 0.65 100.00% 6.3MiB/CPU-s 114.17KiB/CPU-s 1.41KiB/CPU-s 0.0 0.017688 6.34MiB/CPU-s 109.63KiB/CPU-s 1.35KiB/CPU-s 0.0 0.016874 False False
splunk_hec_route_s3 ingress throughput 46.77KiB/CPU-s 0.39 100.00% 11.64MiB/CPU-s 561.63KiB/CPU-s 6.91KiB/CPU-s 0.0 0.047124 11.68MiB/CPU-s 570.95KiB/CPU-s 7.03KiB/CPU-s 0.0 0.047719 False False
http_to_http_acks ingress throughput 18.69KiB/CPU-s 0.35 30.03% 5.21MiB/CPU-s 2.69MiB/CPU-s 33.95KiB/CPU-s 0.0 0.517013 5.23MiB/CPU-s 2.74MiB/CPU-s 34.58KiB/CPU-s 0.0 0.524624 True False
enterprise_http_to_http ingress throughput 5.68KiB/CPU-s 0.04 83.09% 13.62MiB/CPU-s 277.51KiB/CPU-s 3.41KiB/CPU-s 0.0 0.019901 13.62MiB/CPU-s 189.2KiB/CPU-s 2.33KiB/CPU-s 0.0 0.013562 False False
syslog_splunk_hec_logs ingress throughput 3.28KiB/CPU-s 0.04 53.17% 8.89MiB/CPU-s 295.13KiB/CPU-s 3.63KiB/CPU-s 0.0 0.032426 8.89MiB/CPU-s 219.56KiB/CPU-s 2.7KiB/CPU-s 0.0 0.024114 False False
http_to_http_noack ingress throughput 1.6KiB/CPU-s 0.01 19.84% 13.61MiB/CPU-s 374.36KiB/CPU-s 4.6KiB/CPU-s 0.0 0.026864 13.61MiB/CPU-s 359.8KiB/CPU-s 4.43KiB/CPU-s 0.0 0.025816 False False
http_to_http_json ingress throughput 858.92B/CPU-s 0.01 16.71% 13.62MiB/CPU-s 229.58KiB/CPU-s 2.82KiB/CPU-s 0.0 0.01646 13.62MiB/CPU-s 227.55KiB/CPU-s 2.8KiB/CPU-s 0.0 0.016313 False False
fluent_elasticsearch ingress throughput 215.72B/CPU-s 0.00 31.42% 45.41MiB/CPU-s 30.06KiB/CPU-s 374.72B/CPU-s 0.0 0.000646 45.41MiB/CPU-s 30.43KiB/CPU-s 379.31B/CPU-s 0.0 0.000654 False False
splunk_hec_to_splunk_hec_logs_acks ingress throughput -124.45B/CPU-s -0.00 1.56% 13.61MiB/CPU-s 352.29KiB/CPU-s 4.33KiB/CPU-s 0.0 0.025269 13.61MiB/CPU-s 362.83KiB/CPU-s 4.46KiB/CPU-s 0.0 0.026025 False False
splunk_hec_indexer_ack_blackhole ingress throughput -2.6KiB/CPU-s -0.02 43.21% 13.62MiB/CPU-s 253.15KiB/CPU-s 3.11KiB/CPU-s 0.0 0.018154 13.61MiB/CPU-s 270.02KiB/CPU-s 3.32KiB/CPU-s 0.0 0.019368 False False
splunk_hec_to_splunk_hec_logs_noack ingress throughput -3.84KiB/CPU-s -0.03 60.75% 13.62MiB/CPU-s 246.88KiB/CPU-s 3.04KiB/CPU-s 0.0 0.017703 13.61MiB/CPU-s 268.95KiB/CPU-s 3.31KiB/CPU-s 0.0 0.019291 False False
datadog_agent_remap_blackhole_acks ingress throughput -125.5KiB/CPU-s -0.40 100.00% 31.02MiB/CPU-s 1.4MiB/CPU-s 17.59KiB/CPU-s 0.0 0.044992 30.9MiB/CPU-s 1.45MiB/CPU-s 18.33KiB/CPU-s 0.0 0.047055 False False
syslog_loki ingress throughput -101.1KiB/CPU-s -1.17 100.00% 8.41MiB/CPU-s 262.27KiB/CPU-s 3.23KiB/CPU-s 0.0 0.030452 8.31MiB/CPU-s 275.24KiB/CPU-s 3.39KiB/CPU-s 0.0 0.032337 False False
http_text_to_http_json ingress throughput -446.3KiB/CPU-s -1.79 100.00% 24.37MiB/CPU-s 718.81KiB/CPU-s 8.85KiB/CPU-s 0.0 0.028808 23.93MiB/CPU-s 524.01KiB/CPU-s 6.45KiB/CPU-s 0.0 0.021383 False False
otlp_grpc_to_blackhole ingress throughput -19.24KiB/CPU-s -1.83 100.00% 1.03MiB/CPU-s 52.2KiB/CPU-s 657.62B/CPU-s 0.0 0.049647 1.01MiB/CPU-s 52.44KiB/CPU-s 660.77B/CPU-s 0.0 0.050799 False False
syslog_log2metric_splunk_hec_metrics ingress throughput -185.88KiB/CPU-s -1.90 100.00% 9.53MiB/CPU-s 148.67KiB/CPU-s 1.83KiB/CPU-s 0.0 0.015227 9.35MiB/CPU-s 231.07KiB/CPU-s 2.84KiB/CPU-s 0.0 0.024127 False False
file_to_blackhole egress throughput -1.12MiB/CPU-s -15.76 100.00% 7.09MiB/CPU-s 4.24MiB/CPU-s 134.92KiB/CPU-s 6.576402 0.597445 5.97MiB/CPU-s 4.38MiB/CPU-s 117.22KiB/CPU-s 0.0 0.733546 True True

Copy link
Contributor

@spencergilbert spencergilbert left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry - things came up and I won't be able to finish reviewing this.

src/aws/auth.rs Outdated
@@ -114,7 +114,7 @@ pub enum AwsAuthentication {

/// Timeout for assuming the role, in seconds.
///
/// Relevant when the default credentials chain is used or `assume_role`.
/// Relevant when the default credentials chain or `assume_role` is used.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
/// Relevant when the default credentials chain or `assume_role` is used.
/// Relevant when the default credentials chain or `assume_role` is used.

@@ -117,7 +117,7 @@ pub struct StreamingConfig {
#[serde(default = "default_respawn_on_exit")]
respawn_on_exit: bool,

/// The amount of time, in seconds, that Vector will wait before rerunning a streaming command that exited.
/// The amount of time, in seconds, before rerunning a streaming command that exited.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
/// The amount of time, in seconds, before rerunning a streaming command that exited.
/// The amount of time before rerunning a streaming command that exited.

I'll open a PR to add this as an actual unit, which matches how we've been standardizing across the docs.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jszwedko do you mind getting someone to update the types and configurable annotations here, or open an issue to to so?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 we can leave this PR as-is for now and circle back to switch these to use types.

Copy link
Member

@jszwedko jszwedko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome, thanks @maycmlee !

What is the best way to make sure this copy and new copy stays up-to-date with our style guide? Should we tag you / your team for review on changes? Or just do our best and review it every once in a while (say once a quarter?)? Or something else?

@maycmlee
Copy link
Contributor Author

Awesome, thanks @maycmlee !

What is the best way to make sure this copy and new copy stays up-to-date with our style guide? Should we tag you / your team for review on changes? Or just do our best and review it every once in a while (say once a quarter?)? Or something else?

Thanks for reviewing @jszwedko! Yes, tagging the documentation team for review would be best! It'll show up in our on-call PR list and the person on call that day (business hours only) would review it. For for larger reviews, they'd create a docs card and then someone would be assigned to review it. You can also ping in #documentation if you need a quick turnaround.

I've made the changes mentioned in the comments, let me know if I've missed anything!

@github-actions
Copy link

Regression Detector Results

Run ID: ebca7705-f688-410e-b5f2-8aa392cd28dc
Baseline: 81e0fcf
Comparison: 2c17522
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine quickly if vector performance is changed and to what degree by a pull request.

The table below, if present, lists those experiments that have experienced a statistically significant change in mean optimization goal performance between baseline and comparison SHAs with 90.00% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±5.00% change in their mean optimization goal are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting change is observed.

No interesting changes in experiment optimization goals with confidence ≥ 90.00% and |Δ mean %| ≥ 5.00%.

Fine details of change detection per experiment.
experiment goal Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
syslog_splunk_hec_logs ingress throughput 228.03KiB/CPU-s 2.57 100.00% 8.67MiB/CPU-s 307.89KiB/CPU-s 3.79KiB/CPU-s 0.0 0.034685 8.89MiB/CPU-s 207.66KiB/CPU-s 2.55KiB/CPU-s 0.0 0.022808 False False
otlp_grpc_to_blackhole ingress throughput 24.61KiB/CPU-s 2.41 100.00% 1022.23KiB/CPU-s 59.13KiB/CPU-s 744.99B/CPU-s 0.0 0.057842 1.02MiB/CPU-s 53.99KiB/CPU-s 680.27B/CPU-s 0.0 0.051575 False False
otlp_http_to_blackhole ingress throughput 30.26KiB/CPU-s 1.95 100.00% 1.52MiB/CPU-s 117.46KiB/CPU-s 1.45KiB/CPU-s 0.0 0.075514 1.55MiB/CPU-s 116.36KiB/CPU-s 1.43KiB/CPU-s 0.0 0.073378 False False
socket_to_socket_blackhole ingress throughput 186.02KiB/CPU-s 1.38 100.00% 13.19MiB/CPU-s 270.6KiB/CPU-s 3.33KiB/CPU-s 0.0 0.020033 13.37MiB/CPU-s 274.7KiB/CPU-s 3.38KiB/CPU-s 0.0 0.02006 False False
syslog_humio_logs ingress throughput 109.48KiB/CPU-s 1.21 100.00% 8.84MiB/CPU-s 276.07KiB/CPU-s 3.4KiB/CPU-s 0.0 0.030484 8.95MiB/CPU-s 208.12KiB/CPU-s 2.56KiB/CPU-s 0.0 0.022706 False False
datadog_agent_remap_datadog_logs ingress throughput 383.36KiB/CPU-s 1.19 100.00% 31.58MiB/CPU-s 1.22MiB/CPU-s 15.41KiB/CPU-s 0.0 0.038713 31.96MiB/CPU-s 1.3MiB/CPU-s 16.36KiB/CPU-s 0.0 0.040617 False False
http_text_to_http_json ingress throughput 235.73KiB/CPU-s 0.99 100.00% 23.29MiB/CPU-s 1.12MiB/CPU-s 14.09KiB/CPU-s 0.0 0.048016 23.52MiB/CPU-s 601.42KiB/CPU-s 7.4KiB/CPU-s 0.0 0.024969 False False
datadog_agent_remap_blackhole ingress throughput 251.72KiB/CPU-s 0.79 100.00% 31.27MiB/CPU-s 1.06MiB/CPU-s 13.33KiB/CPU-s 0.0 0.033833 31.51MiB/CPU-s 969.93KiB/CPU-s 11.94KiB/CPU-s 0.0 0.030056 False False
syslog_regex_logs2metric_ddmetrics ingress throughput 13.07KiB/CPU-s 0.36 90.74% 3.5MiB/CPU-s 495.34KiB/CPU-s 6.09KiB/CPU-s 0.0 0.138229 3.51MiB/CPU-s 391.73KiB/CPU-s 4.82KiB/CPU-s 0.0 0.108919 True True
datadog_agent_remap_blackhole_acks ingress throughput 69.6KiB/CPU-s 0.22 98.95% 30.23MiB/CPU-s 1.66MiB/CPU-s 20.98KiB/CPU-s 0.0 0.055072 30.3MiB/CPU-s 1.38MiB/CPU-s 17.33KiB/CPU-s 0.0 0.045384 False False
http_to_http_acks ingress throughput 7.25KiB/CPU-s 0.14 11.91% 5.24MiB/CPU-s 2.7MiB/CPU-s 33.96KiB/CPU-s 0.0 0.51407 5.25MiB/CPU-s 2.74MiB/CPU-s 34.47KiB/CPU-s 0.0 0.521052 True False
enterprise_http_to_http ingress throughput 5.44KiB/CPU-s 0.04 81.74% 13.62MiB/CPU-s 272.28KiB/CPU-s 3.35KiB/CPU-s 0.0 0.019525 13.62MiB/CPU-s 189.49KiB/CPU-s 2.33KiB/CPU-s 0.0 0.013583 False False
splunk_hec_to_splunk_hec_logs_noack ingress throughput 639.04B/CPU-s 0.00 10.61% 13.61MiB/CPU-s 269.95KiB/CPU-s 3.32KiB/CPU-s 0.0 0.019362 13.61MiB/CPU-s 268.09KiB/CPU-s 3.3KiB/CPU-s 0.0 0.019228 False False
splunk_hec_indexer_ack_blackhole ingress throughput -149.96B/CPU-s -0.00 2.56% 13.61MiB/CPU-s 262.84KiB/CPU-s 3.23KiB/CPU-s 0.0 0.018851 13.61MiB/CPU-s 262.79KiB/CPU-s 3.23KiB/CPU-s 0.0 0.018848 False False
splunk_hec_to_splunk_hec_logs_acks ingress throughput 44.15B/CPU-s 0.00 0.54% 13.62MiB/CPU-s 362.87KiB/CPU-s 4.46KiB/CPU-s 0.0 0.026025 13.62MiB/CPU-s 371.08KiB/CPU-s 4.56KiB/CPU-s 0.0 0.026615 False False
fluent_elasticsearch ingress throughput -8.21KiB/CPU-s -0.02 91.78% 45.41MiB/CPU-s 30.17KiB/CPU-s 375.84B/CPU-s 0.0 0.000649 45.41MiB/CPU-s 386.89KiB/CPU-s 4.71KiB/CPU-s 0.0 0.00832 False False
datadog_agent_remap_datadog_logs_acks ingress throughput -8.28KiB/CPU-s -0.03 32.46% 31.99MiB/CPU-s 1.14MiB/CPU-s 14.38KiB/CPU-s 0.0 0.035689 31.98MiB/CPU-s 1.08MiB/CPU-s 13.57KiB/CPU-s 0.0 0.033671 False False
http_to_http_noack ingress throughput -3.9KiB/CPU-s -0.03 46.58% 13.61MiB/CPU-s 341.21KiB/CPU-s 4.2KiB/CPU-s 0.0 0.024479 13.61MiB/CPU-s 378.82KiB/CPU-s 4.66KiB/CPU-s 0.0 0.027185 False False
http_to_http_json ingress throughput -16.24KiB/CPU-s -0.12 99.69% 13.57MiB/CPU-s 310.43KiB/CPU-s 3.82KiB/CPU-s 0.0 0.022331 13.56MiB/CPU-s 321.36KiB/CPU-s 3.95KiB/CPU-s 0.0 0.023145 False False
splunk_hec_route_s3 ingress throughput -58.57KiB/CPU-s -0.49 100.00% 11.67MiB/CPU-s 553.78KiB/CPU-s 6.81KiB/CPU-s 0.0 0.046354 11.61MiB/CPU-s 548.65KiB/CPU-s 6.75KiB/CPU-s 0.0 0.046151 False False
syslog_log2metric_splunk_hec_metrics ingress throughput -89.86KiB/CPU-s -0.97 100.00% 9.07MiB/CPU-s 257.06KiB/CPU-s 3.16KiB/CPU-s 0.0 0.027666 8.99MiB/CPU-s 368.5KiB/CPU-s 4.53KiB/CPU-s 0.0 0.040047 False False
syslog_log2metric_humio_metrics ingress throughput -94.57KiB/CPU-s -1.49 100.00% 6.19MiB/CPU-s 241.78KiB/CPU-s 2.98KiB/CPU-s 0.0 0.038152 6.1MiB/CPU-s 167.08KiB/CPU-s 2.06KiB/CPU-s 0.0 0.026764 False False
file_to_blackhole egress throughput -123.49KiB/CPU-s -1.65 49.53% 7.32MiB/CPU-s 3.98MiB/CPU-s 131.55KiB/CPU-s 3.118503 0.543899 7.2MiB/CPU-s 3.86MiB/CPU-s 130.19KiB/CPU-s 0.0 0.536362 True True
syslog_loki ingress throughput -247.89KiB/CPU-s -2.96 100.00% 8.17MiB/CPU-s 317.08KiB/CPU-s 3.9KiB/CPU-s 0.0 0.037894 7.93MiB/CPU-s 328.49KiB/CPU-s 4.04KiB/CPU-s 0.0 0.040456 False False

Signed-off-by: Jesse Szwedko <jesse.szwedko@datadoghq.com>
@jszwedko jszwedko requested a review from a team as a code owner March 28, 2023 13:51
@github-actions github-actions bot added the domain: external docs Anything related to Vector's external, public documentation label Mar 28, 2023
@neuronull neuronull removed the request for review from davidhuie-dd March 28, 2023 15:54
@jszwedko jszwedko merged commit 96a059a into vectordotdev:master Mar 28, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
domain: core Anything related to core crates i.e. vector-core, core-common, etc domain: external docs Anything related to Vector's external, public documentation domain: sinks Anything related to the Vector's sinks domain: sources Anything related to the Vector's sources domain: transforms Anything related to Vector's transform components
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants