Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

chore(vdev): Rewrite compile-vrl-wasm.sh to vdev #16751

Merged
merged 8 commits into from
Mar 16, 2023
Merged

chore(vdev): Rewrite compile-vrl-wasm.sh to vdev #16751

merged 8 commits into from
Mar 16, 2023

Conversation

jonathanpv
Copy link
Contributor

@jonathanpv jonathanpv commented Mar 9, 2023 •
edited
Loading

Problem

We would like to rewrite the compile-vrl-wasm.sh script to native Rust in vdev

Solution

  • Rewrote compile-vrl-wasm.sh into vdev meta subcommand
  • Deleted compile-vrl-wasm.sh
  • Modified Makefile PHONY for compile-vrl-wasm to utilize cargo vdev meta compile-vrl-wasm

Testing

cargo vdev generate vrl-wasm

Related Issues

Checklist

  • Add the ci-condition integration tests enable
  • Delete any rewritten scripts
  • Modify makefile to point to the new cargo vdev
  • Modify workflow if need be
  • Compare the output of the local script to the newly created cargo vdev Rust version
  • All CI integration tests passing
  • Test newly created cargo vdev commands

@jonathanpv jonathanpv added ci-condition: integration tests enable Run integration tests on this PR domain: vdev Anything related to the vdev tooling labels Mar 9, 2023
@jonathanpv jonathanpv self-assigned this Mar 9, 2023
@netlify
Copy link

netlify bot commented Mar 9, 2023 •
edited
Loading

✅ Deploy Preview for vrl-playground canceled.

Name Link
🔨 Latest commit 32c00c3
🔍 Latest deploy log https://app.netlify.com/sites/vrl-playground/deploys/6411d49cc612590008fca64e

@github-actions github-actions bot added the domain: ci Anything related to Vector's CI environment label Mar 9, 2023
@netlify
Copy link

netlify bot commented Mar 9, 2023 •
edited
Loading

✅ Deploy Preview for vector-project canceled.

Name Link
🔨 Latest commit 32c00c3
🔍 Latest deploy log https://app.netlify.com/sites/vector-project/deploys/6411d49cd45a3d00080f0623

bruceg
bruceg requested changes Mar 9, 2023
View reviewed changes
vdev/src/commands/meta/mod.rs Outdated Show resolved Hide resolved
vdev/src/commands/meta/compile_vrl_wasm.rs Outdated Show resolved Hide resolved
vdev/src/commands/meta/compile_vrl_wasm.rs Outdated Show resolved Hide resolved
@github-actions
Copy link

github-actions bot commented Mar 9, 2023

Regression Detector Results

Run ID: 4ec1680f-1e3d-4b37-922d-8d03c63cfef2
Baseline: 0eb721f
Comparison: fe3a9e8
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine quickly if vector performance is changed and to what degree by a pull request.

The table below, if present, lists those experiments that have experienced a statistically significant change in mean optimization goal performance between baseline and comparison SHAs with 90.00% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±5.00% change in their mean optimization goal are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting change is observed.

No interesting changes in experiment optimization goals with confidence ≥ 90.00% and |Δ mean %| ≥ 5.00%.

Fine details of change detection per experiment.
experiment goal Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
file_to_blackhole egress throughput 453.43KiB/CPU-s 7.52 99.32% 5.89MiB/CPU-s 4.22MiB/CPU-s 118.68KiB/CPU-s 0.0 0.715704 6.33MiB/CPU-s 4.21MiB/CPU-s 118.08KiB/CPU-s 1.273408 0.665029 True True
splunk_hec_route_s3 ingress throughput 391.72KiB/CPU-s 3.44 100.00% 11.13MiB/CPU-s 688.57KiB/CPU-s 8.47KiB/CPU-s 0.0 0.060399 11.51MiB/CPU-s 555.98KiB/CPU-s 6.84KiB/CPU-s 0.0 0.047148 False False
syslog_splunk_hec_logs ingress throughput 102.6KiB/CPU-s 1.17 100.00% 8.56MiB/CPU-s 347.96KiB/CPU-s 4.28KiB/CPU-s 0.0 0.039696 8.66MiB/CPU-s 215.29KiB/CPU-s 2.65KiB/CPU-s 0.0 0.024276 False False
otlp_grpc_to_blackhole ingress throughput 8.97KiB/CPU-s 0.86 100.00% 1.02MiB/CPU-s 52.34KiB/CPU-s 659.58B/CPU-s 0.0 0.049955 1.03MiB/CPU-s 45.46KiB/CPU-s 572.85B/CPU-s 0.0 0.043019 False False
syslog_log2metric_splunk_hec_metrics ingress throughput 64.34KiB/CPU-s 0.72 100.00% 8.76MiB/CPU-s 426.07KiB/CPU-s 5.24KiB/CPU-s 0.0 0.047473 8.83MiB/CPU-s 414.92KiB/CPU-s 5.1KiB/CPU-s 0.0 0.045902 False False
datadog_agent_remap_blackhole ingress throughput 190.38KiB/CPU-s 0.60 100.00% 30.93MiB/CPU-s 1.23MiB/CPU-s 15.48KiB/CPU-s 0.0 0.039728 31.11MiB/CPU-s 812.53KiB/CPU-s 10.0KiB/CPU-s 0.0 0.025501 False False
syslog_loki ingress throughput 19.92KiB/CPU-s 0.23 100.00% 8.5MiB/CPU-s 179.06KiB/CPU-s 2.2KiB/CPU-s 0.0 0.020579 8.52MiB/CPU-s 173.86KiB/CPU-s 2.14KiB/CPU-s 0.0 0.019936 False False
enterprise_http_to_http ingress throughput 9.33KiB/CPU-s 0.07 96.06% 13.61MiB/CPU-s 314.47KiB/CPU-s 3.87KiB/CPU-s 0.0 0.022557 13.62MiB/CPU-s 191.28KiB/CPU-s 2.35KiB/CPU-s 0.0 0.013711 False False
http_to_http_noack ingress throughput 5.76KiB/CPU-s 0.04 68.82% 13.61MiB/CPU-s 357.8KiB/CPU-s 4.4KiB/CPU-s 0.0 0.025673 13.62MiB/CPU-s 293.14KiB/CPU-s 3.61KiB/CPU-s 0.0 0.021024 False False
otlp_http_to_blackhole ingress throughput 271.8B/CPU-s 0.02 10.57% 1.54MiB/CPU-s 118.3KiB/CPU-s 1.46KiB/CPU-s 0.0 0.075066 1.54MiB/CPU-s 111.22KiB/CPU-s 1.37KiB/CPU-s 0.0 0.070565 False False
fluent_elasticsearch ingress throughput -29.2B/CPU-s -0.00 4.39% 45.41MiB/CPU-s 30.15KiB/CPU-s 375.77B/CPU-s 0.0 0.000648 45.41MiB/CPU-s 30.04KiB/CPU-s 374.41B/CPU-s 0.0 0.000646 False False
splunk_hec_indexer_ack_blackhole ingress throughput 201.23B/CPU-s 0.00 3.37% 13.61MiB/CPU-s 267.08KiB/CPU-s 3.28KiB/CPU-s 0.0 0.019156 13.61MiB/CPU-s 267.23KiB/CPU-s 3.29KiB/CPU-s 0.0 0.019167 False False
http_to_http_json ingress throughput 199.78B/CPU-s 0.00 4.11% 13.62MiB/CPU-s 217.76KiB/CPU-s 2.68KiB/CPU-s 0.0 0.015612 13.62MiB/CPU-s 217.4KiB/CPU-s 2.67KiB/CPU-s 0.0 0.015586 False False
splunk_hec_to_splunk_hec_logs_noack ingress throughput -1.15KiB/CPU-s -0.01 19.24% 13.61MiB/CPU-s 267.09KiB/CPU-s 3.29KiB/CPU-s 0.0 0.019156 13.61MiB/CPU-s 274.09KiB/CPU-s 3.37KiB/CPU-s 0.0 0.01966 False False
splunk_hec_to_splunk_hec_logs_acks ingress throughput -2.79KiB/CPU-s -0.02 34.81% 13.62MiB/CPU-s 355.22KiB/CPU-s 4.37KiB/CPU-s 0.0 0.025475 13.61MiB/CPU-s 354.84KiB/CPU-s 4.36KiB/CPU-s 0.0 0.025453 False False
http_text_to_http_json ingress throughput -3.9KiB/CPU-s -0.02 23.98% 23.93MiB/CPU-s 709.05KiB/CPU-s 8.72KiB/CPU-s 0.0 0.028932 23.93MiB/CPU-s 759.64KiB/CPU-s 9.35KiB/CPU-s 0.0 0.031001 False False
syslog_humio_logs ingress throughput -23.42KiB/CPU-s -0.26 100.00% 8.76MiB/CPU-s 195.37KiB/CPU-s 2.4KiB/CPU-s 0.0 0.021789 8.73MiB/CPU-s 317.01KiB/CPU-s 3.9KiB/CPU-s 0.0 0.035448 False False
syslog_log2metric_humio_metrics ingress throughput -16.5KiB/CPU-s -0.26 100.00% 6.16MiB/CPU-s 147.62KiB/CPU-s 1.82KiB/CPU-s 0.0 0.023402 6.14MiB/CPU-s 162.49KiB/CPU-s 2.0KiB/CPU-s 0.0 0.025827 False False
datadog_agent_remap_blackhole_acks ingress throughput -89.27KiB/CPU-s -0.28 100.00% 31.69MiB/CPU-s 948.65KiB/CPU-s 11.67KiB/CPU-s 0.0 0.029235 31.6MiB/CPU-s 778.65KiB/CPU-s 9.58KiB/CPU-s 0.0 0.024062 False False
syslog_regex_logs2metric_ddmetrics ingress throughput -44.07KiB/CPU-s -1.15 100.00% 3.74MiB/CPU-s 432.35KiB/CPU-s 5.32KiB/CPU-s 0.0 0.112763 3.7MiB/CPU-s 395.25KiB/CPU-s 4.86KiB/CPU-s 0.0 0.104286 True True
datadog_agent_remap_datadog_logs_acks ingress throughput -424.31KiB/CPU-s -1.29 100.00% 32.04MiB/CPU-s 1.26MiB/CPU-s 15.88KiB/CPU-s 0.0 0.039351 31.62MiB/CPU-s 1.76MiB/CPU-s 22.17KiB/CPU-s 0.0 0.055639 False False
http_to_http_acks ingress throughput -87.63KiB/CPU-s -1.62 92.79% 5.28MiB/CPU-s 2.78MiB/CPU-s 34.99KiB/CPU-s 0.0 0.526159 5.19MiB/CPU-s 2.69MiB/CPU-s 33.9KiB/CPU-s 0.0 0.51825 True False
datadog_agent_remap_datadog_logs ingress throughput -627.1KiB/CPU-s -1.85 100.00% 33.13MiB/CPU-s 969.91KiB/CPU-s 11.93KiB/CPU-s 0.0 0.028585 32.52MiB/CPU-s 1.33MiB/CPU-s 16.76KiB/CPU-s 0.0 0.040892 False False
socket_to_socket_blackhole ingress throughput -374.16KiB/CPU-s -2.77 100.00% 13.19MiB/CPU-s 306.95KiB/CPU-s 3.78KiB/CPU-s 0.0 0.022733 12.82MiB/CPU-s 655.88KiB/CPU-s 8.07KiB/CPU-s 0.0 0.049959 False False

@jonathanpv jonathanpv requested a review from bruceg March 9, 2023 23:33
@github-actions
Copy link

Regression Detector Results

Run ID: 13a0708b-3502-40b6-9e7e-7752642a4345
Baseline: 0eb721f
Comparison: 7ec1898
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine quickly if vector performance is changed and to what degree by a pull request.

The table below, if present, lists those experiments that have experienced a statistically significant change in mean optimization goal performance between baseline and comparison SHAs with 90.00% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±5.00% change in their mean optimization goal are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting change is observed.

No interesting changes in experiment optimization goals with confidence ≥ 90.00% and |Δ mean %| ≥ 5.00%.

Fine details of change detection per experiment.
experiment goal Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
syslog_regex_logs2metric_ddmetrics ingress throughput 98.98KiB/CPU-s 2.70 100.00% 3.59MiB/CPU-s 430.86KiB/CPU-s 5.3KiB/CPU-s 0.0 0.117314 3.68MiB/CPU-s 394.01KiB/CPU-s 4.85KiB/CPU-s 0.0 0.104465 True True
datadog_agent_remap_datadog_logs ingress throughput 858.43KiB/CPU-s 2.62 100.00% 31.97MiB/CPU-s 1.52MiB/CPU-s 19.17KiB/CPU-s 0.0 0.047593 32.81MiB/CPU-s 899.54KiB/CPU-s 11.07KiB/CPU-s 0.0 0.026773 False False
datadog_agent_remap_blackhole ingress throughput 540.82KiB/CPU-s 1.73 100.00% 30.49MiB/CPU-s 1.38MiB/CPU-s 17.39KiB/CPU-s 0.0 0.045272 31.02MiB/CPU-s 1.07MiB/CPU-s 13.52KiB/CPU-s 0.0 0.034601 False False
syslog_log2metric_splunk_hec_metrics ingress throughput 133.39KiB/CPU-s 1.44 100.00% 9.03MiB/CPU-s 402.56KiB/CPU-s 4.95KiB/CPU-s 0.0 0.043541 9.16MiB/CPU-s 244.59KiB/CPU-s 3.01KiB/CPU-s 0.0 0.026079 False False
syslog_loki ingress throughput 105.36KiB/CPU-s 1.23 100.00% 8.39MiB/CPU-s 314.67KiB/CPU-s 3.87KiB/CPU-s 0.0 0.036611 8.5MiB/CPU-s 211.45KiB/CPU-s 2.6KiB/CPU-s 0.0 0.024304 False False
otlp_http_to_blackhole ingress throughput 11.37KiB/CPU-s 0.72 100.00% 1.55MiB/CPU-s 117.65KiB/CPU-s 1.45KiB/CPU-s 0.0 0.074183 1.56MiB/CPU-s 103.58KiB/CPU-s 1.27KiB/CPU-s 0.0 0.064846 False False
syslog_splunk_hec_logs ingress throughput 48.54KiB/CPU-s 0.54 100.00% 8.78MiB/CPU-s 245.26KiB/CPU-s 3.02KiB/CPU-s 0.0 0.027287 8.82MiB/CPU-s 221.31KiB/CPU-s 2.72KiB/CPU-s 0.0 0.02449 False False
syslog_log2metric_humio_metrics ingress throughput 16.34KiB/CPU-s 0.26 100.00% 6.11MiB/CPU-s 154.25KiB/CPU-s 1.99KiB/CPU-s 0.0 0.024658 6.12MiB/CPU-s 276.45KiB/CPU-s 3.4KiB/CPU-s 0.0 0.044076 False False
syslog_humio_logs ingress throughput 20.77KiB/CPU-s 0.23 100.00% 8.82MiB/CPU-s 258.08KiB/CPU-s 3.18KiB/CPU-s 0.0 0.028582 8.84MiB/CPU-s 245.67KiB/CPU-s 3.02KiB/CPU-s 0.0 0.027144 False False
http_to_http_acks ingress throughput 3.41KiB/CPU-s 0.06 5.52% 5.28MiB/CPU-s 2.76MiB/CPU-s 34.76KiB/CPU-s 0.0 0.522826 5.28MiB/CPU-s 2.76MiB/CPU-s 34.83KiB/CPU-s 0.0 0.523635 True False
enterprise_http_to_http ingress throughput 5.67KiB/CPU-s 0.04 88.64% 13.62MiB/CPU-s 248.04KiB/CPU-s 3.05KiB/CPU-s 0.0 0.017785 13.62MiB/CPU-s 152.95KiB/CPU-s 1.88KiB/CPU-s 0.0 0.010962 False False
splunk_hec_to_splunk_hec_logs_acks ingress throughput 933.23B/CPU-s 0.01 11.41% 13.61MiB/CPU-s 363.79KiB/CPU-s 4.47KiB/CPU-s 0.0 0.026094 13.61MiB/CPU-s 366.76KiB/CPU-s 4.51KiB/CPU-s 0.0 0.026305 False False
splunk_hec_indexer_ack_blackhole ingress throughput 961.74B/CPU-s 0.01 16.43% 13.61MiB/CPU-s 263.13KiB/CPU-s 3.24KiB/CPU-s 0.0 0.018873 13.62MiB/CPU-s 257.6KiB/CPU-s 3.17KiB/CPU-s 0.0 0.018475 False False
http_to_http_noack ingress throughput -116.23B/CPU-s -0.00 1.47% 13.61MiB/CPU-s 352.98KiB/CPU-s 4.34KiB/CPU-s 0.0 0.025326 13.61MiB/CPU-s 356.9KiB/CPU-s 4.39KiB/CPU-s 0.0 0.025608 False False
fluent_elasticsearch ingress throughput -292.53B/CPU-s -0.00 42.21% 45.41MiB/CPU-s 29.87KiB/CPU-s 372.16B/CPU-s 0.0 0.000642 45.41MiB/CPU-s 29.79KiB/CPU-s 371.21B/CPU-s 0.0 0.000641 False False
http_to_http_json ingress throughput -23.07B/CPU-s -0.00 0.47% 13.62MiB/CPU-s 218.2KiB/CPU-s 2.68KiB/CPU-s 0.0 0.015643 13.62MiB/CPU-s 218.33KiB/CPU-s 2.69KiB/CPU-s 0.0 0.015653 False False
splunk_hec_to_splunk_hec_logs_noack ingress throughput -1.07KiB/CPU-s -0.01 17.98% 13.61MiB/CPU-s 268.2KiB/CPU-s 3.3KiB/CPU-s 0.0 0.019236 13.61MiB/CPU-s 275.07KiB/CPU-s 3.38KiB/CPU-s 0.0 0.01973 False False
otlp_grpc_to_blackhole ingress throughput -2.15KiB/CPU-s -0.20 98.30% 1.03MiB/CPU-s 49.92KiB/CPU-s 628.94B/CPU-s 0.0 0.047474 1.02MiB/CPU-s 53.51KiB/CPU-s 674.16B/CPU-s 0.0 0.050999 False False
http_text_to_http_json ingress throughput -139.99KiB/CPU-s -0.57 100.00% 24.16MiB/CPU-s 565.01KiB/CPU-s 6.95KiB/CPU-s 0.0 0.022832 24.03MiB/CPU-s 655.48KiB/CPU-s 8.07KiB/CPU-s 0.0 0.026638 False False
datadog_agent_remap_blackhole_acks ingress throughput -229.9KiB/CPU-s -0.73 100.00% 30.73MiB/CPU-s 1.21MiB/CPU-s 15.24KiB/CPU-s 0.0 0.039362 30.5MiB/CPU-s 1.38MiB/CPU-s 17.42KiB/CPU-s 0.0 0.045319 False False
splunk_hec_route_s3 ingress throughput -137.4KiB/CPU-s -1.16 100.00% 11.54MiB/CPU-s 587.39KiB/CPU-s 7.23KiB/CPU-s 0.0 0.049718 11.4MiB/CPU-s 693.99KiB/CPU-s 8.54KiB/CPU-s 0.0 0.059433 False False
socket_to_socket_blackhole ingress throughput -216.3KiB/CPU-s -1.58 100.00% 13.37MiB/CPU-s 324.83KiB/CPU-s 4.0KiB/CPU-s 0.0 0.023732 13.15MiB/CPU-s 305.25KiB/CPU-s 3.76KiB/CPU-s 0.0 0.02266 False False
datadog_agent_remap_datadog_logs_acks ingress throughput -789.81KiB/CPU-s -2.39 100.00% 32.32MiB/CPU-s 1.33MiB/CPU-s 16.76KiB/CPU-s 0.0 0.041154 31.55MiB/CPU-s 1.51MiB/CPU-s 18.99KiB/CPU-s 0.0 0.04777 False False
file_to_blackhole egress throughput -380.58KiB/CPU-s -5.56 97.05% 6.69MiB/CPU-s 4.19MiB/CPU-s 127.98KiB/CPU-s 3.552398 0.626595 6.32MiB/CPU-s 4.21MiB/CPU-s 119.01KiB/CPU-s 1.447068 0.666231 True True

@github-actions
Copy link

Regression Detector Results

Run ID: 73878f20-0389-4c35-a065-b81b5b7f619e
Baseline: 5b49afd
Comparison: bf29919
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine quickly if vector performance is changed and to what degree by a pull request.

The table below, if present, lists those experiments that have experienced a statistically significant change in mean optimization goal performance between baseline and comparison SHAs with 90.00% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±5.00% change in their mean optimization goal are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting change is observed.

Changes in experiment optimization goals with confidence ≥ 90.00% and |Δ mean %| ≥ 5.00%:

experiment goal Δ mean Δ mean % confidence
http_text_to_http_json ingress throughput -1.33MiB/CPU-s -5.18 100.00%
Fine details of change detection per experiment.
experiment goal Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
syslog_log2metric_splunk_hec_metrics ingress throughput 241.58KiB/CPU-s 2.65 100.00% 8.89MiB/CPU-s 260.86KiB/CPU-s 3.21KiB/CPU-s 0.0 0.028644 9.13MiB/CPU-s 246.21KiB/CPU-s 3.03KiB/CPU-s 0.0 0.026336 False False
file_to_blackhole egress throughput 165.64KiB/CPU-s 2.42 66.96% 6.69MiB/CPU-s 3.92MiB/CPU-s 118.14KiB/CPU-s 0.864304 0.586626 6.85MiB/CPU-s 3.79MiB/CPU-s 122.45KiB/CPU-s 0.0 0.552964 True True
datadog_agent_remap_blackhole_acks ingress throughput 676.11KiB/CPU-s 2.16 100.00% 30.53MiB/CPU-s 2.0MiB/CPU-s 25.21KiB/CPU-s 0.0 0.065539 31.19MiB/CPU-s 878.14KiB/CPU-s 10.81KiB/CPU-s 0.0 0.02749 False False
http_to_http_json ingress throughput 246.57KiB/CPU-s 1.80 100.00% 13.37MiB/CPU-s 685.43KiB/CPU-s 8.43KiB/CPU-s 0.0 0.05006 13.61MiB/CPU-s 249.7KiB/CPU-s 3.07KiB/CPU-s 0.0 0.017915 False False
syslog_loki ingress throughput 135.14KiB/CPU-s 1.58 100.00% 8.33MiB/CPU-s 293.22KiB/CPU-s 3.61KiB/CPU-s 0.0 0.034357 8.47MiB/CPU-s 295.19KiB/CPU-s 3.63KiB/CPU-s 0.0 0.034048 False False
syslog_regex_logs2metric_ddmetrics ingress throughput 23.88KiB/CPU-s 0.67 99.95% 3.5MiB/CPU-s 350.32KiB/CPU-s 4.31KiB/CPU-s 0.0 0.097698 3.52MiB/CPU-s 434.5KiB/CPU-s 5.35KiB/CPU-s 0.0 0.120372 True True
syslog_log2metric_humio_metrics ingress throughput 24.46KiB/CPU-s 0.40 100.00% 5.98MiB/CPU-s 226.86KiB/CPU-s 2.79KiB/CPU-s 0.0 0.037029 6.01MiB/CPU-s 308.44KiB/CPU-s 3.8KiB/CPU-s 0.0 0.050145 False False
datadog_agent_remap_datadog_logs ingress throughput 86.99KiB/CPU-s 0.26 99.99% 32.66MiB/CPU-s 1.29MiB/CPU-s 16.28KiB/CPU-s 0.0 0.039544 32.75MiB/CPU-s 1.25MiB/CPU-s 15.78KiB/CPU-s 0.0 0.03823 False False
enterprise_http_to_http ingress throughput 7.06KiB/CPU-s 0.05 87.03% 13.61MiB/CPU-s 308.98KiB/CPU-s 3.8KiB/CPU-s 0.0 0.022163 13.62MiB/CPU-s 218.95KiB/CPU-s 2.69KiB/CPU-s 0.0 0.015697 False False
http_to_http_noack ingress throughput 5.45KiB/CPU-s 0.04 66.92% 13.61MiB/CPU-s 348.33KiB/CPU-s 4.29KiB/CPU-s 0.0 0.024993 13.61MiB/CPU-s 293.59KiB/CPU-s 3.61KiB/CPU-s 0.0 0.021057 False False
splunk_hec_indexer_ack_blackhole ingress throughput 1.94KiB/CPU-s 0.01 33.14% 13.61MiB/CPU-s 265.75KiB/CPU-s 3.27KiB/CPU-s 0.0 0.019061 13.62MiB/CPU-s 256.39KiB/CPU-s 3.15KiB/CPU-s 0.0 0.018387 False False
splunk_hec_to_splunk_hec_logs_acks ingress throughput 1.1KiB/CPU-s 0.01 13.66% 13.61MiB/CPU-s 369.59KiB/CPU-s 4.55KiB/CPU-s 0.0 0.02651 13.61MiB/CPU-s 363.15KiB/CPU-s 4.47KiB/CPU-s 0.0 0.026046 False False
fluent_elasticsearch ingress throughput 87.0B/CPU-s 0.00 13.01% 45.41MiB/CPU-s 29.93KiB/CPU-s 372.94B/CPU-s 0.0 0.000644 45.41MiB/CPU-s 30.36KiB/CPU-s 378.31B/CPU-s 0.0 0.000653 False False
splunk_hec_to_splunk_hec_logs_noack ingress throughput -6.2KiB/CPU-s -0.04 83.95% 13.62MiB/CPU-s 232.69KiB/CPU-s 2.86KiB/CPU-s 0.0 0.016683 13.61MiB/CPU-s 273.86KiB/CPU-s 3.37KiB/CPU-s 0.0 0.019643 False False
syslog_splunk_hec_logs ingress throughput -11.96KiB/CPU-s -0.13 99.14% 8.75MiB/CPU-s 220.95KiB/CPU-s 2.72KiB/CPU-s 0.0 0.024665 8.74MiB/CPU-s 296.85KiB/CPU-s 3.65KiB/CPU-s 0.0 0.033183 False False
datadog_agent_remap_blackhole ingress throughput -66.59KiB/CPU-s -0.21 97.53% 30.4MiB/CPU-s 1.65MiB/CPU-s 20.82KiB/CPU-s 0.0 0.054358 30.34MiB/CPU-s 1.67MiB/CPU-s 21.08KiB/CPU-s 0.0 0.055152 False False
splunk_hec_route_s3 ingress throughput -26.97KiB/CPU-s -0.23 99.58% 11.65MiB/CPU-s 551.02KiB/CPU-s 6.78KiB/CPU-s 0.0 0.046205 11.62MiB/CPU-s 531.06KiB/CPU-s 6.53KiB/CPU-s 0.0 0.044631 False False
http_to_http_acks ingress throughput -13.21KiB/CPU-s -0.24 21.31% 5.32MiB/CPU-s 2.75MiB/CPU-s 34.66KiB/CPU-s 0.0 0.517097 5.31MiB/CPU-s 2.73MiB/CPU-s 34.44KiB/CPU-s 0.0 0.515021 True False
datadog_agent_remap_datadog_logs_acks ingress throughput -132.18KiB/CPU-s -0.39 100.00% 33.27MiB/CPU-s 955.23KiB/CPU-s 11.75KiB/CPU-s 0.0 0.028037 33.14MiB/CPU-s 1.06MiB/CPU-s 13.38KiB/CPU-s 0.0 0.032039 False False
otlp_http_to_blackhole ingress throughput -6.18KiB/CPU-s -0.40 99.60% 1.52MiB/CPU-s 122.07KiB/CPU-s 1.5KiB/CPU-s 0.0 0.078185 1.52MiB/CPU-s 124.79KiB/CPU-s 1.54KiB/CPU-s 0.0 0.08025 False False
syslog_humio_logs ingress throughput -107.65KiB/CPU-s -1.20 100.00% 8.74MiB/CPU-s 264.31KiB/CPU-s 3.25KiB/CPU-s 0.0 0.02954 8.63MiB/CPU-s 282.13KiB/CPU-s 3.47KiB/CPU-s 0.0 0.031916 False False
otlp_grpc_to_blackhole ingress throughput -14.35KiB/CPU-s -1.35 100.00% 1.04MiB/CPU-s 47.86KiB/CPU-s 603.01B/CPU-s 0.0 0.045149 1.02MiB/CPU-s 46.01KiB/CPU-s 579.84B/CPU-s 0.0 0.044007 False False
socket_to_socket_blackhole ingress throughput -201.72KiB/CPU-s -1.47 100.00% 13.42MiB/CPU-s 391.98KiB/CPU-s 4.82KiB/CPU-s 0.0 0.028523 13.22MiB/CPU-s 297.58KiB/CPU-s 3.66KiB/CPU-s 0.0 0.021977 False False
http_text_to_http_json ingress throughput -1.33MiB/CPU-s -5.18 100.00% 25.62MiB/CPU-s 590.28KiB/CPU-s 7.26KiB/CPU-s 0.0 0.022496 24.29MiB/CPU-s 797.05KiB/CPU-s 9.81KiB/CPU-s 0.0 0.032037 False False

bruceg
bruceg reviewed Mar 13, 2023
View reviewed changes
vdev/src/commands/compile_vrl_wasm.rs Outdated Show resolved Hide resolved
vdev/src/commands/compile_vrl_wasm.rs Outdated Show resolved Hide resolved
@jonathanpv jonathanpv requested a review from bruceg March 15, 2023 14:00
@github-actions
Copy link

Regression Detector Results

Run ID: c744c7a4-13c1-411b-a1a9-6f3c03ca453d
Baseline: e8b855a
Comparison: 32c00c3
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine quickly if vector performance is changed and to what degree by a pull request.

The table below, if present, lists those experiments that have experienced a statistically significant change in mean optimization goal performance between baseline and comparison SHAs with 90.00% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±5.00% change in their mean optimization goal are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting change is observed.

Changes in experiment optimization goals with confidence ≥ 90.00% and |Δ mean %| ≥ 5.00%:

experiment goal Δ mean Δ mean % confidence
datadog_agent_remap_blackhole_acks ingress throughput 1.73MiB/CPU-s 5.66 100.00%
Fine details of change detection per experiment.
experiment goal Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
datadog_agent_remap_blackhole_acks ingress throughput 1.73MiB/CPU-s 5.66 100.00% 30.55MiB/CPU-s 1.05MiB/CPU-s 13.22KiB/CPU-s 0.0 0.034338 32.28MiB/CPU-s 433.71KiB/CPU-s 5.34KiB/CPU-s 0.0 0.01312 False False
syslog_humio_logs ingress throughput 421.94KiB/CPU-s 4.84 100.00% 8.52MiB/CPU-s 298.23KiB/CPU-s 3.67KiB/CPU-s 0.0 0.034177 8.93MiB/CPU-s 271.19KiB/CPU-s 3.34KiB/CPU-s 0.0 0.029645 False False
socket_to_socket_blackhole ingress throughput 525.3KiB/CPU-s 3.97 100.00% 12.93MiB/CPU-s 393.65KiB/CPU-s 4.84KiB/CPU-s 0.0 0.029738 13.44MiB/CPU-s 274.97KiB/CPU-s 3.38KiB/CPU-s 0.0 0.019979 False False
http_text_to_http_json ingress throughput 624.06KiB/CPU-s 2.57 100.00% 23.69MiB/CPU-s 727.59KiB/CPU-s 8.95KiB/CPU-s 0.0 0.029995 24.3MiB/CPU-s 588.45KiB/CPU-s 7.24KiB/CPU-s 0.0 0.023651 False False
otlp_grpc_to_blackhole ingress throughput 20.69KiB/CPU-s 2.02 100.00% 1.0MiB/CPU-s 48.66KiB/CPU-s 613.2B/CPU-s 0.0 0.047403 1.02MiB/CPU-s 47.21KiB/CPU-s 594.94B/CPU-s 0.0 0.045083 False False
otlp_http_to_blackhole ingress throughput 29.17KiB/CPU-s 1.91 100.00% 1.49MiB/CPU-s 129.74KiB/CPU-s 1.6KiB/CPU-s 0.0 0.084746 1.52MiB/CPU-s 120.67KiB/CPU-s 1.48KiB/CPU-s 0.0 0.077344 False False
http_to_http_acks ingress throughput 92.61KiB/CPU-s 1.73 94.20% 5.23MiB/CPU-s 2.73MiB/CPU-s 34.44KiB/CPU-s 0.0 0.523021 5.32MiB/CPU-s 2.75MiB/CPU-s 34.63KiB/CPU-s 0.0 0.516905 True False
syslog_log2metric_splunk_hec_metrics ingress throughput 142.6KiB/CPU-s 1.52 100.00% 9.18MiB/CPU-s 341.67KiB/CPU-s 4.21KiB/CPU-s 0.0 0.036347 9.32MiB/CPU-s 158.39KiB/CPU-s 1.95KiB/CPU-s 0.0 0.016598 False False
syslog_loki ingress throughput 116.9KiB/CPU-s 1.35 100.00% 8.48MiB/CPU-s 188.34KiB/CPU-s 2.32KiB/CPU-s 0.0 0.021695 8.59MiB/CPU-s 173.35KiB/CPU-s 2.13KiB/CPU-s 0.0 0.019703 False False
http_to_http_json ingress throughput 129.61KiB/CPU-s 0.94 100.00% 13.45MiB/CPU-s 417.7KiB/CPU-s 5.14KiB/CPU-s 0.0 0.030334 13.57MiB/CPU-s 334.32KiB/CPU-s 4.11KiB/CPU-s 0.0 0.024053 False False
syslog_regex_logs2metric_ddmetrics ingress throughput 33.07KiB/CPU-s 0.89 100.00% 3.63MiB/CPU-s 416.97KiB/CPU-s 5.13KiB/CPU-s 0.0 0.112103 3.66MiB/CPU-s 373.75KiB/CPU-s 4.6KiB/CPU-s 0.0 0.099598 True True
datadog_agent_remap_datadog_logs ingress throughput 257.56KiB/CPU-s 0.76 100.00% 33.24MiB/CPU-s 1.21MiB/CPU-s 15.25KiB/CPU-s 0.0 0.036418 33.49MiB/CPU-s 824.99KiB/CPU-s 10.15KiB/CPU-s 0.0 0.024052 False False
file_to_blackhole egress throughput 11.75KiB/CPU-s 0.17 5.31% 6.74MiB/CPU-s 3.94MiB/CPU-s 121.44KiB/CPU-s 3.807797 0.583699 6.76MiB/CPU-s 4.16MiB/CPU-s 128.06KiB/CPU-s 3.877367 0.61613 True True
enterprise_http_to_http ingress throughput 11.34KiB/CPU-s 0.08 99.13% 13.61MiB/CPU-s 316.74KiB/CPU-s 3.9KiB/CPU-s 0.0 0.02272 13.62MiB/CPU-s 151.94KiB/CPU-s 1.87KiB/CPU-s 0.0 0.01089 False False
http_to_http_noack ingress throughput 3.45KiB/CPU-s 0.02 42.68% 13.61MiB/CPU-s 366.3KiB/CPU-s 4.51KiB/CPU-s 0.0 0.026285 13.61MiB/CPU-s 337.49KiB/CPU-s 4.15KiB/CPU-s 0.0 0.024212 False False
datadog_agent_remap_datadog_logs_acks ingress throughput 5.27KiB/CPU-s 0.02 17.20% 32.14MiB/CPU-s 1.32MiB/CPU-s 16.62KiB/CPU-s 0.0 0.041049 32.14MiB/CPU-s 1.4MiB/CPU-s 17.68KiB/CPU-s 0.0 0.043659 False False
splunk_hec_to_splunk_hec_logs_acks ingress throughput -155.22B/CPU-s -0.00 1.93% 13.61MiB/CPU-s 345.69KiB/CPU-s 4.25KiB/CPU-s 0.0 0.024796 13.61MiB/CPU-s 375.48KiB/CPU-s 4.62KiB/CPU-s 0.0 0.026933 False False
splunk_hec_to_splunk_hec_logs_noack ingress throughput -174.81B/CPU-s -0.00 2.88% 13.62MiB/CPU-s 264.76KiB/CPU-s 3.26KiB/CPU-s 0.0 0.018989 13.61MiB/CPU-s 266.23KiB/CPU-s 3.43KiB/CPU-s 0.0 0.019094 False False
fluent_elasticsearch ingress throughput 294.44B/CPU-s 0.00 42.16% 45.41MiB/CPU-s 30.12KiB/CPU-s 375.3B/CPU-s 0.0 0.000648 45.41MiB/CPU-s 30.02KiB/CPU-s 374.07B/CPU-s 0.0 0.000645 False False
splunk_hec_indexer_ack_blackhole ingress throughput -442.69B/CPU-s -0.00 7.72% 13.62MiB/CPU-s 255.67KiB/CPU-s 3.14KiB/CPU-s 0.0 0.018335 13.62MiB/CPU-s 257.46KiB/CPU-s 3.17KiB/CPU-s 0.0 0.018464 False False
splunk_hec_route_s3 ingress throughput -32.14KiB/CPU-s -0.27 99.68% 11.51MiB/CPU-s 630.21KiB/CPU-s 7.75KiB/CPU-s 0.0 0.053447 11.48MiB/CPU-s 621.35KiB/CPU-s 7.64KiB/CPU-s 0.0 0.052839 False False
syslog_splunk_hec_logs ingress throughput -29.55KiB/CPU-s -0.33 100.00% 8.8MiB/CPU-s 228.57KiB/CPU-s 2.81KiB/CPU-s 0.0 0.025359 8.77MiB/CPU-s 224.65KiB/CPU-s 2.76KiB/CPU-s 0.0 0.025007 False False
datadog_agent_remap_blackhole ingress throughput -370.93KiB/CPU-s -1.15 100.00% 31.39MiB/CPU-s 1.08MiB/CPU-s 13.6KiB/CPU-s 0.0 0.034385 31.02MiB/CPU-s 1.35MiB/CPU-s 17.07KiB/CPU-s 0.0 0.04365 False False
syslog_log2metric_humio_metrics ingress throughput -97.2KiB/CPU-s -1.53 100.00% 6.21MiB/CPU-s 157.79KiB/CPU-s 1.94KiB/CPU-s 0.0 0.024812 6.12MiB/CPU-s 353.83KiB/CPU-s 4.35KiB/CPU-s 0.0 0.056502 False False

@jonathanpv jonathanpv merged commit be9e2c4 into master Mar 16, 2023
@jonathanpv jonathanpv deleted the jonathanpv/rewrite-compile-vrl-wasm branch March 16, 2023 01:17
jeremy-hanna pushed a commit to jeremy-hanna/vector that referenced this pull request Mar 20, 2023
@jonathanpv @jeremy-hanna
chore(vdev): Rewrite compile-vrl-wasm.sh to vdev (vectordotdev#16751)
28da1e2 
* chore(vdev): Rewrite compile-vrl-wasm.sh to vdev

* Move compile-vrl-wasm to the top level

* Remove comment that wasn't an issue to begin with

* Refactor compile_vrl_wasm.rs to utilize for-loop

* Use correct vdev command for compile-vrl-wasm in Makefile

* Use .clone() in compile_vrl_wasm.rs to avoid clippy formatting error

* move vrl-wasm to generate subcommand, modify Makefile to use cargo vdev generate

* remove clone in vrl_wasm.js
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@bruceg bruceg bruceg approved these changes

@spencergilbert spencergilbert Awaiting requested review from spencergilbert

Assignees

@jonathanpv jonathanpv

Labels
ci-condition: integration tests enable Run integration tests on this PR domain: ci Anything related to Vector's CI environment domain: vdev Anything related to the vdev tooling
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@jonathanpv @bruceg