Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(azure service)!: Retrieve .inner() from SensitiveString for Azure ConnectionString #15225

Conversation

archoversight
Copy link
Contributor

@archoversight archoversight commented Nov 16, 2022

azure_common::config::build_client requires a String not a SensitiveString, in the conversion using to_string() we were accidentally converting to the sensitive value to **REDACTED**.

This fixes it by using the same method that seems to have been used in the PR that introduced this change: #14305 to call .inner() to retrieve the value from inside the struct and then converts that to a String.

Closes: #15214
Related:

azure_common::config::build_client requires a String not a SensitiveString, in the conversion using
to_string() we were accidentally converting to the sensitive value to **REDACTED**.

This fixes it by using the same method that seems to have been used in the PR that introduced this
change: vectordotdev#14305 to call `.inner()` to retrieve the value
from inside the struct and then converts that to a String.
@netlify
Copy link

netlify bot commented Nov 16, 2022

Deploy Preview for vrl-playground failed.

Name Link
🔨 Latest commit a18e50f
🔍 Latest deploy log https://app.netlify.com/sites/vrl-playground/deploys/637434c95818c90008469941

@netlify
Copy link

netlify bot commented Nov 16, 2022

Deploy Preview for vector-project ready!

Name Link
🔨 Latest commit a18e50f
🔍 Latest deploy log https://app.netlify.com/sites/vector-project/deploys/637434c9c75948000881e63c
😎 Deploy Preview https://deploy-preview-15225--vector-project.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site settings.

@github-actions github-actions bot added the domain: sinks Anything related to the Vector's sinks label Nov 16, 2022
@github-actions
Copy link

Regression Test Results

Baseline: ecdc5ab
Comparison: a18e50f
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their bytes_written_per_cpu_second performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±5% change in mean bytes_written_per_cpu_second are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in bytes_written_per_cpu_second with confidence ≥ 90.00% and absolute Δ mean >= ±5%:

Fine details of change detection per experiment.
experiment Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
otlp_http_to_blackhole 11.27KiB/CPU-s 0.78 100.00% 1.4MiB/CPU-s 145.06KiB/CPU-s 1.87KiB/CPU-s 0.0 0.100972 1.41MiB/CPU-s 140.22KiB/CPU-s 1.81KiB/CPU-s 0.0 0.096841 True False
socket_to_socket_blackhole 93.73KiB/CPU-s 0.70 100.00% 13.03MiB/CPU-s 298.06KiB/CPU-s 3.85KiB/CPU-s 0.0 0.022343 13.12MiB/CPU-s 183.54KiB/CPU-s 2.37KiB/CPU-s 0.0 0.013663 False False
datadog_agent_remap_blackhole 213.6KiB/CPU-s 0.70 100.00% 30.01MiB/CPU-s 1.15MiB/CPU-s 15.26KiB/CPU-s 0.0 0.038465 30.22MiB/CPU-s 922.3KiB/CPU-s 11.9KiB/CPU-s 0.0 0.029804 False False
syslog_log2metric_humio_metrics 38.32KiB/CPU-s 0.68 100.00% 5.49MiB/CPU-s 128.16KiB/CPU-s 1.65KiB/CPU-s 0.0 0.022777 5.53MiB/CPU-s 115.49KiB/CPU-s 1.49KiB/CPU-s 0.0 0.020388 False False
syslog_loki 55.97KiB/CPU-s 0.63 100.00% 8.72MiB/CPU-s 283.24KiB/CPU-s 3.66KiB/CPU-s 0.0 0.031704 8.78MiB/CPU-s 157.59KiB/CPU-s 2.03KiB/CPU-s 0.0 0.017530 False False
http_to_http_acks 33.28KiB/CPU-s 0.62 48.24% 5.22MiB/CPU-s 2.76MiB/CPU-s 36.42KiB/CPU-s 0.0 0.527606 5.25MiB/CPU-s 2.75MiB/CPU-s 36.31KiB/CPU-s 0.0 0.522512 True False
http_pipelines_blackhole 3.71KiB/CPU-s 0.42 100.00% 874.58KiB/CPU-s 16.27KiB/CPU-s 215.29B/CPU-s 0.0 0.018604 878.29KiB/CPU-s 15.91KiB/CPU-s 210.4B/CPU-s 0.0 0.018116 False False
syslog_regex_logs2metric_ddmetrics 9.04KiB/CPU-s 0.23 69.27% 3.85MiB/CPU-s 484.9KiB/CPU-s 6.26KiB/CPU-s 0.0 0.122930 3.86MiB/CPU-s 485.53KiB/CPU-s 6.27KiB/CPU-s 0.0 0.122808 True False
otlp_grpc_to_blackhole 1.49KiB/CPU-s 0.15 93.20% 965.81KiB/CPU-s 46.97KiB/CPU-s 620.64B/CPU-s 0.0 0.048630 967.3KiB/CPU-s 42.52KiB/CPU-s 561.97B/CPU-s 0.0 0.043954 False False
syslog_humio_logs 12.2KiB/CPU-s 0.13 100.00% 9.24MiB/CPU-s 157.83KiB/CPU-s 2.04KiB/CPU-s 0.0 0.016683 9.25MiB/CPU-s 94.83KiB/CPU-s 1.23KiB/CPU-s 0.0 0.010011 False False
http_pipelines_blackhole_acks 819.24B/CPU-s 0.13 95.12% 638.03KiB/CPU-s 16.61KiB/CPU-s 219.55B/CPU-s 0.0 0.026028 638.83KiB/CPU-s 26.72KiB/CPU-s 353.06B/CPU-s 0.0 0.041830 False False
syslog_splunk_hec_logs 7.73KiB/CPU-s 0.08 99.99% 9.1MiB/CPU-s 105.75KiB/CPU-s 1.37KiB/CPU-s 0.0 0.011350 9.11MiB/CPU-s 109.06KiB/CPU-s 1.41KiB/CPU-s 0.0 0.011696 False False
splunk_hec_indexer_ack_blackhole 1.16KiB/CPU-s 0.01 18.51% 13.62MiB/CPU-s 296.94KiB/CPU-s 3.83KiB/CPU-s 0.0 0.021295 13.62MiB/CPU-s 243.36KiB/CPU-s 3.14KiB/CPU-s 0.0 0.017451 False False
splunk_hec_route_s3 799.52B/CPU-s 0.01 5.71% 10.93MiB/CPU-s 618.44KiB/CPU-s 7.98KiB/CPU-s 0.0 0.055242 10.93MiB/CPU-s 575.0KiB/CPU-s 7.42KiB/CPU-s 0.0 0.051358 False False
splunk_hec_to_splunk_hec_logs_acks -96.53B/CPU-s -0.00 1.14% 13.62MiB/CPU-s 366.0KiB/CPU-s 4.72KiB/CPU-s 0.0 0.026250 13.62MiB/CPU-s 360.4KiB/CPU-s 4.65KiB/CPU-s 0.0 0.025848 False False
http_to_http_json -40.55B/CPU-s -0.00 0.83% 13.62MiB/CPU-s 208.46KiB/CPU-s 2.69KiB/CPU-s 0.0 0.014941 13.62MiB/CPU-s 208.64KiB/CPU-s 2.69KiB/CPU-s 0.0 0.014954 False False
enterprise_http_to_http 185.31B/CPU-s 0.00 5.24% 13.62MiB/CPU-s 150.4KiB/CPU-s 1.94KiB/CPU-s 0.0 0.010780 13.62MiB/CPU-s 150.8KiB/CPU-s 1.95KiB/CPU-s 0.0 0.010808 False False
splunk_hec_to_splunk_hec_logs_noack -146.01B/CPU-s -0.00 2.12% 13.62MiB/CPU-s 257.19KiB/CPU-s 3.32KiB/CPU-s 0.0 0.018445 13.62MiB/CPU-s 325.94KiB/CPU-s 4.21KiB/CPU-s 0.0 0.023375 False False
fluent_elasticsearch -12.28KiB/CPU-s -0.03 93.36% 45.41MiB/CPU-s 30.26KiB/CPU-s 395.91B/CPU-s 0.0 0.000651 45.4MiB/CPU-s 522.87KiB/CPU-s 6.68KiB/CPU-s 0.0 0.011246 False False
file_to_blackhole -21.49KiB/CPU-s -0.04 64.61% 54.5MiB/CPU-s 1.16MiB/CPU-s 15.28KiB/CPU-s 0.0 0.021245 54.47MiB/CPU-s 1.32MiB/CPU-s 17.43KiB/CPU-s 0.0 0.024253 False False
http_to_http_noack -8.54KiB/CPU-s -0.06 90.00% 13.62MiB/CPU-s 223.85KiB/CPU-s 2.89KiB/CPU-s 0.0 0.016049 13.61MiB/CPU-s 334.02KiB/CPU-s 4.31KiB/CPU-s 0.0 0.023962 False False
datadog_agent_remap_datadog_logs -77.4KiB/CPU-s -0.24 100.00% 31.01MiB/CPU-s 607.82KiB/CPU-s 7.85KiB/CPU-s 0.0 0.019142 30.93MiB/CPU-s 1018.47KiB/CPU-s 13.14KiB/CPU-s 0.0 0.032153 False False
datadog_agent_remap_datadog_logs_acks -105.0KiB/CPU-s -0.34 100.00% 30.6MiB/CPU-s 1.01MiB/CPU-s 13.29KiB/CPU-s 0.0 0.032875 30.49MiB/CPU-s 698.04KiB/CPU-s 9.01KiB/CPU-s 0.0 0.022352 False False
http_pipelines_no_grok_blackhole -26.87KiB/CPU-s -0.43 100.00% 6.11MiB/CPU-s 26.01KiB/CPU-s 343.9B/CPU-s 0.0 0.004158 6.08MiB/CPU-s 172.47KiB/CPU-s 2.23KiB/CPU-s 0.0 0.027687 False False
syslog_log2metric_splunk_hec_metrics -58.38KiB/CPU-s -0.61 100.00% 9.31MiB/CPU-s 74.14KiB/CPU-s 980.09B/CPU-s 0.0 0.007775 9.25MiB/CPU-s 104.13KiB/CPU-s 1.34KiB/CPU-s 0.0 0.010988 False False
datadog_agent_remap_blackhole_acks -205.78KiB/CPU-s -0.66 100.00% 30.65MiB/CPU-s 821.47KiB/CPU-s 10.6KiB/CPU-s 0.0 0.026173 30.45MiB/CPU-s 722.98KiB/CPU-s 9.34KiB/CPU-s 0.0 0.023187 False False
http_text_to_http_json -295.04KiB/CPU-s -1.18 100.00% 24.45MiB/CPU-s 532.6KiB/CPU-s 6.88KiB/CPU-s 0.0 0.021273 24.16MiB/CPU-s 686.41KiB/CPU-s 8.87KiB/CPU-s 0.0 0.027743 False False

@github-actions
Copy link

Soak Test Results

Baseline: ecdc5ab
Comparison: a18e50f
Total Vector CPUs: 4

Explanation

A soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±8.87% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed.

Changes in throughput with confidence ≥ 90.00% and absolute Δ mean >= ±8.87%:

experiment Δ mean Δ mean % confidence
datadog_agent_remap_datadog_logs_acks 5.29MiB 9.48 100.00%
datadog_agent_remap_datadog_logs 5.03MiB 8.94 100.00%
Fine details of change detection per experiment.
experiment Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
datadog_agent_remap_datadog_logs_acks 5.29MiB 9.48 100.00% 55.81MiB 2.76MiB 57.72KiB 0 0.0494696 61.1MiB 4.27MiB 88.96KiB 0 0.0699346 False False
datadog_agent_remap_datadog_logs 5.03MiB 8.94 100.00% 56.25MiB 235.77KiB 4.83KiB 0 0.00409266 61.28MiB 4.07MiB 84.71KiB 0 0.0663727 False False
syslog_regex_logs2metric_ddmetrics 382.53KiB 5.41 100.00% 6.91MiB 963.19KiB 19.61KiB 0 0.13616 7.28MiB 995.15KiB 20.27KiB 0 0.13346 False False
socket_to_socket_blackhole 383.08KiB 1.62 100.00% 23.05MiB 1.08MiB 22.42KiB 0 0.0467192 23.43MiB 1.06MiB 22.06KiB 0 0.0452255 False False
syslog_log2metric_splunk_hec_metrics 146.21KiB 0.88 100.00% 16.14MiB 501.27KiB 10.22KiB 0 0.0303291 16.28MiB 633.85KiB 12.91KiB 0 0.0380146 False False
syslog_splunk_hec_logs 94.86KiB 0.58 100.00% 15.85MiB 765.88KiB 15.57KiB 0 0.0471864 15.94MiB 626.96KiB 12.78KiB 0 0.0384029 False False
http_pipelines_blackhole 25.72KiB 0.56 99.80% 4.45MiB 214.13KiB 4.38KiB 0 0.047005 4.47MiB 348.14KiB 7.09KiB 0 0.0759934 False False
datadog_agent_remap_blackhole_acks 282.92KiB 0.5 99.54% 55.14MiB 3.77MiB 78.57KiB 0 0.0684064 55.42MiB 2.95MiB 61.53KiB 0 0.0531451 False False
datadog_agent_remap_blackhole 258.46KiB 0.46 99.47% 55.3MiB 3.77MiB 78.52KiB 0 0.0681691 55.55MiB 2.36MiB 49.26KiB 0 0.0424711 False False
splunk_hec_to_splunk_hec_logs_acks 14.64KiB 0.06 46.91% 23.75MiB 843.69KiB 17.15KiB 0 0.0346844 23.76MiB 779.51KiB 15.86KiB 0 0.0320267 False False
splunk_hec_to_splunk_hec_logs_noack 11.67KiB 0.05 71.29% 23.83MiB 416.8KiB 8.51KiB 0 0.0170774 23.84MiB 338.47KiB 6.91KiB 0 0.0138615 False False
splunk_hec_indexer_ack_blackhole 4.98KiB 0.02 17.65% 23.76MiB 786.37KiB 16.0KiB 0 0.0323091 23.77MiB 766.3KiB 15.59KiB 0 0.0314781 False False
file_to_blackhole 109.22B 0 0.12% 95.31MiB 2.34MiB 48.43KiB 0 0.0245096 95.31MiB 2.31MiB 48.07KiB 0 0.0242716 False False
http_pipelines_blackhole_acks -194.29B -0.01 1.64% 3.33MiB 347.7KiB 7.07KiB 0 0.101953 3.33MiB 291.35KiB 5.93KiB 0 0.0854363 False False
enterprise_http_to_http -2.46KiB -0.01 25.84% 23.84MiB 256.27KiB 5.23KiB 0 0.0104935 23.84MiB 260.64KiB 5.33KiB 0 0.0106738 False False
otlp_http_to_blackhole -4.45KiB -0.18 36.47% 2.47MiB 327.81KiB 6.67KiB 0 0.129703 2.46MiB 324.8KiB 6.61KiB 0 0.128741 False False
http_to_http_json -51.6KiB -0.21 99.99% 23.85MiB 390.6KiB 7.97KiB 0 0.0159876 23.8MiB 527.7KiB 10.75KiB 0 0.0216449 False False
http_to_http_noack -82.43KiB -0.34 99.97% 23.83MiB 402.82KiB 8.24KiB 0 0.0165024 23.75MiB 1.03MiB 21.43KiB 0 0.04326 False False
fluent_elasticsearch -545.84KiB -0.67 100.00% 79.47MiB 51.69KiB 1.01KiB 0 0.000635101 78.94MiB 5.21MiB 103.02KiB 0 0.0659661 False False
http_pipelines_no_grok_blackhole -137.35KiB -1.26 100.00% 10.66MiB 63.71KiB 1.3KiB 0 0.00583685 10.52MiB 953.86KiB 19.41KiB 0 0.0885059 False False
otlp_grpc_to_blackhole -22.25KiB -1.31 100.00% 1.66MiB 58.47KiB 1.2KiB 0 0.0344406 1.64MiB 139.8KiB 2.85KiB 0 0.083447 False False
syslog_humio_logs -245.65KiB -1.52 100.00% 15.77MiB 211.34KiB 4.31KiB 0 0.0130837 15.53MiB 176.09KiB 3.61KiB 0 0.0110698 False False
splunk_hec_route_s3 -370.42KiB -1.65 100.00% 21.95MiB 1.19MiB 24.85KiB 0 0.0543386 21.58MiB 1.24MiB 25.76KiB 0 0.0572863 False False
http_to_http_acks -160.5KiB -1.68 77.92% 9.33MiB 4.39MiB 91.86KiB 0 0.470881 9.17MiB 4.46MiB 93.49KiB 0 0.485715 True True
syslog_loki -381.59KiB -2.39 100.00% 15.57MiB 122.68KiB 2.51KiB 0 0.00769431 15.19MiB 777.56KiB 15.81KiB 0 0.0499629 False False
syslog_log2metric_humio_metrics -236.64KiB -2.41 100.00% 9.58MiB 182.9KiB 3.73KiB 0 0.0186384 9.35MiB 380.14KiB 7.74KiB 0 0.0396951 False False
http_text_to_http_json -1.64MiB -4 100.00% 41.0MiB 723.61KiB 14.77KiB 0 0.0172305 39.36MiB 668.78KiB 13.65KiB 0 0.0165881 False False

Copy link
Contributor

@tobz tobz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good find!

The change is obviously simple, but CI was having some issues so I'll try and shepherd that along until it all passes so that we can get this merged. 👍🏻

@tobz tobz added the ci-condition: integration tests enable Run integration tests on this PR label Nov 16, 2022
@tobz
Copy link
Contributor

tobz commented Nov 16, 2022

Woops, realized the integration tests never ran for this PR.

Just kicked those off, and that should hopefully be done in 20-30 minutes.

@archoversight
Copy link
Contributor Author

@tobz thanks, it would be great appreciated to get this fix out and released :-)

@bruceg bruceg merged commit 787347b into vectordotdev:master Nov 16, 2022
@archoversight archoversight deleted the bugfix/azure-connection-string-sensitive branch November 16, 2022 17:50
@spencergilbert spencergilbert modified the milestone: Vector 0.25.2 Nov 23, 2022
spencergilbert pushed a commit that referenced this pull request Nov 23, 2022
… ConnectionString (#15225)

Retrieve .inner() from SensitiveString for Azure ConnectionString

azure_common::config::build_client requires a String not a SensitiveString, in the conversion using
to_string() we were accidentally converting to the sensitive value to **REDACTED**.

This fixes it by using the same method that seems to have been used in the PR that introduced this
change: #14305 to call `.inner()` to retrieve the value
from inside the struct and then converts that to a String.
@jszwedko
Copy link
Member

Just a heads up that this PR doesn't seem to be breaking, but was marked as such with the ! in the semantic commit message.

jszwedko pushed a commit that referenced this pull request Nov 28, 2022
…ConnectionString (#15225)

Retrieve .inner() from SensitiveString for Azure ConnectionString

azure_common::config::build_client requires a String not a SensitiveString, in the conversion using
to_string() we were accidentally converting to the sensitive value to **REDACTED**.

This fixes it by using the same method that seems to have been used in the PR that introduced this
change: #14305 to call `.inner()` to retrieve the value
from inside the struct and then converts that to a String.

Signed-off-by: Jesse Szwedko <jesse.szwedko@datadoghq.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ci-condition: integration tests enable Run integration tests on this PR domain: sinks Anything related to the Vector's sinks
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Azure Blob sink cannot be configured with connection_string
5 participants