Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Serverless Mini Agent] Bump datadog-serverless-trace-mini-agent version to 0.7.2 #635

Merged

Conversation

duncanpharvey
Copy link
Contributor

@duncanpharvey duncanpharvey commented Sep 17, 2024

What does this PR do?

Bumps Serverless Mini Agent Version to 0.7.2.

Motivation

Upcoming release of version 0.7.2.

Additional Notes

How to test the change?

See https://datadoghq.atlassian.net/wiki/spaces/SLS/pages/2977497119/Serverless+Mini+Agent#Testing

@codecov-commenter
Copy link

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 73.04%. Comparing base (19cc332) to head (19ea324).

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #635      +/-   ##
==========================================
+ Coverage   73.01%   73.04%   +0.03%     
==========================================
  Files         252      252              
  Lines       36093    36093              
==========================================
+ Hits        26352    26365      +13     
+ Misses       9741     9728      -13     
Components Coverage Δ
crashtracker 20.48% <ø> (-0.06%) ⬇️
datadog-alloc 98.73% <ø> (ø)
data-pipeline 90.12% <ø> (ø)
data-pipeline-ffi 0.00% <ø> (ø)
ddcommon 83.08% <ø> (ø)
ddcommon-ffi 69.52% <ø> (ø)
ddtelemetry 59.10% <ø> (ø)
ipc 83.63% <ø> (ø)
profiling 84.26% <ø> (ø)
profiling-ffi 77.42% <ø> (ø)
serverless 0.00% <ø> (ø)
sidecar 40.12% <ø> (ø)
sidecar-ffi 0.00% <ø> (ø)
spawn-worker 50.36% <ø> (ø)
tinybytes 91.66% <ø> (ø)
trace-mini-agent 72.49% <ø> (ø)
trace-normalization 98.25% <ø> (ø)
trace-obfuscation 95.73% <ø> (ø)
trace-protobuf 77.67% <ø> (ø)
trace-utils 92.88% <ø> (ø)

@pr-commenter
Copy link

pr-commenter bot commented Sep 17, 2024

Benchmarks

Comparison

Benchmark execution time: 2024-09-17 01:01:50

Comparing candidate commit 19ea324 in PR branch duncan-harvey/serverless-mini-agent-bump-version with baseline commit 19cc332 in branch main.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 51 metrics, 2 unstable metrics.

Candidate

Candidate benchmark details

Group 1

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 19ea324 1726530791 duncan-harvey/serverless-mini-agent-bump-version
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
two way interface execution_time 18.698µs 24.739µs ± 10.462µs 20.108µs ± 0.473µs 21.605µs 37.214µs 48.759µs 104.884µs 421.61% 3.988 23.782 42.18% 0.740µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
two way interface execution_time [23.289µs; 26.189µs] or [-5.861%; +5.861%] None None None

Group 2

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 19ea324 1726530791 duncan-harvey/serverless-mini-agent-bump-version
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
tags/replace_trace_tags execution_time 2.639µs 2.704µs ± 0.024µs 2.713µs ± 0.007µs 2.718µs 2.736µs 2.742µs 2.758µs 1.65% -0.971 0.355 0.87% 0.002µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
tags/replace_trace_tags execution_time [2.701µs; 2.707µs] or [-0.121%; +0.121%] None None None

Group 3

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 19ea324 1726530791 duncan-harvey/serverless-mini-agent-bump-version
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_trace/test_trace execution_time 292.630ns 305.128ns ± 13.096ns 298.867ns ± 5.027ns 311.288ns 336.284ns 341.753ns 344.999ns 15.44% 1.312 0.747 4.28% 0.926ns 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_trace/test_trace execution_time [303.313ns; 306.943ns] or [-0.595%; +0.595%] None None None

Group 4

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 19ea324 1726530791 duncan-harvey/serverless-mini-agent-bump-version
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
credit_card/is_card_number/ execution_time 1.208µs 1.209µs ± 0.001µs 1.208µs ± 0.000µs 1.209µs 1.211µs 1.212µs 1.218µs 0.84% 3.277 17.827 0.10% 0.000µs 1 200
credit_card/is_card_number/ throughput 820747904.048op/s 827353278.893op/s ± 851278.927op/s 827620276.901op/s ± 264018.567op/s 827871914.536op/s 827977740.011op/s 828041452.912op/s 828099758.204op/s 0.06% -3.249 17.506 0.10% 60194.510op/s 1 200
credit_card/is_card_number/ 3782-8224-6310-005 execution_time 108.994µs 111.131µs ± 0.668µs 111.136µs ± 0.335µs 111.446µs 111.865µs 113.795µs 114.231µs 2.78% 1.151 5.176 0.60% 0.047µs 1 200
credit_card/is_card_number/ 3782-8224-6310-005 throughput 8754218.317op/s 8998728.008op/s ± 53697.425op/s 8997991.692op/s ± 27019.035op/s 9026866.477op/s 9076284.364op/s 9102141.071op/s 9174778.615op/s 1.96% -1.047 4.821 0.60% 3796.981op/s 1 200
credit_card/is_card_number/ 378282246310005 execution_time 101.401µs 103.538µs ± 0.580µs 103.607µs ± 0.281µs 103.855µs 104.206µs 105.223µs 105.930µs 2.24% -0.090 2.788 0.56% 0.041µs 1 200
credit_card/is_card_number/ 378282246310005 throughput 9440190.844op/s 9658552.438op/s ± 54110.250op/s 9651845.496op/s ± 26165.799op/s 9681183.895op/s 9750985.886op/s 9814102.547op/s 9861863.756op/s 2.18% 0.169 2.718 0.56% 3826.173op/s 1 200
credit_card/is_card_number/37828224631 execution_time 1.208µs 1.210µs ± 0.002µs 1.209µs ± 0.001µs 1.211µs 1.212µs 1.212µs 1.215µs 0.55% 0.511 -0.898 0.13% 0.000µs 1 200
credit_card/is_card_number/37828224631 throughput 822832742.571op/s 826769184.617op/s ± 1087692.056op/s 827366720.749op/s ± 546390.896op/s 827786632.138op/s 827949993.164op/s 827979199.728op/s 827994923.047op/s 0.08% -0.508 -0.911 0.13% 76911.443op/s 1 200
credit_card/is_card_number/378282246310005 execution_time 98.697µs 100.523µs ± 0.943µs 100.455µs ± 0.484µs 100.897µs 102.336µs 103.525µs 104.543µs 4.07% 1.027 2.299 0.94% 0.067µs 1 200
credit_card/is_card_number/378282246310005 throughput 9565455.696op/s 9948838.552op/s ± 92473.016op/s 9954697.693op/s ± 48207.150op/s 10006700.240op/s 10082776.953op/s 10121174.610op/s 10132058.207op/s 1.78% -0.938 2.014 0.93% 6538.830op/s 1 200
credit_card/is_card_number/37828224631000521389798 execution_time 93.687µs 94.637µs ± 0.917µs 94.451µs ± 0.210µs 94.722µs 95.383µs 98.134µs 103.959µs 10.07% 6.397 55.198 0.97% 0.065µs 1 200
credit_card/is_card_number/37828224631000521389798 throughput 9619159.333op/s 10567614.604op/s ± 96626.959op/s 10587512.564op/s ± 23552.806op/s 10607747.103op/s 10637599.925op/s 10669625.001op/s 10673793.270op/s 0.81% -5.954 48.586 0.91% 6832.558op/s 1 200
credit_card/is_card_number/x371413321323331 execution_time 20.212µs 21.317µs ± 0.408µs 21.320µs ± 0.274µs 21.585µs 22.061µs 22.286µs 22.361µs 4.88% 0.048 0.003 1.91% 0.029µs 1 200
credit_card/is_card_number/x371413321323331 throughput 44720449.311op/s 46928159.531op/s ± 899194.361op/s 46904164.491op/s ± 596089.955op/s 47556931.979op/s 48472936.928op/s 48879289.692op/s 49475276.029op/s 5.48% 0.066 -0.015 1.91% 63582.643op/s 1 200
credit_card/is_card_number_no_luhn/ execution_time 1.211µs 1.211µs ± 0.000µs 1.211µs ± 0.000µs 1.212µs 1.212µs 1.213µs 1.213µs 0.13% 0.758 0.083 0.04% 0.000µs 1 200
credit_card/is_card_number_no_luhn/ throughput 824463971.094op/s 825520398.584op/s ± 320204.782op/s 825542294.631op/s ± 224303.781op/s 825767878.610op/s 825919907.688op/s 825977279.996op/s 826017221.298op/s 0.06% -0.756 0.078 0.04% 22641.897op/s 1 200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 execution_time 93.466µs 96.114µs ± 0.501µs 96.162µs ± 0.270µs 96.427µs 96.790µs 97.156µs 97.218µs 1.10% -1.059 3.425 0.52% 0.035µs 1 200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 throughput 10286139.635op/s 10404606.174op/s ± 54532.785op/s 10399156.134op/s ± 29222.015op/s 10430968.914op/s 10504356.241op/s 10550082.950op/s 10699026.008op/s 2.88% 1.128 3.760 0.52% 3856.050op/s 1 200
credit_card/is_card_number_no_luhn/ 378282246310005 execution_time 85.074µs 87.226µs ± 0.654µs 87.214µs ± 0.396µs 87.593µs 88.279µs 89.158µs 89.408µs 2.52% 0.130 1.223 0.75% 0.046µs 1 200
credit_card/is_card_number_no_luhn/ 378282246310005 throughput 11184625.383op/s 11465176.646op/s ± 85863.619op/s 11466012.761op/s ± 52027.843op/s 11518460.248op/s 11613261.508op/s 11669862.090op/s 11754407.067op/s 2.52% -0.058 1.201 0.75% 6071.475op/s 1 200
credit_card/is_card_number_no_luhn/37828224631 execution_time 1.208µs 1.209µs ± 0.001µs 1.208µs ± 0.000µs 1.209µs 1.210µs 1.212µs 1.217µs 0.73% 5.038 36.426 0.08% 0.000µs 1 200
credit_card/is_card_number_no_luhn/37828224631 throughput 821479626.620op/s 827438476.634op/s ± 653387.895op/s 827514372.714op/s ± 213152.409op/s 827783111.135op/s 827914863.089op/s 827993711.489op/s 828063587.515op/s 0.07% -5.008 36.006 0.08% 46201.501op/s 1 200
credit_card/is_card_number_no_luhn/378282246310005 execution_time 81.022µs 83.032µs ± 0.562µs 83.107µs ± 0.280µs 83.341µs 83.860µs 84.320µs 84.907µs 2.17% -0.252 1.110 0.68% 0.040µs 1 200
credit_card/is_card_number_no_luhn/378282246310005 throughput 11777645.834op/s 12044108.391op/s ± 81716.715op/s 12032651.446op/s ± 40432.697op/s 12095482.454op/s 12179616.089op/s 12259026.606op/s 12342251.239op/s 2.57% 0.314 1.141 0.68% 5778.244op/s 1 200
credit_card/is_card_number_no_luhn/37828224631000521389798 execution_time 93.930µs 95.445µs ± 0.620µs 95.343µs ± 0.336µs 95.749µs 96.443µs 97.126µs 98.267µs 3.07% 0.895 2.321 0.65% 0.044µs 1 200
credit_card/is_card_number_no_luhn/37828224631000521389798 throughput 10176343.432op/s 10477707.297op/s ± 67723.138op/s 10488420.338op/s ± 36987.495op/s 10520051.484op/s 10569773.529op/s 10619918.334op/s 10646245.343op/s 1.50% -0.828 2.105 0.64% 4788.749op/s 1 200
credit_card/is_card_number_no_luhn/x371413321323331 execution_time 20.253µs 21.104µs ± 0.348µs 21.099µs ± 0.229µs 21.332µs 21.696µs 21.933µs 21.973µs 4.14% 0.113 -0.154 1.65% 0.025µs 1 200
credit_card/is_card_number_no_luhn/x371413321323331 throughput 45511266.155op/s 47398116.736op/s ± 781434.018op/s 47395820.339op/s ± 516994.260op/s 47902382.875op/s 48752312.887op/s 48991013.926op/s 49375555.801op/s 4.18% -0.022 -0.176 1.64% 55255.729op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
credit_card/is_card_number/ execution_time [1.209µs; 1.209µs] or [-0.014%; +0.014%] None None None
credit_card/is_card_number/ throughput [827235299.820op/s; 827471257.965op/s] or [-0.014%; +0.014%] None None None
credit_card/is_card_number/ 3782-8224-6310-005 execution_time [111.038µs; 111.223µs] or [-0.083%; +0.083%] None None None
credit_card/is_card_number/ 3782-8224-6310-005 throughput [8991286.061op/s; 9006169.955op/s] or [-0.083%; +0.083%] None None None
credit_card/is_card_number/ 378282246310005 execution_time [103.458µs; 103.619µs] or [-0.078%; +0.078%] None None None
credit_card/is_card_number/ 378282246310005 throughput [9651053.277op/s; 9666051.598op/s] or [-0.078%; +0.078%] None None None
credit_card/is_card_number/37828224631 execution_time [1.209µs; 1.210µs] or [-0.018%; +0.018%] None None None
credit_card/is_card_number/37828224631 throughput [826618440.959op/s; 826919928.275op/s] or [-0.018%; +0.018%] None None None
credit_card/is_card_number/378282246310005 execution_time [100.392µs; 100.654µs] or [-0.130%; +0.130%] None None None
credit_card/is_card_number/378282246310005 throughput [9936022.682op/s; 9961654.423op/s] or [-0.129%; +0.129%] None None None
credit_card/is_card_number/37828224631000521389798 execution_time [94.510µs; 94.764µs] or [-0.134%; +0.134%] None None None
credit_card/is_card_number/37828224631000521389798 throughput [10554223.036op/s; 10581006.171op/s] or [-0.127%; +0.127%] None None None
credit_card/is_card_number/x371413321323331 execution_time [21.260µs; 21.374µs] or [-0.266%; +0.266%] None None None
credit_card/is_card_number/x371413321323331 throughput [46803539.840op/s; 47052779.221op/s] or [-0.266%; +0.266%] None None None
credit_card/is_card_number_no_luhn/ execution_time [1.211µs; 1.211µs] or [-0.005%; +0.005%] None None None
credit_card/is_card_number_no_luhn/ throughput [825476021.280op/s; 825564775.887op/s] or [-0.005%; +0.005%] None None None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 execution_time [96.044µs; 96.183µs] or [-0.072%; +0.072%] None None None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 throughput [10397048.454op/s; 10412163.893op/s] or [-0.073%; +0.073%] None None None
credit_card/is_card_number_no_luhn/ 378282246310005 execution_time [87.135µs; 87.316µs] or [-0.104%; +0.104%] None None None
credit_card/is_card_number_no_luhn/ 378282246310005 throughput [11453276.775op/s; 11477076.518op/s] or [-0.104%; +0.104%] None None None
credit_card/is_card_number_no_luhn/37828224631 execution_time [1.208µs; 1.209µs] or [-0.011%; +0.011%] None None None
credit_card/is_card_number_no_luhn/37828224631 throughput [827347923.356op/s; 827529029.913op/s] or [-0.011%; +0.011%] None None None
credit_card/is_card_number_no_luhn/378282246310005 execution_time [82.954µs; 83.110µs] or [-0.094%; +0.094%] None None None
credit_card/is_card_number_no_luhn/378282246310005 throughput [12032783.240op/s; 12055433.542op/s] or [-0.094%; +0.094%] None None None
credit_card/is_card_number_no_luhn/37828224631000521389798 execution_time [95.359µs; 95.531µs] or [-0.090%; +0.090%] None None None
credit_card/is_card_number_no_luhn/37828224631000521389798 throughput [10468321.521op/s; 10487093.073op/s] or [-0.090%; +0.090%] None None None
credit_card/is_card_number_no_luhn/x371413321323331 execution_time [21.055µs; 21.152µs] or [-0.229%; +0.229%] None None None
credit_card/is_card_number_no_luhn/x371413321323331 throughput [47289817.496op/s; 47506415.975op/s] or [-0.228%; +0.228%] None None None

Group 5

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 19ea324 1726530791 duncan-harvey/serverless-mini-agent-bump-version
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
benching deserializing traces from msgpack to their internal representation execution_time 1.200µs 1.373µs ± 0.052µs 1.385µs ± 0.031µs 1.413µs 1.442µs 1.461µs 1.462µs 5.53% -0.849 0.929 3.78% 0.004µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
benching deserializing traces from msgpack to their internal representation execution_time [1.365µs; 1.380µs] or [-0.525%; +0.525%] None None None

Group 6

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 19ea324 1726530791 duncan-harvey/serverless-mini-agent-bump-version
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... execution_time 269.772µs 273.586µs ± 1.739µs 273.558µs ± 1.317µs 274.924µs 276.333µs 277.207µs 279.509µs 2.18% 0.187 -0.178 0.63% 0.123µs 1 200
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... throughput 3577701.445op/s 3655308.015op/s ± 23215.423op/s 3655534.682op/s ± 17600.133op/s 3672182.388op/s 3693222.076op/s 3702212.322op/s 3706829.655op/s 1.40% -0.154 -0.222 0.63% 1641.578op/s 1 200
normalization/normalize_name/normalize_name/bad-name execution_time 25.769µs 25.805µs ± 0.033µs 25.811µs ± 0.033µs 25.821µs 25.869µs 25.903µs 25.914µs 0.40% 0.877 0.307 0.13% 0.002µs 1 200
normalization/normalize_name/normalize_name/bad-name throughput 38589797.731op/s 38752662.860op/s ± 49194.546op/s 38743475.973op/s ± 49216.985op/s 38797891.262op/s 38803449.510op/s 38804797.875op/s 38806331.875op/s 0.16% -0.871 0.289 0.13% 3478.580op/s 1 200
normalization/normalize_name/normalize_name/good execution_time 15.305µs 15.332µs ± 0.023µs 15.324µs ± 0.017µs 15.351µs 15.365µs 15.393µs 15.439µs 0.75% 0.781 0.844 0.15% 0.002µs 1 200
normalization/normalize_name/normalize_name/good throughput 64771923.238op/s 65221166.944op/s ± 98234.468op/s 65256138.746op/s ± 71622.536op/s 65313496.483op/s 65327428.546op/s 65333627.387op/s 65337365.055op/s 0.12% -0.771 0.793 0.15% 6946.226op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... execution_time [273.345µs; 273.827µs] or [-0.088%; +0.088%] None None None
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... throughput [3652090.580op/s; 3658525.449op/s] or [-0.088%; +0.088%] None None None
normalization/normalize_name/normalize_name/bad-name execution_time [25.800µs; 25.809µs] or [-0.018%; +0.018%] None None None
normalization/normalize_name/normalize_name/bad-name throughput [38745844.969op/s; 38759480.751op/s] or [-0.018%; +0.018%] None None None
normalization/normalize_name/normalize_name/good execution_time [15.329µs; 15.336µs] or [-0.021%; +0.021%] None None None
normalization/normalize_name/normalize_name/good throughput [65207552.591op/s; 65234781.296op/s] or [-0.021%; +0.021%] None None None

Group 7

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 19ea324 1726530791 duncan-harvey/serverless-mini-agent-bump-version
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
write only interface execution_time 1.378µs 3.185µs ± 1.568µs 3.029µs ± 0.023µs 3.050µs 3.137µs 13.780µs 18.239µs 502.09% 8.039 65.619 49.11% 0.111µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
write only interface execution_time [2.968µs; 3.403µs] or [-6.823%; +6.823%] None None None

Group 8

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 19ea324 1726530791 duncan-harvey/serverless-mini-agent-bump-version
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... execution_time 506.667µs 507.485µs ± 0.347µs 507.452µs ± 0.198µs 507.660µs 507.971µs 508.317µs 509.456µs 0.39% 1.624 8.069 0.07% 0.025µs 1 200
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... throughput 1962878.957op/s 1970500.679op/s ± 1344.758op/s 1970630.191op/s ± 769.820op/s 1971372.254op/s 1972358.246op/s 1973136.308op/s 1973681.995op/s 0.15% -1.608 7.978 0.07% 95.089op/s 1 200
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて execution_time 464.970µs 465.882µs ± 0.465µs 465.804µs ± 0.267µs 466.133µs 466.694µs 467.205µs 467.753µs 0.42% 0.823 1.293 0.10% 0.033µs 1 200
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて throughput 2137882.628op/s 2146468.789op/s ± 2138.398op/s 2146824.612op/s ± 1230.687op/s 2147795.579op/s 2149506.613op/s 2150504.473op/s 2150674.684op/s 0.18% -0.815 1.270 0.10% 151.208op/s 1 200
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters execution_time 179.730µs 180.330µs ± 0.375µs 180.244µs ± 0.221µs 180.525µs 181.021µs 181.528µs 181.781µs 0.85% 1.121 1.216 0.21% 0.027µs 1 200
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters throughput 5501110.605op/s 5545423.258op/s ± 11512.377op/s 5548032.963op/s ± 6797.378op/s 5554067.528op/s 5559043.856op/s 5561918.174op/s 5563915.338op/s 0.29% -1.109 1.175 0.21% 814.048op/s 1 200
normalization/normalize_service/normalize_service/[empty string] execution_time 43.368µs 43.514µs ± 0.072µs 43.511µs ± 0.043µs 43.551µs 43.652µs 43.694µs 43.780µs 0.62% 0.640 0.688 0.16% 0.005µs 1 200
normalization/normalize_service/normalize_service/[empty string] throughput 22841246.627op/s 22980925.118op/s ± 37815.926op/s 22982697.727op/s ± 22858.098op/s 23006545.211op/s 23036683.185op/s 23051289.004op/s 23058282.954op/s 0.33% -0.629 0.664 0.16% 2673.990op/s 1 200
normalization/normalize_service/normalize_service/test_ASCII execution_time 49.780µs 49.976µs ± 0.088µs 49.980µs ± 0.063µs 50.042µs 50.113µs 50.165µs 50.297µs 0.63% 0.018 -0.000 0.18% 0.006µs 1 200
normalization/normalize_service/normalize_service/test_ASCII throughput 19881970.451op/s 20009758.252op/s ± 35414.836op/s 20007849.591op/s ± 25016.239op/s 20033496.705op/s 20069958.744op/s 20084230.982op/s 20088449.912op/s 0.40% -0.008 -0.016 0.18% 2504.207op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... execution_time [507.437µs; 507.534µs] or [-0.009%; +0.009%] None None None
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... throughput [1970314.309op/s; 1970687.050op/s] or [-0.009%; +0.009%] None None None
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて execution_time [465.818µs; 465.946µs] or [-0.014%; +0.014%] None None None
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて throughput [2146172.428op/s; 2146765.151op/s] or [-0.014%; +0.014%] None None None
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters execution_time [180.278µs; 180.382µs] or [-0.029%; +0.029%] None None None
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters throughput [5543827.753op/s; 5547018.762op/s] or [-0.029%; +0.029%] None None None
normalization/normalize_service/normalize_service/[empty string] execution_time [43.505µs; 43.524µs] or [-0.023%; +0.023%] None None None
normalization/normalize_service/normalize_service/[empty string] throughput [22975684.194op/s; 22986166.041op/s] or [-0.023%; +0.023%] None None None
normalization/normalize_service/normalize_service/test_ASCII execution_time [49.964µs; 49.988µs] or [-0.025%; +0.025%] None None None
normalization/normalize_service/normalize_service/test_ASCII throughput [20004850.096op/s; 20014666.408op/s] or [-0.025%; +0.025%] None None None

Group 9

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 19ea324 1726530791 duncan-harvey/serverless-mini-agent-bump-version
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
sql/obfuscate_sql_string execution_time 74.636µs 74.768µs ± 0.147µs 74.749µs ± 0.042µs 74.792µs 74.887µs 75.102µs 76.452µs 2.28% 7.949 85.279 0.20% 0.010µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
sql/obfuscate_sql_string execution_time [74.748µs; 74.789µs] or [-0.027%; +0.027%] None None None

Group 10

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 19ea324 1726530791 duncan-harvey/serverless-mini-agent-bump-version
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
concentrator/add_spans_to_concentrator execution_time 8.945ms 8.971ms ± 0.024ms 8.968ms ± 0.008ms 8.976ms 8.990ms 9.038ms 9.177ms 2.34% 6.021 46.354 0.27% 0.002ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
concentrator/add_spans_to_concentrator execution_time [8.968ms; 8.975ms] or [-0.037%; +0.037%] None None None

Group 11

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 19ea324 1726530791 duncan-harvey/serverless-mini-agent-bump-version
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
benching string interning on wordpress profile execution_time 139.405µs 141.290µs ± 0.420µs 141.276µs ± 0.191µs 141.453µs 142.006µs 142.755µs 143.311µs 1.44% 0.690 6.963 0.30% 0.030µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
benching string interning on wordpress profile execution_time [141.232µs; 141.348µs] or [-0.041%; +0.041%] None None None

Group 12

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 19ea324 1726530791 duncan-harvey/serverless-mini-agent-bump-version
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
redis/obfuscate_redis_string execution_time 38.189µs 38.652µs ± 0.857µs 38.262µs ± 0.028µs 38.315µs 40.417µs 40.442µs 42.662µs 11.50% 1.903 2.474 2.21% 0.061µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
redis/obfuscate_redis_string execution_time [38.533µs; 38.771µs] or [-0.307%; +0.307%] None None None

Baseline

Omitted due to size.

@duncanpharvey duncanpharvey merged commit 89f48d4 into main Sep 17, 2024
66 checks passed
@duncanpharvey duncanpharvey deleted the duncan-harvey/serverless-mini-agent-bump-version branch September 17, 2024 15:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants