Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improve miri job runtime #601

Merged
merged 1 commit into from
Aug 29, 2024
Merged

Improve miri job runtime #601

merged 1 commit into from
Aug 29, 2024

Conversation

VianneyRuhlmann
Copy link
Contributor

@VianneyRuhlmann VianneyRuhlmann commented Aug 28, 2024 •
edited
Loading

What does this PR do?

Improve the miri job runtime by skipping some slow tests.

@codecov-commenter
Copy link

codecov-commenter commented Aug 28, 2024 •
edited
Loading

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 72.81%. Comparing base (24bdc13) to head (9139e27).

Additional details and impacted files 
@@            Coverage Diff             @@
##             main     #601      +/-   ##
==========================================
+ Coverage   72.80%   72.81%   +0.01%     
==========================================
  Files         243      243              
  Lines       34681    34664      -17     
==========================================
- Hits        25249    25241       -8     
+ Misses       9432     9423       -9
Components Coverage Δ
crashtracker 20.38% <ø> (-0.06%) ⬇️
datadog-alloc 98.73% <ø> (ø)
data-pipeline 90.12% <ø> (ø)
data-pipeline-ffi 0.00% <ø> (ø)
ddcommon 82.11% <ø> (ø)
ddcommon-ffi 68.11% <ø> (ø)
ddtelemetry 59.02% <ø> (ø)
ipc 84.00% <ø> (ø)
profiling 84.26% <ø> (ø)
profiling-ffi 77.42% <ø> (ø)
serverless 0.00% <ø> (ø)
sidecar 40.18% <ø> (ø)
sidecar-ffi 0.00% <ø> (ø)
spawn-worker 50.18% <ø> (ø)
tinybytes 91.66% <ø> (ø)
trace-mini-agent 70.88% <ø> (ø)
trace-normalization 98.25% <ø> (ø)
trace-obfuscation 95.73% <ø> (ø)
trace-protobuf 77.67% <ø> (ø)
trace-utils 93.42% <ø> (+0.41%) ⬆️

@pr-commenter
Copy link

pr-commenter bot commented Aug 28, 2024 •
edited
Loading

Benchmarks

Comparison

Benchmark execution time: 2024-08-29 09:07:06

Comparing candidate commit 67c55d9 in PR branch vianney/ddcommon/ignore-slow-tests-in-miri with baseline commit 24bdc13 in branch main.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 50 metrics, 2 unstable metrics.

Candidate

Candidate benchmark details

Group 1

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 67c55d9 1724847528 vianney/ddcommon/ignore-slow-tests-in-miri
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
benching string interning on wordpress profile execution_time 137.483µs 137.892µs ± 0.218µs 137.853µs ± 0.120µs 138.004µs 138.233µs 138.572µs 139.226µs 1.00% 1.825 7.323 0.16% 0.015µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
benching string interning on wordpress profile execution_time [137.862µs; 137.922µs] or [-0.022%; +0.022%] None None None

Group 2

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 67c55d9 1724847528 vianney/ddcommon/ignore-slow-tests-in-miri
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... execution_time 311.461µs 313.674µs ± 0.805µs 313.739µs ± 0.574µs 314.284µs 314.881µs 315.246µs 315.476µs 0.55% -0.185 -0.544 0.26% 0.057µs 1 200
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... throughput 3169810.529op/s 3188046.689op/s ± 8181.448op/s 3187364.503op/s ± 5847.221op/s 3193839.333op/s 3201963.937op/s 3205538.673op/s 3210678.024op/s 0.73% 0.196 -0.538 0.26% 578.516op/s 1 200
normalization/normalize_name/normalize_name/bad-name execution_time 27.945µs 28.004µs ± 0.037µs 27.997µs ± 0.025µs 28.027µs 28.076µs 28.092µs 28.114µs 0.42% 0.658 -0.278 0.13% 0.003µs 1 200
normalization/normalize_name/normalize_name/bad-name throughput 35569430.674op/s 35709612.082op/s ± 47217.324op/s 35718370.267op/s ± 31309.103op/s 35747909.865op/s 35771706.555op/s 35780898.147op/s 35784896.984op/s 0.19% -0.653 -0.287 0.13% 3338.769op/s 1 200
normalization/normalize_name/normalize_name/good execution_time 16.720µs 16.752µs ± 0.020µs 16.745µs ± 0.018µs 16.770µs 16.784µs 16.793µs 16.796µs 0.30% 0.222 -1.382 0.12% 0.001µs 1 200
normalization/normalize_name/normalize_name/good throughput 59537114.075op/s 59695847.677op/s ± 72620.647op/s 59717989.390op/s ± 63768.926op/s 59760542.545op/s 59789254.027op/s 59796397.436op/s 59807581.777op/s 0.15% -0.220 -1.385 0.12% 5135.055op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... execution_time [313.562µs; 313.785µs] or [-0.036%; +0.036%] None None None
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... throughput [3186912.819op/s; 3189180.559op/s] or [-0.036%; +0.036%] None None None
normalization/normalize_name/normalize_name/bad-name execution_time [27.999µs; 28.009µs] or [-0.018%; +0.018%] None None None
normalization/normalize_name/normalize_name/bad-name throughput [35703068.215op/s; 35716155.949op/s] or [-0.018%; +0.018%] None None None
normalization/normalize_name/normalize_name/good execution_time [16.749µs; 16.754µs] or [-0.017%; +0.017%] None None None
normalization/normalize_name/normalize_name/good throughput [59685783.153op/s; 59705912.200op/s] or [-0.017%; +0.017%] None None None

Group 3

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 67c55d9 1724847528 vianney/ddcommon/ignore-slow-tests-in-miri
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
two way interface execution_time 16.792µs 22.299µs ± 13.675µs 16.959µs ± 0.084µs 17.118µs 50.216µs 55.042µs 124.546µs 634.41% 3.781 19.803 61.17% 0.967µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
two way interface execution_time [20.403µs; 24.194µs] or [-8.499%; +8.499%] None None None

Group 4

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 67c55d9 1724847528 vianney/ddcommon/ignore-slow-tests-in-miri
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... execution_time 618.309µs 619.211µs ± 0.552µs 619.089µs ± 0.309µs 619.472µs 620.252µs 620.755µs 621.429µs 0.38% 1.119 1.482 0.09% 0.039µs 1 200
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... throughput 1609195.328op/s 1614958.478op/s ± 1436.995op/s 1615276.032op/s ± 805.947op/s 1615862.339op/s 1616767.535op/s 1617220.197op/s 1617314.843op/s 0.13% -1.113 1.461 0.09% 101.611op/s 1 200
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて execution_time 386.950µs 389.690µs ± 1.125µs 389.712µs ± 0.788µs 390.429µs 391.509µs 392.495µs 393.217µs 0.90% 0.199 -0.116 0.29% 0.080µs 1 200
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて throughput 2543125.337op/s 2566165.531op/s ± 7406.695op/s 2565998.352op/s ± 5190.034op/s 2571551.822op/s 2578379.017op/s 2580311.442op/s 2584313.219op/s 0.71% -0.183 -0.130 0.29% 523.732op/s 1 200
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters execution_time 190.843µs 191.188µs ± 0.144µs 191.183µs ± 0.089µs 191.272µs 191.430µs 191.547µs 191.686µs 0.26% 0.313 0.466 0.08% 0.010µs 1 200
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters throughput 5216867.873op/s 5230450.758op/s ± 3940.377op/s 5230589.885op/s ± 2439.081op/s 5232993.502op/s 5236807.179op/s 5238948.355op/s 5239920.509op/s 0.18% -0.308 0.459 0.08% 278.627op/s 1 200
normalization/normalize_service/normalize_service/[empty string] execution_time 44.621µs 44.820µs ± 0.055µs 44.823µs ± 0.033µs 44.856µs 44.899µs 44.942µs 45.004µs 0.40% -0.517 1.843 0.12% 0.004µs 1 200
normalization/normalize_service/normalize_service/[empty string] throughput 22220444.587op/s 22311368.686op/s ± 27519.788op/s 22310189.885op/s ± 16356.199op/s 22325331.593op/s 22363390.236op/s 22398824.485op/s 22410997.525op/s 0.45% 0.530 1.860 0.12% 1945.943op/s 1 200
normalization/normalize_service/normalize_service/test_ASCII execution_time 48.873µs 49.165µs ± 0.192µs 49.115µs ± 0.111µs 49.258µs 49.581µs 49.756µs 49.784µs 1.36% 1.181 1.038 0.39% 0.014µs 1 200
normalization/normalize_service/normalize_service/test_ASCII throughput 20086770.176op/s 20340078.321op/s ± 79232.297op/s 20360441.371op/s ± 46124.359op/s 20397441.781op/s 20435625.032op/s 20447792.923op/s 20461190.475op/s 0.49% -1.162 0.986 0.39% 5602.569op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... execution_time [619.135µs; 619.288µs] or [-0.012%; +0.012%] None None None
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... throughput [1614759.325op/s; 1615157.632op/s] or [-0.012%; +0.012%] None None None
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて execution_time [389.534µs; 389.846µs] or [-0.040%; +0.040%] None None None
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて throughput [2565139.035op/s; 2567192.028op/s] or [-0.040%; +0.040%] None None None
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters execution_time [191.168µs; 191.208µs] or [-0.010%; +0.010%] None None None
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters throughput [5229904.659op/s; 5230996.856op/s] or [-0.010%; +0.010%] None None None
normalization/normalize_service/normalize_service/[empty string] execution_time [44.813µs; 44.828µs] or [-0.017%; +0.017%] None None None
normalization/normalize_service/normalize_service/[empty string] throughput [22307554.708op/s; 22315182.664op/s] or [-0.017%; +0.017%] None None None
normalization/normalize_service/normalize_service/test_ASCII execution_time [49.138µs; 49.191µs] or [-0.054%; +0.054%] None None None
normalization/normalize_service/normalize_service/test_ASCII throughput [20329097.487op/s; 20351059.155op/s] or [-0.054%; +0.054%] None None None

Group 5

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 67c55d9 1724847528 vianney/ddcommon/ignore-slow-tests-in-miri
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
redis/obfuscate_redis_string execution_time 39.006µs 39.604µs ± 1.114µs 39.093µs ± 0.029µs 39.131µs 42.009µs 42.043µs 42.472µs 8.64% 1.700 0.920 2.81% 0.079µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
redis/obfuscate_redis_string execution_time [39.449µs; 39.758µs] or [-0.390%; +0.390%] None None None

Group 6

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 67c55d9 1724847528 vianney/ddcommon/ignore-slow-tests-in-miri
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
credit_card/is_card_number/ execution_time 1.613µs 1.614µs ± 0.000µs 1.614µs ± 0.000µs 1.614µs 1.614µs 1.614µs 1.616µs 0.17% 0.894 3.838 0.03% 0.000µs 1 200
credit_card/is_card_number/ throughput 618675905.773op/s 619750002.165op/s ± 187597.482op/s 619747534.644op/s ± 147949.398op/s 619897592.360op/s 620010919.980op/s 620079875.217op/s 620125778.359op/s 0.06% -0.890 3.808 0.03% 13265.145op/s 1 200
credit_card/is_card_number/ 3782-8224-6310-005 execution_time 104.430µs 105.215µs ± 0.565µs 105.063µs ± 0.169µs 105.277µs 106.270µs 107.551µs 107.908µs 2.71% 2.684 7.860 0.54% 0.040µs 1 200
credit_card/is_card_number/ 3782-8224-6310-005 throughput 9267114.140op/s 9504597.248op/s ± 50344.784op/s 9518071.776op/s ± 15283.466op/s 9531282.787op/s 9547113.526op/s 9559408.139op/s 9575784.756op/s 0.61% -2.641 7.634 0.53% 3559.914op/s 1 200
credit_card/is_card_number/ 378282246310005 execution_time 96.056µs 97.025µs ± 0.577µs 96.917µs ± 0.240µs 97.169µs 98.197µs 99.428µs 99.935µs 3.11% 2.248 6.982 0.59% 0.041µs 1 200
credit_card/is_card_number/ 378282246310005 throughput 10006546.591op/s 10306967.402op/s ± 60498.022op/s 10318068.858op/s ± 25535.749op/s 10342153.755op/s 10376201.926op/s 10390601.604op/s 10410562.557op/s 0.90% -2.180 6.605 0.59% 4277.856op/s 1 200
credit_card/is_card_number/37828224631 execution_time 1.613µs 1.614µs ± 0.003µs 1.613µs ± 0.000µs 1.614µs 1.614µs 1.616µs 1.660µs 2.86% 13.498 184.895 0.20% 0.000µs 1 200
credit_card/is_card_number/37828224631 throughput 602528168.532op/s 619646731.534op/s ± 1233505.110op/s 619774713.211op/s ± 129120.682op/s 619878073.643op/s 619966612.934op/s 620024502.813op/s 620044421.511op/s 0.04% -13.469 184.349 0.20% 87221.983op/s 1 200
credit_card/is_card_number/378282246310005 execution_time 94.156µs 95.417µs ± 0.703µs 95.320µs ± 0.524µs 95.879µs 96.671µs 97.014µs 97.378µs 2.16% 0.472 -0.422 0.74% 0.050µs 1 200
credit_card/is_card_number/378282246310005 throughput 10269271.936op/s 10480873.353op/s ± 76983.278op/s 10491015.707op/s ± 57905.421op/s 10545504.000op/s 10596373.340op/s 10611400.462op/s 10620694.423op/s 1.24% -0.443 -0.461 0.73% 5443.540op/s 1 200
credit_card/is_card_number/37828224631000521389798 execution_time 94.266µs 94.595µs ± 0.316µs 94.544µs ± 0.171µs 94.708µs 95.086µs 95.501µs 97.278µs 2.89% 3.629 24.972 0.33% 0.022µs 1 200
credit_card/is_card_number/37828224631000521389798 throughput 10279807.636op/s 10571549.614op/s ± 34935.274op/s 10577130.658op/s ± 19166.468op/s 10596444.363op/s 10603418.542op/s 10605293.495op/s 10608303.119op/s 0.29% -3.494 23.413 0.33% 2470.297op/s 1 200
credit_card/is_card_number/x371413321323331 execution_time 22.431µs 22.777µs ± 0.216µs 22.749µs ± 0.125µs 22.881µs 23.185µs 23.392µs 23.497µs 3.29% 0.799 0.452 0.95% 0.015µs 1 200
credit_card/is_card_number/x371413321323331 throughput 42558595.625op/s 43907365.776op/s ± 413067.460op/s 43957617.447op/s ± 241485.804op/s 44189093.893op/s 44478120.171op/s 44548780.733op/s 44580349.632op/s 1.42% -0.748 0.341 0.94% 29208.280op/s 1 200
credit_card/is_card_number_no_luhn/ execution_time 1.613µs 1.614µs ± 0.000µs 1.614µs ± 0.000µs 1.614µs 1.614µs 1.615µs 1.615µs 0.08% 0.252 -0.761 0.03% 0.000µs 1 200
credit_card/is_card_number_no_luhn/ throughput 619252395.107op/s 619748011.207op/s ± 180605.403op/s 619768887.610op/s ± 138511.347op/s 619894750.927op/s 620013027.177op/s 620116580.056op/s 620130715.711op/s 0.06% -0.251 -0.761 0.03% 12770.731op/s 1 200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 execution_time 85.873µs 86.630µs ± 0.315µs 86.597µs ± 0.199µs 86.824µs 87.175µs 87.475µs 87.715µs 1.29% 0.463 0.259 0.36% 0.022µs 1 200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 throughput 11400530.616op/s 11543539.160op/s ± 41885.387op/s 11547796.455op/s ± 26457.818op/s 11571561.730op/s 11600503.826op/s 11623489.095op/s 11645120.183op/s 0.84% -0.441 0.226 0.36% 2961.744op/s 1 200
credit_card/is_card_number_no_luhn/ 378282246310005 execution_time 79.124µs 80.156µs ± 0.556µs 80.108µs ± 0.379µs 80.487µs 81.108µs 81.449µs 82.330µs 2.77% 0.518 0.379 0.69% 0.039µs 1 200
credit_card/is_card_number_no_luhn/ 378282246310005 throughput 12146237.198op/s 12476334.569op/s ± 86285.682op/s 12483107.873op/s ± 58802.951op/s 12542400.126op/s 12601390.227op/s 12631459.502op/s 12638432.543op/s 1.24% -0.475 0.271 0.69% 6101.319op/s 1 200
credit_card/is_card_number_no_luhn/37828224631 execution_time 1.613µs 1.614µs ± 0.000µs 1.614µs ± 0.000µs 1.614µs 1.614µs 1.615µs 1.615µs 0.08% 0.344 -0.686 0.03% 0.000µs 1 200
credit_card/is_card_number_no_luhn/37828224631 throughput 619289534.896op/s 619736168.257op/s ± 167553.676op/s 619760271.686op/s ± 132982.292op/s 619880617.715op/s 619957890.097op/s 620036271.093op/s 620079364.555op/s 0.05% -0.343 -0.687 0.03% 11847.834op/s 1 200
credit_card/is_card_number_no_luhn/378282246310005 execution_time 77.330µs 78.306µs ± 0.436µs 78.322µs ± 0.279µs 78.593µs 79.003µs 79.204µs 79.902µs 2.02% 0.136 0.257 0.56% 0.031µs 1 200
credit_card/is_card_number_no_luhn/378282246310005 throughput 12515310.478op/s 12770824.384op/s ± 71123.482op/s 12767788.784op/s ± 45396.016op/s 12822470.418op/s 12896641.698op/s 12920711.935op/s 12931607.556op/s 1.28% -0.099 0.203 0.56% 5029.190op/s 1 200
credit_card/is_card_number_no_luhn/37828224631000521389798 execution_time 94.102µs 94.600µs ± 0.341µs 94.514µs ± 0.190µs 94.775µs 95.290µs 95.772µs 95.859µs 1.42% 1.209 1.558 0.36% 0.024µs 1 200
credit_card/is_card_number_no_luhn/37828224631000521389798 throughput 10432028.690op/s 10570917.792op/s ± 37941.620op/s 10580441.525op/s ± 21236.484op/s 10599487.780op/s 10616938.854op/s 10623599.918op/s 10626805.053op/s 0.44% -1.187 1.479 0.36% 2682.878op/s 1 200
credit_card/is_card_number_no_luhn/x371413321323331 execution_time 22.442µs 22.784µs ± 0.212µs 22.743µs ± 0.136µs 22.923µs 23.161µs 23.350µs 23.395µs 2.86% 0.649 0.058 0.93% 0.015µs 1 200
credit_card/is_card_number_no_luhn/x371413321323331 throughput 42744142.887op/s 43894226.083op/s ± 406018.886op/s 43968740.892op/s ± 265301.426op/s 44188836.647op/s 44502073.382op/s 44541257.842op/s 44560224.511op/s 1.35% -0.604 -0.021 0.92% 28709.871op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
credit_card/is_card_number/ execution_time [1.613µs; 1.614µs] or [-0.004%; +0.004%] None None None
credit_card/is_card_number/ throughput [619724002.958op/s; 619776001.372op/s] or [-0.004%; +0.004%] None None None
credit_card/is_card_number/ 3782-8224-6310-005 execution_time [105.137µs; 105.294µs] or [-0.074%; +0.074%] None None None
credit_card/is_card_number/ 3782-8224-6310-005 throughput [9497619.946op/s; 9511574.551op/s] or [-0.073%; +0.073%] None None None
credit_card/is_card_number/ 378282246310005 execution_time [96.945µs; 97.105µs] or [-0.082%; +0.082%] None None None
credit_card/is_card_number/ 378282246310005 throughput [10298582.958op/s; 10315351.846op/s] or [-0.081%; +0.081%] None None None
credit_card/is_card_number/37828224631 execution_time [1.613µs; 1.614µs] or [-0.028%; +0.028%] None None None
credit_card/is_card_number/37828224631 throughput [619475779.590op/s; 619817683.479op/s] or [-0.028%; +0.028%] None None None
credit_card/is_card_number/378282246310005 execution_time [95.320µs; 95.514µs] or [-0.102%; +0.102%] None None None
credit_card/is_card_number/378282246310005 throughput [10470204.212op/s; 10491542.495op/s] or [-0.102%; +0.102%] None None None
credit_card/is_card_number/37828224631000521389798 execution_time [94.551µs; 94.638µs] or [-0.046%; +0.046%] None None None
credit_card/is_card_number/37828224631000521389798 throughput [10566707.921op/s; 10576391.307op/s] or [-0.046%; +0.046%] None None None
credit_card/is_card_number/x371413321323331 execution_time [22.747µs; 22.807µs] or [-0.131%; +0.131%] None None None
credit_card/is_card_number/x371413321323331 throughput [43850118.598op/s; 43964612.953op/s] or [-0.130%; +0.130%] None None None
credit_card/is_card_number_no_luhn/ execution_time [1.613µs; 1.614µs] or [-0.004%; +0.004%] None None None
credit_card/is_card_number_no_luhn/ throughput [619722981.035op/s; 619773041.378op/s] or [-0.004%; +0.004%] None None None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 execution_time [86.586µs; 86.673µs] or [-0.050%; +0.050%] None None None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 throughput [11537734.248op/s; 11549344.072op/s] or [-0.050%; +0.050%] None None None
credit_card/is_card_number_no_luhn/ 378282246310005 execution_time [80.078µs; 80.233µs] or [-0.096%; +0.096%] None None None
credit_card/is_card_number_no_luhn/ 378282246310005 throughput [12464376.203op/s; 12488292.934op/s] or [-0.096%; +0.096%] None None None
credit_card/is_card_number_no_luhn/37828224631 execution_time [1.614µs; 1.614µs] or [-0.004%; +0.004%] None None None
credit_card/is_card_number_no_luhn/37828224631 throughput [619712946.929op/s; 619759389.585op/s] or [-0.004%; +0.004%] None None None
credit_card/is_card_number_no_luhn/378282246310005 execution_time [78.245µs; 78.366µs] or [-0.077%; +0.077%] None None None
credit_card/is_card_number_no_luhn/378282246310005 throughput [12760967.354op/s; 12780681.415op/s] or [-0.077%; +0.077%] None None None
credit_card/is_card_number_no_luhn/37828224631000521389798 execution_time [94.553µs; 94.648µs] or [-0.050%; +0.050%] None None None
credit_card/is_card_number_no_luhn/37828224631000521389798 throughput [10565659.448op/s; 10576176.136op/s] or [-0.050%; +0.050%] None None None
credit_card/is_card_number_no_luhn/x371413321323331 execution_time [22.755µs; 22.813µs] or [-0.129%; +0.129%] None None None
credit_card/is_card_number_no_luhn/x371413321323331 throughput [43837955.770op/s; 43950496.395op/s] or [-0.128%; +0.128%] None None None

Group 7

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 67c55d9 1724847528 vianney/ddcommon/ignore-slow-tests-in-miri
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
write only interface execution_time 1.425µs 2.978µs ± 1.447µs 2.814µs ± 0.024µs 2.835µs 3.068µs 13.010µs 16.525µs 487.19% 7.953 63.722 48.48% 0.102µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
write only interface execution_time [2.777µs; 3.178µs] or [-6.736%; +6.736%] None None None

Group 8

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 67c55d9 1724847528 vianney/ddcommon/ignore-slow-tests-in-miri
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_trace/test_trace execution_time 301.359ns 312.014ns ± 13.229ns 305.991ns ± 3.260ns 315.005ns 337.947ns 352.818ns 359.475ns 17.48% 1.763 2.264 4.23% 0.935ns 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_trace/test_trace execution_time [310.180ns; 313.847ns] or [-0.588%; +0.588%] None None None

Group 9

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 67c55d9 1724847528 vianney/ddcommon/ignore-slow-tests-in-miri
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
sql/obfuscate_sql_string execution_time 75.347µs 75.814µs ± 0.199µs 75.858µs ± 0.087µs 75.926µs 75.999µs 76.038µs 77.435µs 2.08% 2.282 20.467 0.26% 0.014µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
sql/obfuscate_sql_string execution_time [75.786µs; 75.842µs] or [-0.036%; +0.036%] None None None

Group 10

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 67c55d9 1724847528 vianney/ddcommon/ignore-slow-tests-in-miri
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
tags/replace_trace_tags execution_time 2.657µs 2.715µs ± 0.021µs 2.719µs ± 0.008µs 2.723µs 2.756µs 2.758µs 2.758µs 1.42% -0.526 0.873 0.76% 0.001µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
tags/replace_trace_tags execution_time [2.712µs; 2.718µs] or [-0.106%; +0.106%] None None None

Group 11

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 67c55d9 1724847528 vianney/ddcommon/ignore-slow-tests-in-miri
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
benching deserializing traces from msgpack to their internal representation execution_time 1.154µs 1.351µs ± 0.062µs 1.356µs ± 0.052µs 1.403µs 1.439µs 1.445µs 1.446µs 6.63% -0.600 0.329 4.56% 0.004µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
benching deserializing traces from msgpack to their internal representation execution_time [1.343µs; 1.360µs] or [-0.633%; +0.633%] None None None

Baseline

Omitted due to size.

@VianneyRuhlmann VianneyRuhlmann changed the title Ignore entity tests in miri Improve miri test time Aug 28, 2024
@VianneyRuhlmann VianneyRuhlmann changed the title Improve miri test time Improve miri job runtime Aug 28, 2024
@github-actions github-actions bot removed the ci-build label Aug 28, 2024
@VianneyRuhlmann VianneyRuhlmann marked this pull request as ready for review August 29, 2024 07:54
@VianneyRuhlmann VianneyRuhlmann requested a review from a team as a code owner August 29, 2024 07:54
bantonsson
bantonsson approved these changes Aug 29, 2024
View reviewed changes
Copy link
Contributor

@bantonsson bantonsson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍🏻

From the diff, it's clear that you reverted the parallel running of the tests, so it would be great if you had removed those commits as well.

It's easy to back up X commits:

git reset --hard HEAD~X
git push -f

@VianneyRuhlmann
Copy link
Contributor Author

Yes I figured those would get squashed anyway

@VianneyRuhlmann VianneyRuhlmann force-pushed the vianney/ddcommon/ignore-slow-tests-in-miri branch from 9139e27 to 67c55d9 Compare August 29, 2024 08:53
@bantonsson
Copy link
Contributor

Yes, they will get squashed. It was only a bit confusing to see when I opened the PR.

@VianneyRuhlmann VianneyRuhlmann merged commit d910738 into main Aug 29, 2024
34 checks passed
@VianneyRuhlmann VianneyRuhlmann deleted the vianney/ddcommon/ignore-slow-tests-in-miri branch August 29, 2024 09:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@bantonsson bantonsson bantonsson approved these changes

Assignees
No one assigned
Labels
common
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@VianneyRuhlmann @codecov-commenter @bantonsson