[WIP] run the benchmarks

[bantonsson]

What does this PR do?

A brief description of the change being made with this pull request.

Motivation

What inspired you to submit this pull request?

Additional Notes

Anything else we should know when reviewing?

How to test the change?

Describe here in detail how the change can be validated.

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 71.30%. Comparing base (11db3f1) to head (5ce0710).

Additional details and impacted files

@@                       Coverage Diff                        @@
##           ban/stabilize-criterion-results     #577   +/-   ##
================================================================
  Coverage                            71.30%   71.30%           
================================================================
  Files                                  220      220           
  Lines                                29904    29904           
================================================================
+ Hits                                 21322    21323    +1     
+ Misses                                8582     8581    -1     

Components	Coverage Δ
crashtracker	21.17% <ø> (+0.05%)	⬆️
datadog-alloc	98.73% <ø> (ø)
data-pipeline	50.00% <ø> (ø)
data-pipeline-ffi	0.00% <ø> (ø)
ddcommon	83.07% <ø> (ø)
ddcommon-ffi	70.20% <ø> (ø)
ddtelemetry	58.95% <ø> (ø)
ipc	84.18% <ø> (ø)
profiling	84.26% <ø> (ø)
profiling-ffi	77.42% <ø> (ø)
serverless	0.00% <ø> (ø)
sidecar	34.55% <ø> (ø)
sidecar-ffi	0.00% <ø> (ø)
spawn-worker	54.87% <ø> (ø)
trace-mini-agent	70.88% <ø> (ø)
trace-normalization	98.24% <ø> (ø)
trace-obfuscation	95.73% <ø> (ø)
trace-protobuf	77.16% <ø> (ø)
trace-utils	93.39% <ø> (ø)

[pr-commenter]

Benchmarks

Comparison

Benchmark execution time: 2024-08-07 15:48:14

Comparing candidate commit 5ce0710 in PR branch ban/tmp-test-criterion with baseline commit 11db3f1 in branch ban/stabilize-criterion-results.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 50 metrics, 2 unstable metrics.

Candidate

Candidate benchmark details

Group 1

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	5ce0710	1723042321	ban/tmp-test-criterion

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
sql/obfuscate_sql_string	execution_time	73.719µs	73.960µs ± 0.147µs	73.959µs ± 0.058µs	74.013µs	74.101µs	74.186µs	75.510µs	2.10%	5.966	60.588	0.20%	0.010µs	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
sql/obfuscate_sql_string	execution_time	[73.940µs; 73.980µs] or [-0.028%; +0.028%]	None	None	None

Group 2

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	5ce0710	1723042321	ban/tmp-test-criterion

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
write only interface	execution_time	1.466µs	2.908µs ± 1.420µs	2.744µs ± 0.024µs	2.766µs	2.980µs	12.747µs	16.229µs	491.35%	8.006	64.301	48.72%	0.100µs	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
write only interface	execution_time	[2.711µs; 3.105µs] or [-6.769%; +6.769%]	None	None	None

Group 3

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	5ce0710	1723042321	ban/tmp-test-criterion

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo...	execution_time	282.611µs	285.559µs ± 1.140µs	285.577µs ± 0.379µs	285.920µs	286.558µs	286.826µs	298.164µs	4.41%	6.485	73.357	0.40%	0.081µs	1	200
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo...	throughput	3353856.540op/s	3501957.774op/s ± 13639.021op/s	3501679.276op/s ± 4656.235op/s	3506817.253op/s	3520109.462op/s	3531213.781op/s	3538434.475op/s	1.05%	-6.093	68.195	0.39%	964.424op/s	1	200
normalization/normalize_name/normalize_name/bad-name	execution_time	36.854µs	37.122µs ± 0.076µs	37.120µs ± 0.048µs	37.168µs	37.238µs	37.310µs	37.351µs	0.62%	0.053	0.497	0.20%	0.005µs	1	200
normalization/normalize_name/normalize_name/bad-name	throughput	26772954.356op/s	26938079.714op/s ± 54993.430op/s	26939425.995op/s ± 35022.321op/s	26974555.187op/s	27019387.805op/s	27053937.539op/s	27134073.456op/s	0.72%	-0.038	0.503	0.20%	3888.623op/s	1	200
normalization/normalize_name/normalize_name/good	execution_time	24.253µs	24.421µs ± 0.078µs	24.418µs ± 0.053µs	24.469µs	24.549µs	24.598µs	24.792µs	1.53%	0.540	1.668	0.32%	0.006µs	1	200
normalization/normalize_name/normalize_name/good	throughput	40336048.289op/s	40949152.574op/s ± 130928.007op/s	40953865.311op/s ± 89290.525op/s	41045986.064op/s	41155407.694op/s	41212374.635op/s	41232632.199op/s	0.68%	-0.508	1.551	0.32%	9258.008op/s	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo...	execution_time	[285.401µs; 285.717µs] or [-0.055%; +0.055%]	None	None	None
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo...	throughput	[3500067.537op/s; 3503848.011op/s] or [-0.054%; +0.054%]	None	None	None
normalization/normalize_name/normalize_name/bad-name	execution_time	[37.112µs; 37.133µs] or [-0.028%; +0.028%]	None	None	None
normalization/normalize_name/normalize_name/bad-name	throughput	[26930458.153op/s; 26945701.274op/s] or [-0.028%; +0.028%]	None	None	None
normalization/normalize_name/normalize_name/good	execution_time	[24.410µs; 24.432µs] or [-0.044%; +0.044%]	None	None	None
normalization/normalize_name/normalize_name/good	throughput	[40931007.211op/s; 40967297.936op/s] or [-0.044%; +0.044%]	None	None	None

Group 4

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	5ce0710	1723042321	ban/tmp-test-criterion

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
redis/obfuscate_redis_string	execution_time	37.273µs	37.868µs ± 1.060µs	37.362µs ± 0.044µs	37.498µs	40.155µs	40.173µs	40.633µs	8.76%	1.689	0.904	2.79%	0.075µs	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
redis/obfuscate_redis_string	execution_time	[37.721µs; 38.015µs] or [-0.388%; +0.388%]	None	None	None

Group 5

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	5ce0710	1723042321	ban/tmp-test-criterion

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
two way interface	execution_time	16.614µs	22.503µs ± 15.461µs	16.835µs ± 0.075µs	17.015µs	50.528µs	53.616µs	147.633µs	776.92%	4.587	28.798	68.54%	1.093µs	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
two way interface	execution_time	[20.360µs; 24.646µs] or [-9.522%; +9.522%]	None	None	None

Group 6

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	5ce0710	1723042321	ban/tmp-test-criterion

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
credit_card/is_card_number/	execution_time	2.415µs	2.416µs ± 0.001µs	2.416µs ± 0.001µs	2.416µs	2.417µs	2.418µs	2.420µs	0.19%	1.388	5.201	0.03%	0.000µs	1	200
credit_card/is_card_number/	throughput	413140936.158op/s	413910440.772op/s ± 130014.079op/s	413942387.701op/s ± 91511.313op/s	414008190.691op/s	414068101.299op/s	414087674.922op/s	414145049.626op/s	0.05%	-1.383	5.166	0.03%	9193.384op/s	1	200
credit_card/is_card_number/ 3782-8224-6310-005	execution_time	105.245µs	107.866µs ± 2.444µs	107.002µs ± 1.383µs	109.387µs	112.692µs	113.632µs	114.041µs	6.58%	0.932	-0.340	2.26%	0.173µs	1	200
credit_card/is_card_number/ 3782-8224-6310-005	throughput	8768738.110op/s	9275435.585op/s ± 205958.327op/s	9345646.997op/s ± 122402.370op/s	9447235.645op/s	9491667.814op/s	9500131.801op/s	9501657.477op/s	1.67%	-0.880	-0.447	2.21%	14563.453op/s	1	200
credit_card/is_card_number/ 378282246310005	execution_time	100.841µs	104.408µs ± 2.410µs	103.697µs ± 1.586µs	106.145µs	108.916µs	109.999µs	111.596µs	7.62%	0.702	-0.408	2.30%	0.170µs	1	200
credit_card/is_card_number/ 378282246310005	throughput	8960909.712op/s	9582776.577op/s ± 217871.345op/s	9643518.305op/s ± 149825.332op/s	9747077.842op/s	9857738.573op/s	9893303.926op/s	9916565.530op/s	2.83%	-0.629	-0.552	2.27%	15405.831op/s	1	200
credit_card/is_card_number/37828224631	execution_time	2.415µs	2.416µs ± 0.005µs	2.416µs ± 0.001µs	2.417µs	2.417µs	2.418µs	2.483µs	2.77%	13.300	180.950	0.20%	0.000µs	1	200
credit_card/is_card_number/37828224631	throughput	402748495.822op/s	413838506.417op/s ± 803579.702op/s	413921665.344op/s ± 93073.147op/s	413999047.802op/s	414052334.145op/s	414090294.063op/s	414157545.530op/s	0.06%	-13.264	180.252	0.19%	56821.666op/s	1	200
credit_card/is_card_number/378282246310005	execution_time	98.005µs	102.181µs ± 2.890µs	102.508µs ± 2.731µs	104.830µs	106.155µs	107.073µs	107.319µs	4.69%	-0.027	-1.572	2.82%	0.204µs	1	200
credit_card/is_card_number/378282246310005	throughput	9317982.945op/s	9794316.094op/s ± 277373.758op/s	9755369.059op/s ± 255359.358op/s	10092844.001op/s	10172343.431op/s	10197246.246op/s	10203575.766op/s	4.59%	0.062	-1.581	2.82%	19613.287op/s	1	200
credit_card/is_card_number/37828224631000521389798	execution_time	103.445µs	107.909µs ± 1.152µs	107.849µs ± 0.650µs	108.506µs	109.859µs	110.719µs	112.689µs	4.49%	0.374	1.984	1.07%	0.081µs	1	200
credit_card/is_card_number/37828224631000521389798	throughput	8873992.456op/s	9268149.010op/s ± 98639.580op/s	9272230.379op/s ± 55892.894op/s	9326795.017op/s	9405762.489op/s	9474432.054op/s	9666976.807op/s	4.26%	-0.251	1.946	1.06%	6974.872op/s	1	200
credit_card/is_card_number/x371413321323331	execution_time	22.345µs	23.741µs ± 0.527µs	23.730µs ± 0.392µs	24.124µs	24.598µs	24.765µs	25.166µs	6.05%	-0.030	-0.433	2.21%	0.037µs	1	200
credit_card/is_card_number/x371413321323331	throughput	39736020.371op/s	42142566.382op/s ± 937054.532op/s	42140399.183op/s ± 687862.478op/s	42818541.852op/s	43571708.627op/s	44550778.405op/s	44752016.291op/s	6.20%	0.135	-0.386	2.22%	66259.761op/s	1	200
credit_card/is_card_number_no_luhn/	execution_time	2.415µs	2.416µs ± 0.001µs	2.416µs ± 0.001µs	2.417µs	2.417µs	2.418µs	2.418µs	0.08%	0.385	-0.705	0.03%	0.000µs	1	200
credit_card/is_card_number_no_luhn/	throughput	413559145.950op/s	413894985.928op/s ± 121279.318op/s	413896070.700op/s ± 102537.008op/s	414003964.094op/s	414053535.052op/s	414082255.370op/s	414104851.348op/s	0.05%	-0.384	-0.706	0.03%	8575.743op/s	1	200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005	execution_time	88.906µs	89.744µs ± 0.359µs	89.739µs ± 0.177µs	89.865µs	90.389µs	90.682µs	91.265µs	1.70%	0.786	1.852	0.40%	0.025µs	1	200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005	throughput	10957138.635op/s	11142936.516op/s ± 44491.845op/s	11143415.863op/s ± 21905.749op/s	11174112.928op/s	11204961.812op/s	11241493.516op/s	11247881.522op/s	0.94%	-0.748	1.756	0.40%	3146.049op/s	1	200
credit_card/is_card_number_no_luhn/ 378282246310005	execution_time	83.015µs	85.757µs ± 1.550µs	85.823µs ± 1.336µs	86.981µs	88.319µs	88.854µs	88.917µs	3.61%	0.150	-1.014	1.80%	0.110µs	1	200
credit_card/is_card_number_no_luhn/ 378282246310005	throughput	11246408.719op/s	11664628.167op/s ± 210383.816op/s	11651936.920op/s ± 180484.755op/s	11859369.298op/s	11969557.548op/s	12001431.606op/s	12046020.741op/s	3.38%	-0.099	-1.052	1.80%	14876.382op/s	1	200
credit_card/is_card_number_no_luhn/37828224631	execution_time	2.415µs	2.416µs ± 0.001µs	2.416µs ± 0.001µs	2.416µs	2.417µs	2.418µs	2.420µs	0.16%	1.213	3.893	0.03%	0.000µs	1	200
credit_card/is_card_number_no_luhn/37828224631	throughput	413240885.082op/s	413907290.136op/s ± 118428.818op/s	413922148.428op/s ± 87882.951op/s	414002312.581op/s	414053906.136op/s	414086956.879op/s	414102128.913op/s	0.04%	-1.209	3.868	0.03%	8374.182op/s	1	200
credit_card/is_card_number_no_luhn/378282246310005	execution_time	82.189µs	84.742µs ± 0.776µs	84.872µs ± 0.486µs	85.265µs	85.791µs	86.190µs	86.324µs	1.71%	-0.593	0.016	0.91%	0.055µs	1	200
credit_card/is_card_number_no_luhn/378282246310005	throughput	11584207.074op/s	11801569.143op/s ± 108626.435op/s	11782417.692op/s ± 67126.270op/s	11866261.187op/s	11989108.559op/s	12070777.709op/s	12167120.975op/s	3.27%	0.640	0.099	0.92%	7681.049op/s	1	200
credit_card/is_card_number_no_luhn/37828224631000521389798	execution_time	103.497µs	108.201µs ± 1.192µs	108.133µs ± 0.734µs	108.863µs	110.134µs	111.678µs	112.626µs	4.15%	0.252	2.313	1.10%	0.084µs	1	200
credit_card/is_card_number_no_luhn/37828224631000521389798	throughput	8878953.796op/s	9243198.724op/s ± 101625.167op/s	9247828.844op/s ± 62466.204op/s	9308328.131op/s	9385992.776op/s	9488753.371op/s	9662104.768op/s	4.48%	-0.111	2.334	1.10%	7185.984op/s	1	200
credit_card/is_card_number_no_luhn/x371413321323331	execution_time	22.592µs	23.775µs ± 0.491µs	23.793µs ± 0.336µs	24.108µs	24.582µs	24.792µs	25.144µs	5.68%	0.003	-0.256	2.06%	0.035µs	1	200
credit_card/is_card_number_no_luhn/x371413321323331	throughput	39771448.002op/s	42079108.902op/s ± 870635.566op/s	42029208.545op/s ± 599403.195op/s	42659552.610op/s	43591039.573op/s	44091426.744op/s	44263976.088op/s	5.32%	0.104	-0.258	2.06%	61563.231op/s	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
credit_card/is_card_number/	execution_time	[2.416µs; 2.416µs] or [-0.004%; +0.004%]	None	None	None
credit_card/is_card_number/	throughput	[413892422.071op/s; 413928459.473op/s] or [-0.004%; +0.004%]	None	None	None
credit_card/is_card_number/ 3782-8224-6310-005	execution_time	[107.527µs; 108.204µs] or [-0.314%; +0.314%]	None	None	None
credit_card/is_card_number/ 3782-8224-6310-005	throughput	[9246891.742op/s; 9303979.429op/s] or [-0.308%; +0.308%]	None	None	None
credit_card/is_card_number/ 378282246310005	execution_time	[104.074µs; 104.742µs] or [-0.320%; +0.320%]	None	None	None
credit_card/is_card_number/ 378282246310005	throughput	[9552581.704op/s; 9612971.450op/s] or [-0.315%; +0.315%]	None	None	None
credit_card/is_card_number/37828224631	execution_time	[2.416µs; 2.417µs] or [-0.028%; +0.028%]	None	None	None
credit_card/is_card_number/37828224631	throughput	[413727137.999op/s; 413949874.835op/s] or [-0.027%; +0.027%]	None	None	None
credit_card/is_card_number/378282246310005	execution_time	[101.781µs; 102.582µs] or [-0.392%; +0.392%]	None	None	None
credit_card/is_card_number/378282246310005	throughput	[9755874.758op/s; 9832757.429op/s] or [-0.392%; +0.392%]	None	None	None
credit_card/is_card_number/37828224631000521389798	execution_time	[107.749µs; 108.068µs] or [-0.148%; +0.148%]	None	None	None
credit_card/is_card_number/37828224631000521389798	throughput	[9254478.513op/s; 9281819.507op/s] or [-0.147%; +0.147%]	None	None	None
credit_card/is_card_number/x371413321323331	execution_time	[23.668µs; 23.814µs] or [-0.308%; +0.308%]	None	None	None
credit_card/is_card_number/x371413321323331	throughput	[42012699.636op/s; 42272433.128op/s] or [-0.308%; +0.308%]	None	None	None
credit_card/is_card_number_no_luhn/	execution_time	[2.416µs; 2.416µs] or [-0.004%; +0.004%]	None	None	None
credit_card/is_card_number_no_luhn/	throughput	[413878177.781op/s; 413911794.075op/s] or [-0.004%; +0.004%]	None	None	None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005	execution_time	[89.695µs; 89.794µs] or [-0.056%; +0.056%]	None	None	None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005	throughput	[11136770.374op/s; 11149102.658op/s] or [-0.055%; +0.055%]	None	None	None
credit_card/is_card_number_no_luhn/ 378282246310005	execution_time	[85.542µs; 85.972µs] or [-0.251%; +0.251%]	None	None	None
credit_card/is_card_number_no_luhn/ 378282246310005	throughput	[11635470.994op/s; 11693785.341op/s] or [-0.250%; +0.250%]	None	None	None
credit_card/is_card_number_no_luhn/37828224631	execution_time	[2.416µs; 2.416µs] or [-0.004%; +0.004%]	None	None	None
credit_card/is_card_number_no_luhn/37828224631	throughput	[413890877.041op/s; 413923703.231op/s] or [-0.004%; +0.004%]	None	None	None
credit_card/is_card_number_no_luhn/378282246310005	execution_time	[84.634µs; 84.849µs] or [-0.127%; +0.127%]	None	None	None
credit_card/is_card_number_no_luhn/378282246310005	throughput	[11786514.564op/s; 11816623.722op/s] or [-0.128%; +0.128%]	None	None	None
credit_card/is_card_number_no_luhn/37828224631000521389798	execution_time	[108.035µs; 108.366µs] or [-0.153%; +0.153%]	None	None	None
credit_card/is_card_number_no_luhn/37828224631000521389798	throughput	[9229114.453op/s; 9257282.995op/s] or [-0.152%; +0.152%]	None	None	None
credit_card/is_card_number_no_luhn/x371413321323331	execution_time	[23.707µs; 23.843µs] or [-0.286%; +0.286%]	None	None	None
credit_card/is_card_number_no_luhn/x371413321323331	throughput	[41958447.186op/s; 42199770.618op/s] or [-0.287%; +0.287%]	None	None	None

Group 7

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	5ce0710	1723042321	ban/tmp-test-criterion

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
normalization/normalize_trace/test_trace	execution_time	332.854ns	349.311ns ± 16.065ns	343.161ns ± 7.839ns	360.440ns	376.484ns	397.392ns	403.740ns	17.65%	1.199	0.725	4.59%	1.136ns	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
normalization/normalize_trace/test_trace	execution_time	[347.085ns; 351.538ns] or [-0.637%; +0.637%]	None	None	None

Group 8

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	5ce0710	1723042321	ban/tmp-test-criterion

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
tags/replace_trace_tags	execution_time	2.630µs	2.690µs ± 0.018µs	2.694µs ± 0.009µs	2.703µs	2.709µs	2.712µs	2.728µs	1.24%	-1.557	2.245	0.68%	0.001µs	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
tags/replace_trace_tags	execution_time	[2.688µs; 2.693µs] or [-0.094%; +0.094%]	None	None	None

Group 9

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	5ce0710	1723042321	ban/tmp-test-criterion

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
benching deserializing traces from msgpack to their internal representation	execution_time	1.190µs	1.392µs ± 0.062µs	1.395µs ± 0.048µs	1.439µs	1.478µs	1.487µs	1.489µs	6.77%	-0.762	0.957	4.45%	0.004µs	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
benching deserializing traces from msgpack to their internal representation	execution_time	[1.383µs; 1.400µs] or [-0.619%; +0.619%]	None	None	None

Group 10

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	5ce0710	1723042321	ban/tmp-test-criterion

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000...	execution_time	555.698µs	557.529µs ± 1.613µs	557.754µs ± 0.757µs	558.312µs	558.591µs	559.059µs	576.066µs	3.28%	7.558	85.282	0.29%	0.114µs	1	200
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000...	throughput	1735911.074op/s	1793641.821op/s ± 5080.208op/s	1792905.920op/s ± 2432.334op/s	1796823.206op/s	1798298.704op/s	1799194.020op/s	1799539.610op/s	0.37%	-7.300	81.421	0.28%	359.225op/s	1	200
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて	execution_time	416.049µs	420.846µs ± 2.756µs	420.522µs ± 1.614µs	422.101µs	425.972µs	428.413µs	430.375µs	2.34%	0.743	0.663	0.65%	0.195µs	1	200
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて	throughput	2323556.325op/s	2376269.092op/s ± 15486.616op/s	2377998.898op/s ± 9119.742op/s	2387189.048op/s	2399193.693op/s	2403259.306op/s	2403562.767op/s	1.08%	-0.702	0.577	0.65%	1095.069op/s	1	200
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters	execution_time	200.662µs	202.048µs ± 1.017µs	201.841µs ± 0.233µs	202.151µs	203.491µs	204.330µs	212.894µs	5.48%	6.412	63.230	0.50%	0.072µs	1	200
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters	throughput	4697168.471op/s	4949432.803op/s ± 24149.303op/s	4954397.450op/s ± 5736.813op/s	4959935.285op/s	4970391.951op/s	4979808.244op/s	4983500.572op/s	0.59%	-6.060	58.041	0.49%	1707.614op/s	1	200
normalization/normalize_service/normalize_service/[empty string]	execution_time	48.718µs	48.845µs ± 0.051µs	48.847µs ± 0.038µs	48.882µs	48.926µs	48.956µs	48.968µs	0.25%	-0.113	-0.592	0.10%	0.004µs	1	200
normalization/normalize_service/normalize_service/[empty string]	throughput	20421628.316op/s	20473078.229op/s ± 21347.806op/s	20472291.139op/s ± 15792.843op/s	20489931.992op/s	20508892.390op/s	20516136.509op/s	20526218.154op/s	0.26%	0.117	-0.592	0.10%	1509.518op/s	1	200
normalization/normalize_service/normalize_service/test_ASCII	execution_time	63.215µs	63.970µs ± 0.322µs	64.049µs ± 0.195µs	64.211µs	64.408µs	64.565µs	64.585µs	0.84%	-0.473	-0.647	0.50%	0.023µs	1	200
normalization/normalize_service/normalize_service/test_ASCII	throughput	15483452.468op/s	15632833.509op/s ± 78914.585op/s	15613096.008op/s ± 47531.129op/s	15693352.406op/s	15775613.518op/s	15806874.484op/s	15818954.563op/s	1.32%	0.490	-0.637	0.50%	5580.104op/s	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000...	execution_time	[557.306µs; 557.753µs] or [-0.040%; +0.040%]	None	None	None
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000...	throughput	[1792937.754op/s; 1794345.889op/s] or [-0.039%; +0.039%]	None	None	None
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて	execution_time	[420.464µs; 421.228µs] or [-0.091%; +0.091%]	None	None	None
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて	throughput	[2374122.796op/s; 2378415.388op/s] or [-0.090%; +0.090%]	None	None	None
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters	execution_time	[201.907µs; 202.189µs] or [-0.070%; +0.070%]	None	None	None
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters	throughput	[4946085.942op/s; 4952779.664op/s] or [-0.068%; +0.068%]	None	None	None
normalization/normalize_service/normalize_service/[empty string]	execution_time	[48.838µs; 48.852µs] or [-0.014%; +0.014%]	None	None	None
normalization/normalize_service/normalize_service/[empty string]	throughput	[20470119.629op/s; 20476036.830op/s] or [-0.014%; +0.014%]	None	None	None
normalization/normalize_service/normalize_service/test_ASCII	execution_time	[63.925µs; 64.014µs] or [-0.070%; +0.070%]	None	None	None
normalization/normalize_service/normalize_service/test_ASCII	throughput	[15621896.706op/s; 15643770.311op/s] or [-0.070%; +0.070%]	None	None	None

Group 11

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	5ce0710	1723042321	ban/tmp-test-criterion

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
benching string interning on wordpress profile	execution_time	133.646µs	134.092µs ± 0.232µs	134.071µs ± 0.174µs	134.265µs	134.437µs	134.738µs	134.780µs	0.53%	0.283	-0.349	0.17%	0.016µs	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
benching string interning on wordpress profile	execution_time	[134.059µs; 134.124µs] or [-0.024%; +0.024%]	None	None	None

Baseline

Omitted due to size.