[WIP] benchmarks with dogstatsd

[VianneyRuhlmann]

What does this PR do?

Attempt to stabilize benchmarks

Motivation

What inspired you to submit this pull request?

Additional Notes

Anything else we should know when reviewing?

How to test the change?

Describe here in detail how the change can be validated.

[pr-commenter]

Benchmarks

Comparison

Benchmark execution time: 2024-09-19 12:10:45

Comparing candidate commit 7f4735c in PR branch vianney/benchmarks-with-dogstatsd with baseline commit 2f1c45b in branch vianney/main-with-iterations.

Found 7 performance improvements and 24 performance regressions! Performance is the same for 20 metrics, 2 unstable metrics.

scenario:credit_card/is_card_number/

• 🟥 execution_time [+120.153µs; +120.225µs] or [+99.981%; +100.040%]
• 🟥 throughput [-416146731.916op/s; -416004826.948op/s] or [-50.011%; -49.994%]

scenario:credit_card/is_card_number/ 3782-8224-6310-005

• 🟥 execution_time [+516.242µs; +559.557µs] or [+4.639%; +5.028%]
• 🟥 throughput [-429683.613op/s; -397277.253op/s] or [-4.782%; -4.421%]

scenario:credit_card/is_card_number/ 378282246310005

• 🟥 execution_time [+400.646µs; +455.357µs] or [+3.853%; +4.379%]
• 🟥 throughput [-402725.371op/s; -355012.005op/s] or [-4.188%; -3.691%]

scenario:credit_card/is_card_number/37828224631

• 🟥 execution_time [+120.137µs; +120.165µs] or [+99.961%; +99.984%]
• 🟥 throughput [-416021184.296op/s; -415924418.259op/s] or [-49.999%; -49.987%]

scenario:credit_card/is_card_number/378282246310005

• 🟥 execution_time [+409.153µs; +458.864µs] or [+4.069%; +4.563%]
• 🟥 throughput [-433906.708op/s; -387423.340op/s] or [-4.363%; -3.895%]

scenario:credit_card/is_card_number/37828224631000521389798

• 🟥 execution_time [+835.689µs; +880.791µs] or [+8.740%; +9.212%]
• 🟥 throughput [-881947.018op/s; -838840.446op/s] or [-8.432%; -8.020%]

scenario:credit_card/is_card_number_no_luhn/

• 🟥 execution_time [+120.023µs; +120.067µs] or [+99.776%; +99.813%]
• 🟥 throughput [-415359346.174op/s; -415094375.200op/s] or [-49.965%; -49.933%]

scenario:credit_card/is_card_number_no_luhn/ 3782-8224-6310-005

• 🟩 execution_time [-371.977µs; -330.066µs] or [-3.976%; -3.528%]
• 🟩 throughput [+390879.272op/s; +439530.841op/s] or [+3.656%; +4.111%]

scenario:credit_card/is_card_number_no_luhn/ 378282246310005

• 🟩 execution_time [-436.810µs; -400.049µs] or [-5.089%; -4.661%]
• 🟩 throughput [+569567.241op/s; +619670.213op/s] or [+4.888%; +5.318%]

scenario:credit_card/is_card_number_no_luhn/37828224631

• 🟥 execution_time [+120.150µs; +120.178µs] or [+99.970%; +99.993%]
• 🟥 throughput [-416032814.593op/s; -415938126.403op/s] or [-50.001%; -49.990%]

scenario:credit_card/is_card_number_no_luhn/378282246310005

• 🟩 execution_time [-463.096µs; -428.167µs] or [-5.601%; -5.178%]
• 🟩 throughput [+660833.541op/s; +712116.848op/s] or [+5.463%; +5.887%]

scenario:credit_card/is_card_number_no_luhn/37828224631000521389798

• 🟥 execution_time [+728.675µs; +785.401µs] or [+7.561%; +8.150%]
• 🟥 throughput [-784576.389op/s; -729113.354op/s] or [-7.559%; -7.025%]

scenario:normalization/normalize_name/normalize_name/good

• 🟥 execution_time [+362.225ns; +372.929ns] or [+2.359%; +2.428%]
• 🟥 throughput [-1543990.251op/s; -1500077.792op/s] or [-2.371%; -2.304%]

scenario:normalization/normalize_service/normalize_service/[empty string]

• 🟥 execution_time [+1.223µs; +1.259µs] or [+2.800%; +2.882%]
• 🟥 throughput [-641417.916op/s; -623196.678op/s] or [-2.802%; -2.723%]

scenario:normalization/normalize_service/normalize_service/test_ASCII

• 🟥 execution_time [+1.510µs; +1.559µs] or [+3.002%; +3.101%]
• 🟥 throughput [-597848.768op/s; -579239.471op/s] or [-3.007%; -2.913%]

scenario:sql/obfuscate_sql_string

• 🟩 execution_time [-7.477µs; -7.412µs] or [-9.885%; -9.798%]

Candidate

Candidate benchmark details

Group 1

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	7f4735c	1726747207	vianney/benchmarks-with-dogstatsd

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
benching deserializing traces from msgpack to their internal representation	execution_time	1.193µs	1.385µs ± 0.049µs	1.395µs ± 0.027µs	1.419µs	1.446µs	1.471µs	1.472µs	5.49%	-1.226	2.788	3.55%	0.003µs	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
benching deserializing traces from msgpack to their internal representation	execution_time	[1.378µs; 1.392µs] or [-0.494%; +0.494%]	None	None	None

Group 2

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	7f4735c	1726747207	vianney/benchmarks-with-dogstatsd

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
two way interface	execution_time	18.249µs	25.007µs ± 10.734µs	20.136µs ± 1.434µs	33.854µs	43.230µs	46.893µs	99.904µs	396.14%	3.138	15.413	42.82%	0.759µs	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
two way interface	execution_time	[23.519µs; 26.494µs] or [-5.949%; +5.949%]	None	None	None

Group 3

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	7f4735c	1726747207	vianney/benchmarks-with-dogstatsd

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
normalization/normalize_trace/test_trace	execution_time	297.167ns	308.957ns ± 13.351ns	304.138ns ± 4.523ns	309.565ns	344.147ns	349.252ns	350.698ns	15.31%	1.736	2.059	4.31%	0.944ns	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
normalization/normalize_trace/test_trace	execution_time	[307.106ns; 310.807ns] or [-0.599%; +0.599%]	None	None	None

Group 4

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	7f4735c	1726747207	vianney/benchmarks-with-dogstatsd

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
sql/obfuscate_sql_string	execution_time	67.901µs	68.201µs ± 0.192µs	68.154µs ± 0.068µs	68.265µs	68.443µs	68.618µs	70.136µs	2.91%	5.407	49.851	0.28%	0.014µs	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
sql/obfuscate_sql_string	execution_time	[68.175µs; 68.228µs] or [-0.039%; +0.039%]	None	None	None

Group 5

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	7f4735c	1726747207	vianney/benchmarks-with-dogstatsd

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
credit_card/is_card_number/	execution_time	240.160µs	240.366µs ± 0.256µs	240.345µs ± 0.067µs	240.401µs	240.531µs	240.750µs	243.666µs	1.38%	10.861	136.276	0.11%	0.018µs	1	200
credit_card/is_card_number/	throughput	410397198.042op/s	416033442.219op/s ± 438509.210op/s	416068856.632op/s ± 115850.604op/s	416199103.311op/s	416266883.210op/s	416293915.938op/s	416389899.571op/s	0.08%	-10.797	135.120	0.11%	31007.284op/s	1	200
credit_card/is_card_number/ 3782-8224-6310-005	execution_time	11.337ms	11.667ms ± 0.138ms	11.658ms ± 0.087ms	11.745ms	11.899ms	11.975ms	12.195ms	4.61%	0.439	0.401	1.18%	0.010ms	1	200
credit_card/is_card_number/ 3782-8224-6310-005	throughput	8199882.299op/s	8572674.201op/s ± 101262.483op/s	8578077.735op/s ± 63961.889op/s	8642144.527op/s	8730012.632op/s	8781896.458op/s	8820681.888op/s	2.83%	-0.362	0.266	1.18%	7160.339op/s	1	200
credit_card/is_card_number/ 378282246310005	execution_time	10.407ms	10.827ms ± 0.168ms	10.827ms ± 0.118ms	10.952ms	11.089ms	11.204ms	11.247ms	3.88%	-0.070	-0.401	1.55%	0.012ms	1	200
credit_card/is_card_number/ 378282246310005	throughput	8891646.485op/s	9238223.828op/s ± 143352.482op/s	9236498.277op/s ± 100671.696op/s	9333895.963op/s	9493317.317op/s	9561284.589op/s	9609219.008op/s	4.04%	0.144	-0.388	1.55%	10136.551op/s	1	200
credit_card/is_card_number/37828224631	execution_time	240.204µs	240.335µs ± 0.089µs	240.329µs ± 0.069µs	240.389µs	240.501µs	240.609µs	240.682µs	0.15%	0.964	1.006	0.04%	0.006µs	1	200
credit_card/is_card_number/37828224631	throughput	415485295.233op/s	416086269.152op/s ± 153792.874op/s	416095546.635op/s ± 120061.475op/s	416219787.832op/s	416270713.928op/s	416301311.055op/s	416313337.242op/s	0.05%	-0.962	0.998	0.04%	10874.798op/s	1	200
credit_card/is_card_number/378282246310005	execution_time	10.100ms	10.490ms ± 0.146ms	10.485ms ± 0.101ms	10.589ms	10.714ms	10.814ms	10.839ms	3.37%	-0.127	-0.172	1.39%	0.010ms	1	200
credit_card/is_card_number/378282246310005	throughput	9226197.863op/s	9535101.684op/s ± 133191.087op/s	9537246.629op/s ± 91253.152op/s	9625395.490op/s	9765382.235op/s	9886658.058op/s	9901260.668op/s	3.82%	0.202	-0.133	1.39%	9418.032op/s	1	200
credit_card/is_card_number/37828224631000521389798	execution_time	10.221ms	10.420ms ± 0.141ms	10.373ms ± 0.085ms	10.516ms	10.665ms	10.757ms	11.032ms	6.35%	1.025	0.862	1.35%	0.010ms	1	200
credit_card/is_card_number/37828224631000521389798	throughput	9064196.903op/s	9598620.039op/s ± 128416.720op/s	9640141.338op/s ± 78814.658op/s	9701746.254op/s	9743930.997op/s	9777627.241op/s	9783506.651op/s	1.49%	-0.955	0.572	1.33%	9080.433op/s	1	200
credit_card/is_card_number/x371413321323331	execution_time	2.163ms	2.195ms ± 0.025ms	2.189ms ± 0.012ms	2.202ms	2.243ms	2.273ms	2.335ms	6.70%	1.766	5.055	1.14%	0.002ms	1	200
credit_card/is_card_number/x371413321323331	throughput	42821219.524op/s	45568030.900op/s ± 508892.639op/s	45692249.008op/s ± 253931.045op/s	45929077.421op/s	46180909.689op/s	46234526.927op/s	46238840.256op/s	1.20%	-1.639	4.264	1.11%	35984.144op/s	1	200
credit_card/is_card_number_no_luhn/	execution_time	240.178µs	240.337µs ± 0.081µs	240.338µs ± 0.063µs	240.392µs	240.481µs	240.538µs	240.553µs	0.09%	0.393	-0.539	0.03%	0.006µs	1	200
credit_card/is_card_number_no_luhn/	throughput	415709215.759op/s	416081971.618op/s ± 140191.410op/s	416080603.930op/s ± 109797.233op/s	416201340.768op/s	416267634.082op/s	416330439.368op/s	416358487.037op/s	0.07%	-0.391	-0.541	0.03%	9913.030op/s	1	200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005	execution_time	8.929ms	9.005ms ± 0.072ms	8.964ms ± 0.021ms	9.066ms	9.138ms	9.207ms	9.292ms	3.66%	1.272	1.075	0.80%	0.005ms	1	200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005	throughput	10762027.992op/s	11105803.917op/s ± 88035.525op/s	11155698.585op/s ± 25700.982op/s	11169514.331op/s	11186338.244op/s	11196480.152op/s	11199844.367op/s	0.40%	-1.238	0.924	0.79%	6225.052op/s	1	200
credit_card/is_card_number_no_luhn/ 378282246310005	execution_time	8.135ms	8.165ms ± 0.034ms	8.158ms ± 0.008ms	8.165ms	8.247ms	8.307ms	8.382ms	2.75%	3.252	12.621	0.41%	0.002ms	1	200
credit_card/is_card_number_no_luhn/ 378282246310005	throughput	11930375.284op/s	12247094.520op/s ± 50059.054op/s	12258655.051op/s ± 12327.762op/s	12270994.671op/s	12284300.694op/s	12289385.534op/s	12291818.842op/s	0.27%	-3.203	12.185	0.41%	3539.710op/s	1	200
credit_card/is_card_number_no_luhn/37828224631	execution_time	240.217µs	240.349µs ± 0.089µs	240.353µs ± 0.066µs	240.399µs	240.503µs	240.634µs	240.712µs	0.15%	0.929	1.122	0.04%	0.006µs	1	200
credit_card/is_card_number_no_luhn/37828224631	throughput	415434910.400op/s	416061131.555op/s ± 154153.302op/s	416054369.206op/s ± 113618.183op/s	416189483.546op/s	416251334.141op/s	416277297.484op/s	416289982.051op/s	0.06%	-0.926	1.113	0.04%	10900.285op/s	1	200
credit_card/is_card_number_no_luhn/378282246310005	execution_time	7.783ms	7.823ms ± 0.029ms	7.816ms ± 0.009ms	7.826ms	7.884ms	7.932ms	7.974ms	2.02%	2.635	8.367	0.37%	0.002ms	1	200
credit_card/is_card_number_no_luhn/378282246310005	throughput	12541134.136op/s	12783268.023op/s ± 46480.854op/s	12794686.425op/s ± 14347.833op/s	12807149.673op/s	12824844.130op/s	12844668.584op/s	12848115.679op/s	0.42%	-2.598	8.137	0.36%	3286.693op/s	1	200
credit_card/is_card_number_no_luhn/37828224631000521389798	execution_time	10.219ms	10.394ms ± 0.133ms	10.353ms ± 0.069ms	10.464ms	10.660ms	10.808ms	10.916ms	5.44%	1.452	2.186	1.27%	0.009ms	1	200
credit_card/is_card_number_no_luhn/37828224631000521389798	throughput	9160968.010op/s	9622625.096op/s ± 120513.297op/s	9658880.225op/s ± 65105.459op/s	9711518.148op/s	9752427.897op/s	9767722.214op/s	9785703.850op/s	1.31%	-1.375	1.882	1.25%	8521.577op/s	1	200
credit_card/is_card_number_no_luhn/x371413321323331	execution_time	2.162ms	2.193ms ± 0.020ms	2.188ms ± 0.012ms	2.206ms	2.228ms	2.259ms	2.276ms	4.02%	1.102	1.720	0.92%	0.001ms	1	200
credit_card/is_card_number_no_luhn/x371413321323331	throughput	43936067.047op/s	45605107.782op/s ± 417078.878op/s	45700438.745op/s ± 258889.504op/s	45904138.997op/s	46181879.629op/s	46234894.385op/s	46251402.458op/s	1.21%	-1.035	1.468	0.91%	29491.930op/s	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
credit_card/is_card_number/	execution_time	[240.330µs; 240.401µs] or [-0.015%; +0.015%]	None	None	None
credit_card/is_card_number/	throughput	[415972669.060op/s; 416094215.378op/s] or [-0.015%; +0.015%]	None	None	None
credit_card/is_card_number/ 3782-8224-6310-005	execution_time	[11.647ms; 11.686ms] or [-0.164%; +0.164%]	None	None	None
credit_card/is_card_number/ 3782-8224-6310-005	throughput	[8558640.194op/s; 8586708.207op/s] or [-0.164%; +0.164%]	None	None	None
credit_card/is_card_number/ 378282246310005	execution_time	[10.804ms; 10.850ms] or [-0.215%; +0.215%]	None	None	None
credit_card/is_card_number/ 378282246310005	throughput	[9218356.553op/s; 9258091.104op/s] or [-0.215%; +0.215%]	None	None	None
credit_card/is_card_number/37828224631	execution_time	[240.322µs; 240.347µs] or [-0.005%; +0.005%]	None	None	None
credit_card/is_card_number/37828224631	throughput	[416064954.939op/s; 416107583.366op/s] or [-0.005%; +0.005%]	None	None	None
credit_card/is_card_number/378282246310005	execution_time	[10.469ms; 10.510ms] or [-0.193%; +0.193%]	None	None	None
credit_card/is_card_number/378282246310005	throughput	[9516642.680op/s; 9553560.688op/s] or [-0.194%; +0.194%]	None	None	None
credit_card/is_card_number/37828224631000521389798	execution_time	[10.400ms; 10.440ms] or [-0.188%; +0.188%]	None	None	None
credit_card/is_card_number/37828224631000521389798	throughput	[9580822.717op/s; 9616417.362op/s] or [-0.185%; +0.185%]	None	None	None
credit_card/is_card_number/x371413321323331	execution_time	[2.191ms; 2.198ms] or [-0.158%; +0.158%]	None	None	None
credit_card/is_card_number/x371413321323331	throughput	[45497503.275op/s; 45638558.526op/s] or [-0.155%; +0.155%]	None	None	None
credit_card/is_card_number_no_luhn/	execution_time	[240.326µs; 240.349µs] or [-0.005%; +0.005%]	None	None	None
credit_card/is_card_number_no_luhn/	throughput	[416062542.437op/s; 416101400.799op/s] or [-0.005%; +0.005%]	None	None	None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005	execution_time	[8.995ms; 9.015ms] or [-0.111%; +0.111%]	None	None	None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005	throughput	[11093603.040op/s; 11118004.794op/s] or [-0.110%; +0.110%]	None	None	None
credit_card/is_card_number_no_luhn/ 378282246310005	execution_time	[8.161ms; 8.170ms] or [-0.057%; +0.057%]	None	None	None
credit_card/is_card_number_no_luhn/ 378282246310005	throughput	[12240156.816op/s; 12254032.223op/s] or [-0.057%; +0.057%]	None	None	None
credit_card/is_card_number_no_luhn/37828224631	execution_time	[240.337µs; 240.362µs] or [-0.005%; +0.005%]	None	None	None
credit_card/is_card_number_no_luhn/37828224631	throughput	[416039767.390op/s; 416082495.721op/s] or [-0.005%; +0.005%]	None	None	None
credit_card/is_card_number_no_luhn/378282246310005	execution_time	[7.819ms; 7.827ms] or [-0.051%; +0.051%]	None	None	None
credit_card/is_card_number_no_luhn/378282246310005	throughput	[12776826.224op/s; 12789709.822op/s] or [-0.050%; +0.050%]	None	None	None
credit_card/is_card_number_no_luhn/37828224631000521389798	execution_time	[10.375ms; 10.412ms] or [-0.177%; +0.177%]	None	None	None
credit_card/is_card_number_no_luhn/37828224631000521389798	throughput	[9605923.112op/s; 9639327.080op/s] or [-0.174%; +0.174%]	None	None	None
credit_card/is_card_number_no_luhn/x371413321323331	execution_time	[2.190ms; 2.196ms] or [-0.128%; +0.128%]	None	None	None
credit_card/is_card_number_no_luhn/x371413321323331	throughput	[45547304.661op/s; 45662910.903op/s] or [-0.127%; +0.127%]	None	None	None

Group 6

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	7f4735c	1726747207	vianney/benchmarks-with-dogstatsd

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo...	execution_time	273.311µs	273.760µs ± 0.196µs	273.711µs ± 0.108µs	273.872µs	274.128µs	274.302µs	274.454µs	0.27%	0.909	0.706	0.07%	0.014µs	1	200
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo...	throughput	3643592.341op/s	3652839.120op/s ± 2619.033op/s	3653494.287op/s ± 1446.877op/s	3654568.773op/s	3656329.741op/s	3657200.950op/s	3658841.540op/s	0.15%	-0.905	0.697	0.07%	185.194op/s	1	200
normalization/normalize_name/normalize_name/bad-name	execution_time	25.745µs	25.786µs ± 0.032µs	25.788µs ± 0.024µs	25.802µs	25.844µs	25.884µs	25.893µs	0.41%	0.876	0.748	0.12%	0.002µs	1	200
normalization/normalize_name/normalize_name/bad-name	throughput	38620323.241op/s	38780959.411op/s ± 47633.864op/s	38777782.289op/s ± 36650.805op/s	38824249.337op/s	38839780.262op/s	38841634.713op/s	38842000.182op/s	0.17%	-0.869	0.726	0.12%	3368.223op/s	1	200
normalization/normalize_name/normalize_name/good	execution_time	15.698µs	15.725µs ± 0.030µs	15.731µs ± 0.021µs	15.742µs	15.752µs	15.787µs	16.012µs	1.79%	4.535	40.651	0.19%	0.002µs	1	200
normalization/normalize_name/normalize_name/good	throughput	62453291.236op/s	63593457.015op/s ± 119792.740op/s	63569989.241op/s ± 85237.764op/s	63686393.215op/s	63695970.462op/s	63699674.982op/s	63703985.246op/s	0.21%	-4.411	39.070	0.19%	8470.626op/s	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo...	execution_time	[273.733µs; 273.787µs] or [-0.010%; +0.010%]	None	None	None
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo...	throughput	[3652476.147op/s; 3653202.092op/s] or [-0.010%; +0.010%]	None	None	None
normalization/normalize_name/normalize_name/bad-name	execution_time	[25.781µs; 25.790µs] or [-0.017%; +0.017%]	None	None	None
normalization/normalize_name/normalize_name/bad-name	throughput	[38774357.816op/s; 38787561.007op/s] or [-0.017%; +0.017%]	None	None	None
normalization/normalize_name/normalize_name/good	execution_time	[15.721µs; 15.729µs] or [-0.026%; +0.026%]	None	None	None
normalization/normalize_name/normalize_name/good	throughput	[63576854.894op/s; 63610059.137op/s] or [-0.026%; +0.026%]	None	None	None

Group 7

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	7f4735c	1726747207	vianney/benchmarks-with-dogstatsd

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000...	execution_time	507.765µs	510.193µs ± 1.182µs	510.157µs ± 0.911µs	511.105µs	511.953µs	512.510µs	514.711µs	0.89%	0.208	-0.157	0.23%	0.084µs	1	200
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000...	throughput	1942839.158op/s	1960054.157op/s ± 4538.832op/s	1960181.564op/s ± 3492.642op/s	1963401.392op/s	1967212.697op/s	1968417.793op/s	1969416.582op/s	0.47%	-0.195	-0.185	0.23%	320.944op/s	1	200
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて	execution_time	471.669µs	472.620µs ± 0.365µs	472.643µs ± 0.265µs	472.910µs	473.215µs	473.344µs	473.422µs	0.16%	-0.275	-0.362	0.08%	0.026µs	1	200
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて	throughput	2112282.254op/s	2115863.881op/s ± 1633.903op/s	2115760.377op/s ± 1186.613op/s	2116915.458op/s	2118593.598op/s	2119703.539op/s	2120131.091op/s	0.21%	0.278	-0.359	0.08%	115.534op/s	1	200
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters	execution_time	182.583µs	182.847µs ± 0.163µs	182.837µs ± 0.107µs	182.941µs	183.101µs	183.391µs	183.514µs	0.37%	0.743	1.663	0.09%	0.011µs	1	200
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters	throughput	5449165.147op/s	5469069.246op/s ± 4858.546op/s	5469348.370op/s ± 3186.193op/s	5472571.982op/s	5476656.147op/s	5476892.178op/s	5476950.664op/s	0.14%	-0.735	1.638	0.09%	343.551op/s	1	200
normalization/normalize_service/normalize_service/[empty string]	execution_time	44.773µs	44.929µs ± 0.092µs	44.911µs ± 0.056µs	44.981µs	45.098µs	45.191µs	45.280µs	0.82%	0.984	1.003	0.20%	0.007µs	1	200
normalization/normalize_service/normalize_service/[empty string]	throughput	22084812.621op/s	22257297.581op/s ± 45457.712op/s	22266382.114op/s ± 27891.307op/s	22289729.144op/s	22316664.576op/s	22324666.814op/s	22334643.473op/s	0.31%	-0.972	0.965	0.20%	3214.346op/s	1	200
normalization/normalize_service/normalize_service/test_ASCII	execution_time	51.219µs	51.832µs ± 0.159µs	51.831µs ± 0.084µs	51.913µs	52.003µs	52.089µs	53.193µs	2.63%	2.682	26.326	0.31%	0.011µs	1	200
normalization/normalize_service/normalize_service/test_ASCII	throughput	18799474.411op/s	19293183.846op/s ± 58707.181op/s	19293405.182op/s ± 31195.478op/s	19324555.628op/s	19360135.718op/s	19408664.254op/s	19524013.733op/s	1.20%	-2.493	24.615	0.30%	4151.225op/s	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000...	execution_time	[510.029µs; 510.357µs] or [-0.032%; +0.032%]	None	None	None
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000...	throughput	[1959425.118op/s; 1960683.195op/s] or [-0.032%; +0.032%]	None	None	None
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて	execution_time	[472.570µs; 472.671µs] or [-0.011%; +0.011%]	None	None	None
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて	throughput	[2115637.438op/s; 2116090.325op/s] or [-0.011%; +0.011%]	None	None	None
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters	execution_time	[182.824µs; 182.869µs] or [-0.012%; +0.012%]	None	None	None
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters	throughput	[5468395.898op/s; 5469742.593op/s] or [-0.012%; +0.012%]	None	None	None
normalization/normalize_service/normalize_service/[empty string]	execution_time	[44.917µs; 44.942µs] or [-0.028%; +0.028%]	None	None	None
normalization/normalize_service/normalize_service/[empty string]	throughput	[22250997.579op/s; 22263597.583op/s] or [-0.028%; +0.028%]	None	None	None
normalization/normalize_service/normalize_service/test_ASCII	execution_time	[51.810µs; 51.854µs] or [-0.043%; +0.043%]	None	None	None
normalization/normalize_service/normalize_service/test_ASCII	throughput	[19285047.596op/s; 19301320.097op/s] or [-0.042%; +0.042%]	None	None	None

Group 8

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	7f4735c	1726747207	vianney/benchmarks-with-dogstatsd

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
benching string interning on wordpress profile	execution_time	138.317µs	139.733µs ± 0.374µs	139.704µs ± 0.191µs	139.920µs	140.390µs	140.941µs	140.988µs	0.92%	0.218	3.071	0.27%	0.026µs	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
benching string interning on wordpress profile	execution_time	[139.681µs; 139.784µs] or [-0.037%; +0.037%]	None	None	None

Group 9

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	7f4735c	1726747207	vianney/benchmarks-with-dogstatsd

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
write only interface	execution_time	1.371µs	3.194µs ± 1.381µs	3.070µs ± 0.033µs	3.097µs	3.160µs	13.724µs	14.920µs	386.04%	7.643	58.726	43.14%	0.098µs	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
write only interface	execution_time	[3.002µs; 3.385µs] or [-5.994%; +5.994%]	None	None	None

Group 10

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	7f4735c	1726747207	vianney/benchmarks-with-dogstatsd

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
concentrator/add_spans_to_concentrator	execution_time	9.066ms	9.113ms ± 0.025ms	9.112ms ± 0.011ms	9.121ms	9.134ms	9.148ms	9.387ms	3.01%	6.764	73.601	0.27%	0.002ms	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
concentrator/add_spans_to_concentrator	execution_time	[9.109ms; 9.116ms] or [-0.038%; +0.038%]	None	None	None

Group 11

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	7f4735c	1726747207	vianney/benchmarks-with-dogstatsd

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
tags/replace_trace_tags	execution_time	2.665µs	2.712µs ± 0.019µs	2.713µs ± 0.011µs	2.723µs	2.745µs	2.759µs	2.768µs	2.02%	-0.016	0.303	0.69%	0.001µs	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
tags/replace_trace_tags	execution_time	[2.710µs; 2.715µs] or [-0.096%; +0.096%]	None	None	None

Group 12

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	7f4735c	1726747207	vianney/benchmarks-with-dogstatsd

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
redis/obfuscate_redis_string	execution_time	36.756µs	37.489µs ± 1.356µs	36.861µs ± 0.051µs	36.964µs	40.405µs	40.444µs	41.420µs	12.37%	1.712	0.983	3.61%	0.096µs	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
redis/obfuscate_redis_string	execution_time	[37.301µs; 37.677µs] or [-0.501%; +0.501%]	None	None	None

Baseline

Omitted due to size.

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 81.34111% with 64 lines in your changes missing coverage. Please review.

Project coverage is 73.11%. Comparing base (89f48d4) to head (7f4735c).
Report is 1 commits behind head on vianney/main-with-iterations.

Additional details and impacted files

@@                       Coverage Diff                        @@
##           vianney/main-with-iterations     #641      +/-   ##
================================================================
+ Coverage                         73.05%   73.11%   +0.06%     
================================================================
  Files                               252      252              
  Lines                             36093    36276     +183     
================================================================
+ Hits                              26366    26522     +156     
- Misses                             9727     9754      +27     

Components	Coverage Δ
crashtracker	20.53% <ø> (ø)
datadog-alloc	98.73% <ø> (ø)
data-pipeline	93.95% <89.67%> (+3.83%)	⬆️
data-pipeline-ffi	0.00% <ø> (ø)
ddcommon	83.08% <ø> (ø)
ddcommon-ffi	69.52% <ø> (ø)
ddtelemetry	59.10% <ø> (ø)
ipc	83.63% <ø> (ø)
profiling	83.57% <ø> (-0.70%)	⬇️
profiling-ffi	77.42% <ø> (ø)
serverless	0.00% <ø> (ø)
sidecar	38.46% <0.00%> (-1.67%)	⬇️
sidecar-ffi	0.00% <0.00%> (ø)
spawn-worker	50.36% <ø> (ø)
tinybytes	91.66% <ø> (ø)
trace-mini-agent	72.49% <ø> (ø)
trace-normalization	98.25% <ø> (ø)
trace-obfuscation	95.73% <ø> (ø)
trace-protobuf	77.67% <ø> (ø)
trace-utils	92.88% <ø> (ø)