Increase test delay

[VianneyRuhlmann]

What does this PR do?

A brief description of the change being made with this pull request.

Motivation

What inspired you to submit this pull request?

Additional Notes

Anything else we should know when reviewing?

How to test the change?

Describe here in detail how the change can be validated.

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 73.85%. Comparing base (b8e5f6d) to head (e4efeb3).

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #653   +/-   ##
=======================================
  Coverage   73.85%   73.85%           
=======================================
  Files         257      257           
  Lines       37301    37301           
=======================================
  Hits        27549    27549           
  Misses       9752     9752           

Components	Coverage Δ
crashtracker	20.53% <ø> (ø)
crashtracker-ffi	9.28% <ø> (ø)
datadog-alloc	98.73% <ø> (ø)
data-pipeline	93.33% <100.00%> (ø)
data-pipeline-ffi	0.00% <ø> (ø)
ddcommon	83.08% <ø> (ø)
ddcommon-ffi	69.52% <ø> (ø)
ddtelemetry	59.10% <ø> (ø)
ddtelemetry-ffi	22.13% <ø> (ø)
dogstatsd	90.42% <ø> (ø)
dogstatsd-client	79.77% <ø> (ø)
ipc	83.58% <ø> (ø)
profiling	84.26% <ø> (ø)
profiling-ffi	77.42% <ø> (ø)
serverless	0.00% <ø> (ø)
sidecar	38.19% <ø> (ø)
sidecar-ffi	0.00% <ø> (ø)
spawn-worker	50.36% <ø> (ø)
tinybytes	94.77% <ø> (ø)
trace-mini-agent	72.47% <ø> (ø)
trace-normalization	98.25% <ø> (ø)
trace-obfuscation	95.73% <ø> (ø)
trace-protobuf	77.67% <ø> (ø)
trace-utils	93.87% <ø> (ø)

[pr-commenter]

Benchmarks

Comparison

Benchmark execution time: 2024-09-30 09:01:55

Comparing candidate commit e4efeb3 in PR branch vianney/increase_test_run_delay with baseline commit b8e5f6d in branch main.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 51 metrics, 2 unstable metrics.

Candidate

Candidate benchmark details

Group 1

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	e4efeb3	1727685524	vianney/increase_test_run_delay

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000...	execution_time	500.587µs	502.702µs ± 0.333µs	502.707µs ± 0.164µs	502.851µs	503.133µs	503.317µs	504.622µs	0.38%	-0.529	12.852	0.07%	0.024µs	1	200
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000...	throughput	1981681.774op/s	1989250.845op/s ± 1316.489op/s	1989229.538op/s ± 647.863op/s	1989953.114op/s	1990619.835op/s	1992976.204op/s	1997655.640op/s	0.42%	0.558	12.893	0.07%	93.090op/s	1	200
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて	execution_time	464.764µs	465.338µs ± 0.270µs	465.299µs ± 0.179µs	465.501µs	465.778µs	466.197µs	466.288µs	0.21%	0.593	0.785	0.06%	0.019µs	1	200
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて	throughput	2144596.362op/s	2148978.546op/s ± 1245.158op/s	2149155.947op/s ± 827.632op/s	2149820.723op/s	2150849.015op/s	2151365.461op/s	2151628.913op/s	0.12%	-0.589	0.775	0.06%	88.046op/s	1	200
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters	execution_time	178.543µs	180.059µs ± 0.732µs	179.961µs ± 0.490µs	180.497µs	181.321µs	181.821µs	182.216µs	1.25%	0.433	-0.204	0.41%	0.052µs	1	200
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters	throughput	5488002.557op/s	5553812.615op/s ± 22551.705op/s	5556761.403op/s ± 15074.333op/s	5570354.735op/s	5587542.038op/s	5598166.555op/s	5600884.776op/s	0.79%	-0.414	-0.226	0.41%	1594.646op/s	1	200
normalization/normalize_service/normalize_service/[empty string]	execution_time	44.409µs	44.802µs ± 0.143µs	44.803µs ± 0.107µs	44.922µs	45.013µs	45.128µs	45.149µs	0.77%	-0.110	-0.404	0.32%	0.010µs	1	200
normalization/normalize_service/normalize_service/[empty string]	throughput	22148782.601op/s	22320600.018op/s ± 71385.085op/s	22319724.636op/s ± 53237.462op/s	22367027.150op/s	22440340.789op/s	22476837.376op/s	22517951.029op/s	0.89%	0.125	-0.400	0.32%	5047.688op/s	1	200
normalization/normalize_service/normalize_service/test_ASCII	execution_time	50.260µs	50.747µs ± 0.172µs	50.749µs ± 0.121µs	50.874µs	50.995µs	51.118µs	51.142µs	0.77%	-0.441	0.034	0.34%	0.012µs	1	200
normalization/normalize_service/normalize_service/test_ASCII	throughput	19553242.905op/s	19705862.336op/s ± 66982.510op/s	19704705.295op/s ± 46945.394op/s	19748469.552op/s	19829824.440op/s	19877865.173op/s	19896486.772op/s	0.97%	0.460	0.055	0.34%	4736.379op/s	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000...	execution_time	[502.656µs; 502.748µs] or [-0.009%; +0.009%]	None	None	None
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000...	throughput	[1989068.392op/s; 1989433.297op/s] or [-0.009%; +0.009%]	None	None	None
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて	execution_time	[465.300µs; 465.375µs] or [-0.008%; +0.008%]	None	None	None
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて	throughput	[2148805.979op/s; 2149151.113op/s] or [-0.008%; +0.008%]	None	None	None
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters	execution_time	[179.958µs; 180.161µs] or [-0.056%; +0.056%]	None	None	None
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters	throughput	[5550687.166op/s; 5556938.065op/s] or [-0.056%; +0.056%]	None	None	None
normalization/normalize_service/normalize_service/[empty string]	execution_time	[44.782µs; 44.822µs] or [-0.044%; +0.044%]	None	None	None
normalization/normalize_service/normalize_service/[empty string]	throughput	[22310706.732op/s; 22330493.305op/s] or [-0.044%; +0.044%]	None	None	None
normalization/normalize_service/normalize_service/test_ASCII	execution_time	[50.723µs; 50.771µs] or [-0.047%; +0.047%]	None	None	None
normalization/normalize_service/normalize_service/test_ASCII	throughput	[19696579.204op/s; 19715145.468op/s] or [-0.047%; +0.047%]	None	None	None

Group 2

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	e4efeb3	1727685524	vianney/increase_test_run_delay

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
benching deserializing traces from msgpack to their internal representation	execution_time	1.143µs	1.203µs ± 0.021µs	1.209µs ± 0.011µs	1.220µs	1.223µs	1.224µs	1.224µs	1.22%	-1.400	1.178	1.77%	0.002µs	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
benching deserializing traces from msgpack to their internal representation	execution_time	[1.200µs; 1.206µs] or [-0.246%; +0.246%]	None	None	None

Group 3

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	e4efeb3	1727685524	vianney/increase_test_run_delay

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
benching string interning on wordpress profile	execution_time	140.219µs	141.723µs ± 0.622µs	141.694µs ± 0.258µs	141.942µs	142.345µs	143.282µs	148.171µs	4.57%	5.629	56.262	0.44%	0.044µs	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
benching string interning on wordpress profile	execution_time	[141.637µs; 141.809µs] or [-0.061%; +0.061%]	None	None	None

Group 4

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	e4efeb3	1727685524	vianney/increase_test_run_delay

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
redis/obfuscate_redis_string	execution_time	38.226µs	38.759µs ± 0.956µs	38.321µs ± 0.031µs	38.370µs	40.807µs	40.830µs	41.758µs	8.97%	1.718	1.040	2.46%	0.068µs	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
redis/obfuscate_redis_string	execution_time	[38.627µs; 38.892µs] or [-0.342%; +0.342%]	None	None	None

Group 5

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	e4efeb3	1727685524	vianney/increase_test_run_delay

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
concentrator/add_spans_to_concentrator	execution_time	9.062ms	9.093ms ± 0.014ms	9.094ms ± 0.010ms	9.102ms	9.117ms	9.131ms	9.142ms	0.53%	0.337	0.253	0.16%	0.001ms	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
concentrator/add_spans_to_concentrator	execution_time	[9.091ms; 9.095ms] or [-0.022%; +0.022%]	None	None	None

Group 6

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	e4efeb3	1727685524	vianney/increase_test_run_delay

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo...	execution_time	318.278µs	318.541µs ± 0.166µs	318.509µs ± 0.052µs	318.593µs	318.780µs	318.952µs	319.864µs	0.43%	2.869	19.294	0.05%	0.012µs	1	200
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo...	throughput	3126324.179op/s	3139318.421op/s ± 1629.304op/s	3139631.551op/s ± 508.409op/s	3140023.048op/s	3141598.932op/s	3141698.293op/s	3141908.148op/s	0.07%	-2.848	19.087	0.05%	115.209op/s	1	200
normalization/normalize_name/normalize_name/bad-name	execution_time	29.293µs	29.533µs ± 0.182µs	29.430µs ± 0.112µs	29.737µs	29.767µs	29.793µs	29.801µs	1.26%	0.211	-1.784	0.61%	0.013µs	1	200
normalization/normalize_name/normalize_name/bad-name	throughput	33555822.554op/s	33862144.525op/s ± 208295.294op/s	33978961.085op/s ± 130059.974op/s	34047102.184op/s	34114825.044op/s	34128765.400op/s	34138357.793op/s	0.47%	-0.208	-1.783	0.61%	14728.702op/s	1	200
normalization/normalize_name/normalize_name/good	execution_time	17.280µs	17.300µs ± 0.027µs	17.287µs ± 0.005µs	17.322µs	17.335µs	17.426µs	17.460µs	1.00%	2.632	9.767	0.16%	0.002µs	1	200
normalization/normalize_name/normalize_name/good	throughput	57274077.018op/s	57802159.600op/s ± 90892.093op/s	57845435.513op/s ± 15261.644op/s	57857034.774op/s	57864596.103op/s	57867440.591op/s	57869568.916op/s	0.04%	-2.609	9.588	0.16%	6427.042op/s	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo...	execution_time	[318.518µs; 318.564µs] or [-0.007%; +0.007%]	None	None	None
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo...	throughput	[3139092.615op/s; 3139544.227op/s] or [-0.007%; +0.007%]	None	None	None
normalization/normalize_name/normalize_name/bad-name	execution_time	[29.507µs; 29.558µs] or [-0.085%; +0.085%]	None	None	None
normalization/normalize_name/normalize_name/bad-name	throughput	[33833276.800op/s; 33891012.249op/s] or [-0.085%; +0.085%]	None	None	None
normalization/normalize_name/normalize_name/good	execution_time	[17.297µs; 17.304µs] or [-0.022%; +0.022%]	None	None	None
normalization/normalize_name/normalize_name/good	throughput	[57789562.830op/s; 57814756.370op/s] or [-0.022%; +0.022%]	None	None	None

Group 7

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	e4efeb3	1727685524	vianney/increase_test_run_delay

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
write only interface	execution_time	1.365µs	3.232µs ± 1.613µs	3.060µs ± 0.033µs	3.091µs	3.131µs	13.857µs	18.700µs	511.16%	7.813	63.048	49.79%	0.114µs	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
write only interface	execution_time	[3.008µs; 3.455µs] or [-6.918%; +6.918%]	None	None	None

Group 8

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	e4efeb3	1727685524	vianney/increase_test_run_delay

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
credit_card/is_card_number/	execution_time	2.015µs	2.017µs ± 0.002µs	2.016µs ± 0.001µs	2.017µs	2.018µs	2.019µs	2.044µs	1.36%	10.993	139.613	0.10%	0.000µs	1	200
credit_card/is_card_number/	throughput	489290071.673op/s	495903578.180op/s ± 511269.626op/s	495959529.716op/s ± 144436.598op/s	496102792.500op/s	496169987.838op/s	496220761.190op/s	496253337.329op/s	0.06%	-10.928	138.475	0.10%	36152.222op/s	1	200
credit_card/is_card_number/ 3782-8224-6310-005	execution_time	111.285µs	112.789µs ± 0.772µs	112.671µs ± 0.445µs	113.174µs	114.261µs	114.701µs	116.260µs	3.19%	1.227	2.530	0.68%	0.055µs	1	200
credit_card/is_card_number/ 3782-8224-6310-005	throughput	8601400.656op/s	8866539.045op/s ± 60192.400op/s	8875370.406op/s ± 35153.384op/s	8908412.221op/s	8940463.465op/s	8951420.474op/s	8985932.472op/s	1.25%	-1.167	2.263	0.68%	4256.245op/s	1	200
credit_card/is_card_number/ 378282246310005	execution_time	103.375µs	104.457µs ± 0.616µs	104.362µs ± 0.401µs	104.772µs	105.522µs	106.284µs	106.839µs	2.37%	0.967	1.252	0.59%	0.044µs	1	200
credit_card/is_card_number/ 378282246310005	throughput	9359900.762op/s	9573680.466op/s ± 56114.753op/s	9582039.909op/s ± 36900.266op/s	9618662.371op/s	9648397.084op/s	9660803.844op/s	9673482.676op/s	0.95%	-0.926	1.121	0.58%	3967.912op/s	1	200
credit_card/is_card_number/37828224631	execution_time	2.015µs	2.016µs ± 0.002µs	2.016µs ± 0.001µs	2.017µs	2.018µs	2.018µs	2.039µs	1.12%	10.408	129.645	0.09%	0.000µs	1	200
credit_card/is_card_number/37828224631	throughput	490463064.374op/s	495928350.135op/s ± 430238.917op/s	495972572.722op/s ± 143724.571op/s	496116463.343op/s	496181992.112op/s	496211040.044op/s	496234669.375op/s	0.05%	-10.347	128.596	0.09%	30422.486op/s	1	200
credit_card/is_card_number/378282246310005	execution_time	99.934µs	100.978µs ± 0.549µs	100.884µs ± 0.316µs	101.250µs	101.836µs	102.427µs	104.335µs	3.42%	1.533	6.239	0.54%	0.039µs	1	200
credit_card/is_card_number/378282246310005	throughput	9584516.069op/s	9903431.762op/s ± 53392.781op/s	9912378.578op/s ± 31164.350op/s	9941687.235op/s	9971681.485op/s	9995399.692op/s	10006557.833op/s	0.95%	-1.440	5.609	0.54%	3775.440op/s	1	200
credit_card/is_card_number/37828224631000521389798	execution_time	100.502µs	100.901µs ± 0.386µs	100.775µs ± 0.200µs	101.146µs	101.511µs	102.321µs	102.755µs	1.97%	1.670	3.886	0.38%	0.027µs	1	200
credit_card/is_card_number/37828224631000521389798	throughput	9731865.690op/s	9910823.991op/s ± 37696.627op/s	9923140.325op/s ± 19723.012op/s	9939249.076op/s	9947424.381op/s	9949435.972op/s	9950088.367op/s	0.27%	-1.635	3.685	0.38%	2665.554op/s	1	200
credit_card/is_card_number/x371413321323331	execution_time	20.836µs	20.858µs ± 0.027µs	20.851µs ± 0.008µs	20.860µs	20.902µs	20.970µs	21.026µs	0.84%	3.380	14.428	0.13%	0.002µs	1	200
credit_card/is_card_number/x371413321323331	throughput	47560790.369op/s	47943181.921op/s ± 61609.793op/s	47959083.709op/s ± 18999.902op/s	47977493.758op/s	47988095.353op/s	47991979.820op/s	47994495.031op/s	0.07%	-3.361	14.265	0.13%	4356.470op/s	1	200
credit_card/is_card_number_no_luhn/	execution_time	2.015µs	2.016µs ± 0.001µs	2.016µs ± 0.000µs	2.017µs	2.017µs	2.017µs	2.019µs	0.12%	0.458	0.799	0.03%	0.000µs	1	200
credit_card/is_card_number_no_luhn/	throughput	495352227.892op/s	495968774.056op/s ± 140183.223op/s	495956950.937op/s ± 110241.601op/s	496093079.464op/s	496171037.599op/s	496219904.318op/s	496230151.544op/s	0.06%	-0.455	0.791	0.03%	9912.451op/s	1	200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005	execution_time	87.496µs	89.739µs ± 0.859µs	89.584µs ± 0.435µs	90.117µs	91.268µs	92.113µs	92.611µs	3.38%	0.626	0.868	0.95%	0.061µs	1	200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005	throughput	10797900.233op/s	11144384.810op/s ± 106029.001op/s	11162694.283op/s ± 54157.508op/s	11208839.858op/s	11301430.322op/s	11376021.570op/s	11429156.881op/s	2.39%	-0.556	0.795	0.95%	7497.383op/s	1	200
credit_card/is_card_number_no_luhn/ 378282246310005	execution_time	80.949µs	81.430µs ± 0.325µs	81.356µs ± 0.054µs	81.443µs	82.077µs	82.466µs	83.359µs	2.46%	2.362	8.630	0.40%	0.023µs	1	200
credit_card/is_card_number_no_luhn/ 378282246310005	throughput	11996321.518op/s	12280743.148op/s ± 48626.713op/s	12291609.133op/s ± 8164.553op/s	12298424.879op/s	12340850.251op/s	12347537.315op/s	12353381.432op/s	0.50%	-2.303	8.226	0.39%	3438.428op/s	1	200
credit_card/is_card_number_no_luhn/37828224631	execution_time	2.015µs	2.016µs ± 0.002µs	2.016µs ± 0.001µs	2.017µs	2.018µs	2.018µs	2.043µs	1.30%	11.165	142.858	0.10%	0.000µs	1	200
credit_card/is_card_number_no_luhn/37828224631	throughput	489589075.846op/s	495911469.605op/s ± 485907.516op/s	495967439.764op/s ± 136592.457op/s	496098115.805op/s	496169995.284op/s	496218445.289op/s	496247031.772op/s	0.06%	-11.105	141.807	0.10%	34358.850op/s	1	200
credit_card/is_card_number_no_luhn/378282246310005	execution_time	77.288µs	77.818µs ± 0.221µs	77.773µs ± 0.059µs	77.854µs	78.225µs	78.517µs	79.156µs	1.78%	1.756	7.014	0.28%	0.016µs	1	200
credit_card/is_card_number_no_luhn/378282246310005	throughput	12633316.995op/s	12850551.318op/s ± 36326.324op/s	12857914.760op/s ± 9776.271op/s	12866794.343op/s	12902008.265op/s	12920017.088op/s	12938537.434op/s	0.63%	-1.706	6.717	0.28%	2568.659op/s	1	200
credit_card/is_card_number_no_luhn/37828224631000521389798	execution_time	100.492µs	101.078µs ± 0.470µs	100.949µs ± 0.294µs	101.344µs	101.970µs	102.320µs	103.473µs	2.50%	1.320	2.719	0.46%	0.033µs	1	200
credit_card/is_card_number_no_luhn/37828224631000521389798	throughput	9664372.924op/s	9893534.148op/s ± 45679.762op/s	9905960.443op/s ± 28884.536op/s	9928329.746op/s	9945756.497op/s	9948511.749op/s	9951075.887op/s	0.46%	-1.280	2.493	0.46%	3230.047op/s	1	200
credit_card/is_card_number_no_luhn/x371413321323331	execution_time	20.838µs	20.856µs ± 0.022µs	20.851µs ± 0.006µs	20.857µs	20.894µs	20.941µs	21.036µs	0.88%	3.980	24.364	0.10%	0.002µs	1	200
credit_card/is_card_number_no_luhn/x371413321323331	throughput	47538124.338op/s	47947532.654op/s ± 49793.321op/s	47958319.425op/s ± 14667.342op/s	47975309.508op/s	47987115.460op/s	47988854.541op/s	47989207.227op/s	0.06%	-3.947	23.976	0.10%	3520.920op/s	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
credit_card/is_card_number/	execution_time	[2.016µs; 2.017µs] or [-0.014%; +0.014%]	None	None	None
credit_card/is_card_number/	throughput	[495832721.127op/s; 495974435.234op/s] or [-0.014%; +0.014%]	None	None	None
credit_card/is_card_number/ 3782-8224-6310-005	execution_time	[112.682µs; 112.896µs] or [-0.095%; +0.095%]	None	None	None
credit_card/is_card_number/ 3782-8224-6310-005	throughput	[8858196.957op/s; 8874881.132op/s] or [-0.094%; +0.094%]	None	None	None
credit_card/is_card_number/ 378282246310005	execution_time	[104.371µs; 104.542µs] or [-0.082%; +0.082%]	None	None	None
credit_card/is_card_number/ 378282246310005	throughput	[9565903.501op/s; 9581457.431op/s] or [-0.081%; +0.081%]	None	None	None
credit_card/is_card_number/37828224631	execution_time	[2.016µs; 2.017µs] or [-0.012%; +0.012%]	None	None	None
credit_card/is_card_number/37828224631	throughput	[495868723.159op/s; 495987977.111op/s] or [-0.012%; +0.012%]	None	None	None
credit_card/is_card_number/378282246310005	execution_time	[100.902µs; 101.054µs] or [-0.075%; +0.075%]	None	None	None
credit_card/is_card_number/378282246310005	throughput	[9896032.036op/s; 9910831.488op/s] or [-0.075%; +0.075%]	None	None	None
credit_card/is_card_number/37828224631000521389798	execution_time	[100.848µs; 100.955µs] or [-0.053%; +0.053%]	None	None	None
credit_card/is_card_number/37828224631000521389798	throughput	[9905599.601op/s; 9916048.381op/s] or [-0.053%; +0.053%]	None	None	None
credit_card/is_card_number/x371413321323331	execution_time	[20.854µs; 20.862µs] or [-0.018%; +0.018%]	None	None	None
credit_card/is_card_number/x371413321323331	throughput	[47934643.397op/s; 47951720.446op/s] or [-0.018%; +0.018%]	None	None	None
credit_card/is_card_number_no_luhn/	execution_time	[2.016µs; 2.016µs] or [-0.004%; +0.004%]	None	None	None
credit_card/is_card_number_no_luhn/	throughput	[495949346.010op/s; 495988202.102op/s] or [-0.004%; +0.004%]	None	None	None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005	execution_time	[89.620µs; 89.858µs] or [-0.133%; +0.133%]	None	None	None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005	throughput	[11129690.210op/s; 11159079.409op/s] or [-0.132%; +0.132%]	None	None	None
credit_card/is_card_number_no_luhn/ 378282246310005	execution_time	[81.384µs; 81.475µs] or [-0.055%; +0.055%]	None	None	None
credit_card/is_card_number_no_luhn/ 378282246310005	throughput	[12274003.953op/s; 12287482.343op/s] or [-0.055%; +0.055%]	None	None	None
credit_card/is_card_number_no_luhn/37828224631	execution_time	[2.016µs; 2.017µs] or [-0.014%; +0.014%]	None	None	None
credit_card/is_card_number_no_luhn/37828224631	throughput	[495844127.496op/s; 495978811.713op/s] or [-0.014%; +0.014%]	None	None	None
credit_card/is_card_number_no_luhn/378282246310005	execution_time	[77.788µs; 77.849µs] or [-0.039%; +0.039%]	None	None	None
credit_card/is_card_number_no_luhn/378282246310005	throughput	[12845516.839op/s; 12855585.797op/s] or [-0.039%; +0.039%]	None	None	None
credit_card/is_card_number_no_luhn/37828224631000521389798	execution_time	[101.013µs; 101.143µs] or [-0.064%; +0.064%]	None	None	None
credit_card/is_card_number_no_luhn/37828224631000521389798	throughput	[9887203.373op/s; 9899864.924op/s] or [-0.064%; +0.064%]	None	None	None
credit_card/is_card_number_no_luhn/x371413321323331	execution_time	[20.853µs; 20.859µs] or [-0.014%; +0.014%]	None	None	None
credit_card/is_card_number_no_luhn/x371413321323331	throughput	[47940631.778op/s; 47954433.529op/s] or [-0.014%; +0.014%]	None	None	None

Group 9

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	e4efeb3	1727685524	vianney/increase_test_run_delay

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
sql/obfuscate_sql_string	execution_time	76.950µs	77.214µs ± 0.150µs	77.198µs ± 0.051µs	77.244µs	77.348µs	77.669µs	78.824µs	2.11%	6.631	65.040	0.19%	0.011µs	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
sql/obfuscate_sql_string	execution_time	[77.193µs; 77.235µs] or [-0.027%; +0.027%]	None	None	None

Group 10

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	e4efeb3	1727685524	vianney/increase_test_run_delay

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
normalization/normalize_trace/test_trace	execution_time	307.611ns	316.513ns ± 12.478ns	311.138ns ± 2.290ns	316.309ns	346.471ns	357.438ns	358.204ns	15.13%	2.042	3.218	3.93%	0.882ns	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
normalization/normalize_trace/test_trace	execution_time	[314.783ns; 318.242ns] or [-0.546%; +0.546%]	None	None	None

Group 11

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	e4efeb3	1727685524	vianney/increase_test_run_delay

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
two way interface	execution_time	18.427µs	24.392µs ± 11.057µs	19.754µs ± 0.465µs	20.793µs	37.585µs	40.644µs	120.824µs	511.65%	4.675	32.936	45.22%	0.782µs	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
two way interface	execution_time	[22.860µs; 25.925µs] or [-6.282%; +6.282%]	None	None	None

Group 12

cpu_model	git_commit_sha	git_commit_date	git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	e4efeb3	1727685524	vianney/increase_test_run_delay

scenario	metric	min	mean ± sd	median ± mad	p75	p95	p99	max	peak_to_median_ratio	skewness	kurtosis	cv	sem	runs	sample_size
tags/replace_trace_tags	execution_time	2.647µs	2.710µs ± 0.022µs	2.709µs ± 0.006µs	2.716µs	2.762µs	2.765µs	2.771µs	2.28%	0.135	2.382	0.82%	0.002µs	1	200

scenario	metric	95% CI mean	Shapiro-Wilk pvalue	Ljung-Box pvalue (lag=1)	Dip test pvalue
tags/replace_trace_tags	execution_time	[2.707µs; 2.713µs] or [-0.114%; +0.114%]	None	None	None

Baseline

Omitted due to size.