[Perf] Linux/x64: 29 Regressions on 3/1/2023 5:53:50 PM #13755

performanceautofiler · 2023-03-08T05:44:13Z

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	605d05ba54ccea0ff5486926a7dc630cee5db8cd
Compare	ef0241ed9316bc4343e715daca4d88c1ff90231b
Diff	Diff

Regressions in System.Tests.Perf_String

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
IndexerCheckLengthHoisting - Duration of single invocation	26.39 ns	41.83 ns	1.59	0.52	False
ToLower - Duration of single invocation	225.62 ns	353.43 ns	1.57	0.15	False
CtorCharCount - Duration of single invocation	59.70 ns	77.58 ns	1.30	0.32	False
PadLeft - Duration of single invocation	79.43 ns	103.09 ns	1.30	0.40	False
ToUpper - Duration of single invocation	224.64 ns	377.85 ns	1.68	0.16	False
ToUpperInvariant - Duration of single invocation	193.00 ns	306.12 ns	1.59	0.11	False
ToLowerInvariant - Duration of single invocation	205.16 ns	303.99 ns	1.48	0.10	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_String*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_String.IndexerCheckLengthHoisting

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 41.82894696760661 > 27.703327538697756.
IsChangePoint: Marked as a change because one of 3/1/2023 3:09:21 PM, 3/7/2023 7:56:26 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 7:56:26 PM.
IsRegressionStdDev: Marked as regression because -32.191395005720636 (T) = (0 -39.227952731190776) / Math.Sqrt((2.2987827611203038 / (29)) + (2.130444531773961 / (39))) is less than -1.9965644189515832 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (39) - 2, .025) and -0.42914333445426117 = (27.448578309446148 - 39.227952731190776) / 27.448578309446148 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.ToLower(s: "This is a much longer piece of text that might benefit more from vectorization.")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 353.4337818960568 > 232.82118714013097.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 3/1/2023 3:09:21 PM, 3/7/2023 7:56:26 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 7:56:26 PM.
IsRegressionStdDev: Marked as regression because -40.634085757075816 (T) = (0 -362.1090027188929) / Math.Sqrt((20.297529142740533 / (30)) + (420.02307397867094 / (39))) is less than -1.9960083540247138 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (39) - 2, .025) and -0.6119985744079897 = (224.6335750339471 - 362.1090027188929) / 224.6335750339471 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.CtorCharCount(size: 10)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 77.57732561551073 > 66.44490945482254.
IsChangePoint: Marked as a change because one of 1/18/2023 3:58:17 AM, 3/1/2023 3:09:21 PM, 3/7/2023 7:56:26 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 7:56:26 PM.
IsRegressionStdDev: Marked as regression because -15.416099279258859 (T) = (0 -76.09339517367445) / Math.Sqrt((5.6031448767946035 / (30)) + (16.326698069258953 / (39))) is less than -1.9960083540247138 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (39) - 2, .025) and -0.18713272215230525 = (64.09847336674788 - 76.09339517367445) / 64.09847336674788 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.PadLeft(n: 18)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 103.08926884112797 > 85.43390280392919.
IsChangePoint: Marked as a change because one of 1/14/2023 10:43:56 AM, 3/1/2023 5:53:50 PM, 3/7/2023 7:56:26 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 7:56:26 PM.
IsRegressionStdDev: Marked as regression because -9.68525725823996 (T) = (0 -98.39761883754494) / Math.Sqrt((16.14965701158545 / (31)) + (69.64983621634943 / (38))) is less than -1.9960083540247138 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (38) - 2, .025) and -0.17787499411268176 = (83.538252640867 - 98.39761883754494) / 83.538252640867 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.ToUpper(s: "This is a much longer piece of text that might benefit more from vectorization.")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 377.84674911210425 > 237.63925886733796.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 3/1/2023 3:09:21 PM, 3/7/2023 7:56:26 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 7:56:26 PM.
IsRegressionStdDev: Marked as regression because -39.156693044760175 (T) = (0 -364.3480862142918) / Math.Sqrt((17.79434553648585 / (30)) + (475.4163860480634 / (39))) is less than -1.9960083540247138 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (39) - 2, .025) and -0.6240293682749661 = (224.34821274279054 - 364.3480862142918) / 224.34821274279054 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.ToUpperInvariant(s: "This is a much longer piece of text that might benefit more from vectorization.")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 306.12151880255936 > 203.10880620425826.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 3/1/2023 3:09:21 PM, 3/7/2023 7:56:26 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 7:56:26 PM.
IsRegressionStdDev: Marked as regression because -50.77168771921563 (T) = (0 -303.4604725821685) / Math.Sqrt((17.974454628641865 / (30)) + (160.15292965899772 / (39))) is less than -1.9960083540247138 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (39) - 2, .025) and -0.5696978903175034 = (193.3241259060286 - 303.4604725821685) / 193.3241259060286 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.ToLowerInvariant(s: "This is a much longer piece of text that might benefit more from vectorization.")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 303.9909950767776 > 200.74636420269337.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 3/1/2023 3:09:21 PM, 3/7/2023 7:56:26 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 7:56:26 PM.
IsRegressionStdDev: Marked as regression because -53.66505143777063 (T) = (0 -305.4980945172726) / Math.Sqrt((20.166320219948606 / (30)) + (140.9072518715231 / (39))) is less than -1.9960083540247138 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (39) - 2, .025) and -0.5714330257579686 = (194.40732726736346 - 305.4980945172726) / 194.40732726736346 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	605d05ba54ccea0ff5486926a7dc630cee5db8cd
Compare	ef0241ed9316bc4343e715daca4d88c1ff90231b
Diff	Diff

Regressions in System.Text.Tests.Perf_Encoding

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
GetByteCount - Duration of single invocation	38.61 ns	44.33 ns	1.15	0.63	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Tests.Perf_Encoding*'

Payloads

Baseline
Compare

Histogram

System.Text.Tests.Perf_Encoding.GetByteCount(size: 16, encName: "utf-8")

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 44.33202723180661 > 36.7779864338579.
IsChangePoint: Marked as a change because one of 1/17/2023 8:36:05 PM, 3/1/2023 3:09:21 PM, 3/7/2023 7:56:26 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 7:56:26 PM.
IsRegressionStdDev: Marked as regression because -17.565347295110854 (T) = (0 -49.46813840273708) / Math.Sqrt((7.920079201110455 / (30)) + (11.12025465017022 / (39))) is less than -1.9960083540247138 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (39) - 2, .025) and -0.35709294277888853 = (36.45154789578545 - 49.46813840273708) / 36.45154789578545 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	605d05ba54ccea0ff5486926a7dc630cee5db8cd
Compare	ef0241ed9316bc4343e715daca4d88c1ff90231b
Diff	Diff

Regressions in System.Numerics.Tests.Perf_BitOperations

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
Log2_uint - Duration of single invocation	563.73 ns	5.07 μs	8.99	0.12	False
TrailingZeroCount_ulong - Duration of single invocation	392.74 ns	985.58 ns	2.51	0.01	False
PopCount_uint - Duration of single invocation	393.95 ns	762.57 ns	1.94	0.00	False
TrailingZeroCount_uint - Duration of single invocation	396.29 ns	762.06 ns	1.92	0.01	False
Log2_ulong - Duration of single invocation	661.24 ns	5.76 μs	8.71	0.11	False
PopCount_ulong - Duration of single invocation	393.10 ns	991.15 ns	2.52	0.00	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_BitOperations*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_BitOperations.Log2_uint

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.068070551373357 > 590.4776095649311.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 3/1/2023 3:09:21 PM, 3/7/2023 7:56:26 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 7:56:26 PM.
IsRegressionStdDev: Marked as regression because -174.47162115911692 (T) = (0 -5134.6709385560725) / Math.Sqrt((2.129295899276432 / (30)) + (26099.15980276876 / (38))) is less than -1.9965644189515832 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (38) - 2, .025) and -8.13621135799764 = (562.0131515522891 - 5134.6709385560725) / 562.0131515522891 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_BitOperations.TrailingZeroCount_ulong

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 985.5805571480137 > 413.39812363644256.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 3/1/2023 3:09:21 PM, 3/7/2023 7:56:26 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 7:56:26 PM.
IsRegressionStdDev: Marked as regression because -1780.7560343152702 (T) = (0 -986.650709538981) / Math.Sqrt((1.0358434123149214 / (30)) + (2.9825231910354155 / (39))) is less than -1.9960083540247138 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (39) - 2, .025) and -1.5083019964203095 = (393.35403430171755 - 986.650709538981) / 393.35403430171755 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_BitOperations.PopCount_uint

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 762.5707871256872 > 414.27920255393366.
IsChangePoint: Marked as a change because one of 3/1/2023 3:09:21 PM, 3/7/2023 7:56:26 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 7:56:26 PM.
IsRegressionStdDev: Marked as regression because -819.3642406221456 (T) = (0 -763.8301086861795) / Math.Sqrt((4.258602725129243 / (29)) + (2.132076404945987 / (39))) is less than -1.9965644189515832 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (39) - 2, .025) and -0.9288043859507215 = (396.0122209643785 - 763.8301086861795) / 396.0122209643785 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_BitOperations.TrailingZeroCount_uint

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 762.059634245763 > 416.76240386667666.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 3/1/2023 3:09:21 PM, 3/7/2023 7:56:26 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 7:56:26 PM.
IsRegressionStdDev: Marked as regression because -769.6917580990686 (T) = (0 -762.9330275182712) / Math.Sqrt((4.816246841893949 / (30)) + (2.5027707526853127 / (39))) is less than -1.9960083540247138 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (39) - 2, .025) and -0.9165929101516619 = (398.06733264911196 - 762.9330275182712) / 398.06733264911196 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_BitOperations.Log2_ulong

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.7620901760057475 > 694.1558815702053.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 3/1/2023 3:09:21 PM, 3/7/2023 7:56:26 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 7:56:26 PM.
IsRegressionStdDev: Marked as regression because -115.0729870104527 (T) = (0 -5307.71323745104) / Math.Sqrt((2.3267998044654035 / (30)) + (63571.89594998186 / (39))) is less than -1.9960083540247138 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (39) - 2, .025) and -7.021789668906295 = (661.6619802466977 - 5307.71323745104) / 661.6619802466977 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_BitOperations.PopCount_ulong

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 991.1470007149413 > 412.91586513153925.
IsChangePoint: Marked as a change because one of 3/1/2023 3:09:21 PM, 3/7/2023 7:56:26 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 7:56:26 PM.
IsRegressionStdDev: Marked as regression because -1406.0311449078295 (T) = (0 -986.5106706700818) / Math.Sqrt((0.7114944230762575 / (29)) + (5.980295080109642 / (39))) is less than -1.9965644189515832 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (39) - 2, .025) and -1.5069297445718326 = (393.513489082069 - 986.5106706700818) / 393.513489082069 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	605d05ba54ccea0ff5486926a7dc630cee5db8cd
Compare	ef0241ed9316bc4343e715daca4d88c1ff90231b
Diff	Diff

Regressions in SIMD.RayTracerBench

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Bench - Duration of single invocation	264.54 ms	278.23 ms	1.05	0.12	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'SIMD.RayTracerBench*'

Payloads

Baseline
Compare

Histogram

SIMD.RayTracerBench.Bench

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 278.23386653333336 > 274.60366500875.
IsChangePoint: Marked as a change because one of 3/1/2023 3:09:21 PM, 3/7/2023 7:56:26 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 7:56:26 PM.
IsRegressionStdDev: Marked as regression because -12.046987083462755 (T) = (0 -283216617.77246165) / Math.Sqrt((22437991164822.406 / (30)) + (67962349326024.69 / (39))) is less than -1.9960083540247138 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (39) - 2, .025) and -0.07195918545318261 = (264204665.2669231 - 283216617.77246165) / 264204665.2669231 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	605d05ba54ccea0ff5486926a7dc630cee5db8cd
Compare	ef0241ed9316bc4343e715daca4d88c1ff90231b
Diff	Diff

Regressions in System.Collections.IndexerSetReverse<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Span - Duration of single invocation	81.87 ns	158.14 ns	1.93	0.06	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IndexerSetReverse&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.IndexerSetReverse<String>.Span(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 158.13722887851608 > 86.07958356953412.
IsChangePoint: Marked as a change because one of 3/1/2023 3:09:21 PM, 3/7/2023 7:56:26 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 7:56:26 PM.
IsRegressionStdDev: Marked as regression because -230.94519911821197 (T) = (0 -157.46604371176812) / Math.Sqrt((1.6813944950185855 / (29)) + (1.8704954154538767 / (39))) is less than -1.9965644189515832 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (39) - 2, .025) and -0.9133914642648328 = (82.2968256379622 - 157.46604371176812) / 82.2968256379622 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	605d05ba54ccea0ff5486926a7dc630cee5db8cd
Compare	ef0241ed9316bc4343e715daca4d88c1ff90231b
Diff	Diff

Regressions in System.Net.Primitives.Tests.IPAddressPerformanceTests

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
GetAddressBytes - Duration of single invocation	32.81 ns	85.81 ns	2.62	0.47	False
Ctor_Span - Duration of single invocation	27.88 ns	106.76 ns	3.83	0.62	False
TryWriteBytes - Duration of single invocation	30.17 ns	83.28 ns	2.76	0.59	False
Ctor_Bytes - Duration of single invocation	35.10 ns	99.33 ns	2.83	0.54	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Net.Primitives.Tests.IPAddressPerformanceTests*'

Payloads

Baseline
Compare

Histogram

System.Net.Primitives.Tests.IPAddressPerformanceTests.GetAddressBytes(address: 1020:3040:5060:7080:9010:1112:1314:1516)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 85.80709392409577 > 37.17487266650631.
IsChangePoint: Marked as a change because one of 3/1/2023 3:09:21 PM, 3/7/2023 7:56:26 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 7:56:26 PM.
IsRegressionStdDev: Marked as regression because -54.34721839420134 (T) = (0 -89.51410323062314) / Math.Sqrt((5.68208174265619 / (30)) + (30.466003459880707 / (39))) is less than -1.9960083540247138 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (39) - 2, .025) and -1.4884219235465466 = (35.97223701640031 - 89.51410323062314) / 35.97223701640031 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Net.Primitives.Tests.IPAddressPerformanceTests.Ctor_Span(address: [16, 32, 48, 64, 80, ...])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 106.75993455005616 > 28.909951048556067.
IsChangePoint: Marked as a change because one of 3/1/2023 3:09:21 PM, 3/7/2023 7:56:26 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 7:56:26 PM.
IsRegressionStdDev: Marked as regression because -63.22911949403745 (T) = (0 -94.17354292883925) / Math.Sqrt((4.847315333432165 / (30)) + (36.66677318412625 / (39))) is less than -1.9960083540247138 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (39) - 2, .025) and -2.386864095511788 = (27.805527553832572 - 94.17354292883925) / 27.805527553832572 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Net.Primitives.Tests.IPAddressPerformanceTests.TryWriteBytes(address: 1020:3040:5060:7080:9010:1112:1314:1516)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 83.28338875001727 > 26.574624959354292.
IsChangePoint: Marked as a change because one of 3/1/2023 3:09:21 PM, 3/7/2023 7:56:26 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 7:56:26 PM.
IsRegressionStdDev: Marked as regression because -73.88415004942134 (T) = (0 -82.28313738745834) / Math.Sqrt((5.709251910682509 / (30)) + (16.354092329816343 / (39))) is less than -1.9960083540247138 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (39) - 2, .025) and -2.345576298810557 = (24.594607935473544 - 82.28313738745834) / 24.594607935473544 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Net.Primitives.Tests.IPAddressPerformanceTests.Ctor_Bytes(address: [16, 32, 48, 64, 80, ...])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 99.32686038016432 > 33.51097992913993.
IsChangePoint: Marked as a change because one of 3/1/2023 3:09:21 PM, 3/7/2023 7:56:26 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 7:56:26 PM.
IsRegressionStdDev: Marked as regression because -81.3548538018689 (T) = (0 -97.93560255792029) / Math.Sqrt((3.450421306542092 / (30)) + (20.53673333631725 / (39))) is less than -1.9960083540247138 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (39) - 2, .025) and -1.9885229795121284 = (32.77057035509498 - 97.93560255792029) / 32.77057035509498 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

The text was updated successfully, but these errors were encountered:

performanceautofiler · 2023-03-08T05:44:14Z

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	605d05ba54ccea0ff5486926a7dc630cee5db8cd
Compare	ef0241ed9316bc4343e715daca4d88c1ff90231b
Diff	Diff

Regressions in System.Collections.IndexerSet<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Span - Duration of single invocation	133.80 ns	144.12 ns	1.08	0.05	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IndexerSet&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.IndexerSet<Int32>.Span(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 144.11902028947483 > 141.34569567412836.
IsChangePoint: Marked as a change because one of 3/1/2023 3:09:21 PM, 3/7/2023 7:56:26 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 7:56:26 PM.
IsRegressionStdDev: Marked as regression because -23.49265979090468 (T) = (0 -144.0953503166835) / Math.Sqrt((3.5017311922259293 / (30)) + (5.2919579778514745 / (39))) is less than -1.9960083540247138 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (39) - 2, .025) and -0.08921859239032066 = (132.29240789992596 - 144.0953503166835) / 132.29240789992596 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	605d05ba54ccea0ff5486926a7dc630cee5db8cd
Compare	ef0241ed9316bc4343e715daca4d88c1ff90231b
Diff	Diff

Regressions in System.Memory.Span<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Fill - Duration of single invocation	12.85 ns	24.97 ns	1.94	0.56	False
Fill - Duration of single invocation	19.26 ns	23.59 ns	1.23	0.58	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Int32>.Fill(Size: 4)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 24.972412263865422 > 8.651933865433968.
IsChangePoint: Marked as a change because one of 3/1/2023 3:09:21 PM, 3/7/2023 7:56:26 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 7:56:26 PM.
IsRegressionStdDev: Marked as regression because -28.509300535426533 (T) = (0 -20.36500864141318) / Math.Sqrt((1.0078491413413302 / (30)) + (5.1596951955284025 / (39))) is less than -1.9960083540247138 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (39) - 2, .025) and -1.3266010231075704 = (8.75311600018651 - 20.36500864141318) / 8.75311600018651 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Int32&gt;.Fill(Size: 33)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 23.59415710102026 > 19.413896372474806.
IsChangePoint: Marked as a change because one of 3/1/2023 3:09:21 PM, 3/7/2023 7:56:26 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 7:56:26 PM.
IsRegressionStdDev: Marked as regression because -10.225250919880816 (T) = (0 -25.620942838680637) / Math.Sqrt((2.883996988859358 / (30)) + (9.714602365771329 / (39))) is less than -1.9960083540247138 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (39) - 2, .025) and -0.30632422604655296 = (19.613004434756302 - 25.620942838680637) / 19.613004434756302 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	605d05ba54ccea0ff5486926a7dc630cee5db8cd
Compare	ef0241ed9316bc4343e715daca4d88c1ff90231b
Diff	Diff

Regressions in System.Buffers.Tests.NonStandardArrayPoolTests<Byte>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
RentNoReturn - Duration of single invocation	41.00 ns	52.48 ns	1.28	0.49	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Tests.NonStandardArrayPoolTests&lt;Byte&gt;*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Tests.NonStandardArrayPoolTests<Byte>.RentNoReturn(RentalSize: 64, UseSharedPool: True)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 52.48282363839286 > 45.14362043284255.
IsChangePoint: Marked as a change because one of 1/9/2023 10:41:51 AM, 3/1/2023 3:09:21 PM, 3/7/2023 7:56:26 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 7:56:26 PM.
IsRegressionStdDev: Marked as regression because -7.631430622003726 (T) = (0 -47.71589258278886) / Math.Sqrt((3.695863536626771 / (30)) + (7.379439973042226 / (39))) is less than -1.9960083540247138 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (39) - 2, .025) and -0.09816936683792507 = (43.45039483315972 - 47.71589258278886) / 43.45039483315972 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	605d05ba54ccea0ff5486926a7dc630cee5db8cd
Compare	ef0241ed9316bc4343e715daca4d88c1ff90231b
Diff	Diff

Regressions in System.Tests.Perf_Uri

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
CtorIdnHostPathAndQuery - Duration of single invocation	1.03 μs	1.10 μs	1.06	0.07	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Uri*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Uri.CtorIdnHostPathAndQuery(input: "https://CONTOSO.com/path/with?key=value#fragment")

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.1007272628702398 > 1.0873435686501092.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 3/1/2023 3:09:21 PM, 3/7/2023 7:56:26 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 7:56:26 PM.
IsRegressionStdDev: Marked as regression because -18.19089575169683 (T) = (0 -1124.74145533751) / Math.Sqrt((185.41387314917856 / (30)) + (632.2415325772278 / (39))) is less than -1.9960083540247138 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (39) - 2, .025) and -0.08287510631174996 = (1038.6622139355995 - 1124.74145533751) / 1038.6622139355995 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	605d05ba54ccea0ff5486926a7dc630cee5db8cd
Compare	ef0241ed9316bc4343e715daca4d88c1ff90231b
Diff	Diff

Regressions in System.Memory.Span<Char>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Fill - Duration of single invocation	18.76 ns	26.77 ns	1.43	0.38	False
Fill - Duration of single invocation	10.24 ns	14.63 ns	1.43	0.60	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span&lt;Char&gt;*'

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Char>.Fill(Size: 33)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 26.773031722500836 > 19.53761558708493.
IsChangePoint: Marked as a change because one of 3/1/2023 3:09:21 PM, 3/7/2023 7:56:26 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 7:56:26 PM.
IsRegressionStdDev: Marked as regression because -11.300407296339161 (T) = (0 -24.510894331443517) / Math.Sqrt((1.7736024141639313 / (30)) + (6.802665520848794 / (39))) is less than -1.9960083540247138 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (39) - 2, .025) and -0.28667627446602106 = (19.049775625664424 - 24.510894331443517) / 19.049775625664424 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.Fill(Size: 4)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.634598247350224 > 9.301240787928934.
IsChangePoint: Marked as a change because one of 3/1/2023 3:09:21 PM, 3/7/2023 7:56:26 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 7:56:26 PM.
IsRegressionStdDev: Marked as regression because -22.07553225069095 (T) = (0 -15.318591370745121) / Math.Sqrt((1.0694733067868334 / (30)) + (1.969616711850759 / (39))) is less than -1.9960083540247138 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (39) - 2, .025) and -0.7330581550807849 = (8.839052126344288 - 15.318591370745121) / 8.839052126344288 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	605d05ba54ccea0ff5486926a7dc630cee5db8cd
Compare	ef0241ed9316bc4343e715daca4d88c1ff90231b
Diff	Diff

Regressions in System.MathBenchmarks.Single

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
FusedMultiplyAdd - Duration of single invocation	16.10 μs	29.98 μs	1.86	0.05	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.MathBenchmarks.Single*'

Payloads

Baseline
Compare

Histogram

System.MathBenchmarks.Single.FusedMultiplyAdd

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 29.978362761699508 > 16.98462961324419.
IsChangePoint: Marked as a change because one of 3/1/2023 3:09:21 PM, 3/7/2023 7:56:26 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 7:56:26 PM.
IsRegressionStdDev: Marked as regression because -126.61503364990018 (T) = (0 -29997.7778563922) / Math.Sqrt((349832.56561743055 / (30)) + (551.968320891274 / (39))) is less than -1.9960083540247138 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (39) - 2, .025) and -0.8384632229204377 = (16316.767984480046 - 29997.7778563922) / 16316.767984480046 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	605d05ba54ccea0ff5486926a7dc630cee5db8cd
Compare	ef0241ed9316bc4343e715daca4d88c1ff90231b
Diff	Diff

Regressions in System.Collections.IndexerSetReverse<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Span - Duration of single invocation	66.80 ns	168.49 ns	2.52	0.16	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IndexerSetReverse&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.IndexerSetReverse<Int32>.Span(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 168.48959534321574 > 69.07912354636562.
IsChangePoint: Marked as a change because one of 3/1/2023 3:09:21 PM, 3/7/2023 7:56:26 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 7:56:26 PM.
IsRegressionStdDev: Marked as regression because -138.81602025333282 (T) = (0 -163.46595290008602) / Math.Sqrt((9.815690904786074 / (30)) + (5.889383824765517 / (38))) is less than -1.9965644189515832 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (38) - 2, .025) and -1.437102419329749 = (67.07389546026644 - 163.46595290008602) / 67.07389546026644 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

radekdoulik · 2023-03-08T12:15:44Z

that is probably enabled SIMD dotnet/runtime#80797

performanceautofiler bot added AOT=true untriaged labels Mar 8, 2023

radekdoulik removed the untriaged label Mar 8, 2023

radekdoulik self-assigned this Mar 8, 2023

performanceautofiler bot added perf-regression arch-x64 os-linux branch-refs/heads/main arch-wasm kind-micro wasm-aot and removed refs/heads/main labels Mar 23, 2023

radekdoulik mentioned this issue Jun 29, 2023

[wasm] NET8 previews performance tracking dotnet/runtime#88178

Open

[Perf] Linux/x64: 29 Regressions on 3/1/2023 5:53:50 PM #13755

[Perf] Linux/x64: 29 Regressions on 3/1/2023 5:53:50 PM #13755

Comments

performanceautofiler bot commented Mar 8, 2023 • edited Loading

Run Information

Regressions in System.Tests.Perf_String

Repro

Payloads

Payloads

Histogram

System.Tests.Perf_String.IndexerCheckLengthHoisting

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Docs

Run Information

Regressions in System.Text.Tests.Perf_Encoding

Repro

Payloads

Payloads

Histogram

System.Text.Tests.Perf_Encoding.GetByteCount(size: 16, encName: "utf-8")

Description of detection logic

Docs

Run Information

Regressions in System.Numerics.Tests.Perf_BitOperations

Repro

Payloads

Payloads

Histogram

System.Numerics.Tests.Perf_BitOperations.Log2_uint

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Docs

Run Information

Regressions in SIMD.RayTracerBench

Repro

Payloads

Payloads

Histogram

SIMD.RayTracerBench.Bench

Description of detection logic

Docs

Run Information

Regressions in System.Collections.IndexerSetReverse<String>

Repro

Payloads

Payloads

Histogram

System.Collections.IndexerSetReverse<String>.Span(Size: 512)

Description of detection logic

Docs

Run Information

Regressions in System.Net.Primitives.Tests.IPAddressPerformanceTests

Repro

Payloads

Payloads

Histogram

System.Net.Primitives.Tests.IPAddressPerformanceTests.GetAddressBytes(address: 1020:3040:5060:7080:9010:1112:1314:1516)

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Docs

performanceautofiler bot commented Mar 8, 2023 • edited Loading

Run Information

Regressions in System.Collections.IndexerSet<Int32>

Repro

Payloads

Payloads

Histogram

System.Collections.IndexerSet<Int32>.Span(Size: 512)

performanceautofiler bot commented Mar 8, 2023 •

edited

Loading

performanceautofiler bot commented Mar 8, 2023 •

edited

Loading