[Perf] Linux/x64: 12 Regressions on 6/27/2023 10:31:09 PM

[performanceautofiler]

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	a0964f9e3793cb36cc01d66c14a61e89ada5e7da
Compare	f6230892a0a0678c8e78163d13e4ea231584044c
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in Microsoft.Extensions.Configuration.Xml.XmlConfigurationProviderBenchmarks

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Load - Duration of single invocation	134.43 μs	154.99 μs	1.15	0.02	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Configuration.Xml.XmlConfigurationProviderBenchmarks*'

Payloads

Baseline
Compare

Microsoft.Extensions.Configuration.Xml.XmlConfigurationProviderBenchmarks.Load(FileName: "deep.xml")

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 154.9908795053933 > 141.68073654930708.
IsChangePoint: Marked as a change because one of 4/12/2023 9:47:42 AM, 6/9/2023 7:03:36 PM, 6/26/2023 9:59:38 AM, 7/3/2023 9:53:34 PM falls between 6/24/2023 6:56:43 PM and 7/3/2023 9:53:34 PM.
IsRegressionStdDev: Marked as regression because -28.16842637060429 (T) = (0 -154385.4457153401) / Math.Sqrt((5360023.00668486 / (16)) + (1788057.5763426449 / (21))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (21) - 2, .025) and -0.13412782702776604 = (136127.02381171746 - 154385.4457153401) / 136127.02381171746 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	a0964f9e3793cb36cc01d66c14a61e89ada5e7da
Compare	f6230892a0a0678c8e78163d13e4ea231584044c
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in Span.IndexerBench

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
SameIndex2 - Duration of single invocation	1.13 μs	1.35 μs	1.20	0.00	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Span.IndexerBench*'

Payloads

Baseline
Compare

Span.IndexerBench.SameIndex2(length: 1024)

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.3482893042620474 > 1.1844126212725186.
IsChangePoint: Marked as a change because one of 6/26/2023 9:59:38 AM, 7/3/2023 9:53:34 PM falls between 6/24/2023 6:56:43 PM and 7/3/2023 9:53:34 PM.
IsRegressionStdDev: Marked as regression because -619.6091214707404 (T) = (0 -1348.3219316873221) / Math.Sqrt((1.2115988911722804 / (16)) + (1.0011574485096881 / (20))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (20) - 2, .025) and -0.19471486376935043 = (1128.5721577392435 - 1348.3219316873221) / 1128.5721577392435 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	a0964f9e3793cb36cc01d66c14a61e89ada5e7da
Compare	f6230892a0a0678c8e78163d13e4ea231584044c
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in Benchstone.BenchI.Array2

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Test - Duration of single invocation	1.25 secs	1.51 secs	1.20	0.03	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchI.Array2*'

Payloads

Baseline
Compare

Benchstone.BenchI.Array2.Test

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.5064999333333333 > 1.3126217625375.
IsChangePoint: Marked as a change because one of 6/26/2023 9:59:38 AM, 7/3/2023 9:53:34 PM falls between 6/24/2023 6:56:43 PM and 7/3/2023 9:53:34 PM.
IsRegressionStdDev: Marked as regression because -195.273642131397 (T) = (0 -1494586221.048315) / Math.Sqrt((15251037350282.465 / (16)) + (11773632669696.559 / (20))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (20) - 2, .025) and -0.19365332096877727 = (1252110805.367926 - 1494586221.048315) / 1252110805.367926 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	a0964f9e3793cb36cc01d66c14a61e89ada5e7da
Compare	f6230892a0a0678c8e78163d13e4ea231584044c
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Collections.CtorFromCollection<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
FrozenDictionary - Duration of single invocation	13.36 μs	120.75 μs	9.04	0.05	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorFromCollection<String>*'

Payloads

Baseline
Compare

System.Collections.CtorFromCollection<String>.FrozenDictionary(Size: 512)

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 120.75397283591732 > 13.862370387376536.
IsChangePoint: Marked as a change because one of 6/26/2023 9:59:38 AM, 7/3/2023 9:53:34 PM falls between 6/24/2023 6:56:43 PM and 7/3/2023 9:53:34 PM.
IsRegressionStdDev: Marked as regression because -179.88746862172334 (T) = (0 -122297.33581507113) / Math.Sqrt((29512.170994644464 / (16)) + (7676060.5316357585 / (21))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (21) - 2, .025) and -8.219227465084463 = (13265.464625777155 - 122297.33581507113) / 13265.464625777155 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	a0964f9e3793cb36cc01d66c14a61e89ada5e7da
Compare	293b250c9e6195a094422f4dacbd395087bcd5f3
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Collections.CtorFromCollection<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
FrozenDictionary - Duration of single invocation	8.07 μs	51.25 μs	6.35	0.09	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorFromCollection<Int32>*'

Payloads

Baseline
Compare

System.Collections.CtorFromCollection<Int32>.FrozenDictionary(Size: 512)

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 51.24991411389803 > 8.515590435146166.
IsChangePoint: Marked as a change because one of 6/26/2023 9:59:38 AM, 7/3/2023 9:53:34 PM falls between 6/24/2023 6:56:43 PM and 7/3/2023 9:53:34 PM.
IsRegressionStdDev: Marked as regression because -78.04808061153017 (T) = (0 -52343.694653834275) / Math.Sqrt((13475.19924653808 / (16)) + (6397684.823414225 / (20))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (20) - 2, .025) and -5.428101636948613 = (8142.947577705314 - 52343.694653834275) / 8142.947577705314 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	a0964f9e3793cb36cc01d66c14a61e89ada5e7da
Compare	f6230892a0a0678c8e78163d13e4ea231584044c
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.IO.Tests.Perf_FileStream

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
SeekBackward - Duration of single invocation	2.32 ms	2.54 ms	1.10	0.22	False
LockUnlock - Duration of single invocation	11.51 μs	13.45 μs	1.17	0.28	False
LockUnlock - Duration of single invocation	12.19 μs	13.22 μs	1.08	0.30	False
OpenClose - Duration of single invocation	7.87 μs	8.56 μs	1.09	0.24	False
GetLength - Duration of single invocation	2.26 μs	2.49 μs	1.10	0.21	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.Perf_FileStream*'

Payloads

Baseline
Compare

System.IO.Tests.Perf_FileStream.SeekBackward(fileSize: 1024, options: None)

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.539431757015306 > 2.4935906231250002.
IsChangePoint: Marked as a change because one of 6/26/2023 9:59:38 AM, 7/3/2023 9:53:34 PM falls between 6/24/2023 6:56:43 PM and 7/3/2023 9:53:34 PM.
IsRegressionStdDev: Marked as regression because -4.696741514903719 (T) = (0 -2634891.2191123418) / Math.Sqrt((34458776388.64561 / (16)) + (2960487800.0422087 / (21))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (21) - 2, .025) and -0.09335869600499751 = (2409905.5769528523 - 2634891.2191123418) / 2409905.5769528523 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.IO.Tests.Perf_FileStream.LockUnlock(fileSize: 1024, options: Asynchronous)

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.446095144865172 > 12.566861262948663.
IsChangePoint: Marked as a change because one of 5/3/2023 6:00:09 PM, 6/27/2023 10:31:09 PM, 7/3/2023 9:53:34 PM falls between 6/24/2023 6:56:43 PM and 7/3/2023 9:53:34 PM.
IsRegressionStdDev: Marked as regression because -6.8347601228259895 (T) = (0 -13032.053428252986) / Math.Sqrt((51084.23205723244 / (16)) + (429417.92314388475 / (19))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (19) - 2, .025) and -0.09197736170957053 = (11934.3622727218 - 13032.053428252986) / 11934.3622727218 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.IO.Tests.Perf_FileStream.LockUnlock(fileSize: 1024, options: None)

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.215890233277122 > 12.6499385269532.
IsChangePoint: Marked as a change because one of 5/3/2023 11:54:24 PM, 6/26/2023 9:59:38 AM, 7/3/2023 9:53:34 PM falls between 6/24/2023 6:56:43 PM and 7/3/2023 9:53:34 PM.
IsRegressionStdDev: Marked as regression because -7.8747068646951766 (T) = (0 -13039.868590759783) / Math.Sqrt((126115.73178273732 / (16)) + (205180.99274240568 / (20))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (20) - 2, .025) and -0.08854001194168466 = (11979.22763307515 - 13039.868590759783) / 11979.22763307515 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.IO.Tests.Perf_FileStream.OpenClose(fileSize: 1024, options: None)

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.559359136497065 > 8.271225805150667.
IsChangePoint: Marked as a change because one of 5/3/2023 1:06:54 PM, 6/27/2023 10:31:09 PM, 7/3/2023 9:53:34 PM falls between 6/24/2023 6:56:43 PM and 7/3/2023 9:53:34 PM.
IsRegressionStdDev: Marked as regression because -7.810925783548814 (T) = (0 -8888.360394018313) / Math.Sqrt((79938.66130286621 / (16)) + (84296.6399503107 / (19))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (19) - 2, .025) and -0.09331399982889658 = (8129.741680257766 - 8888.360394018313) / 8129.741680257766 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.IO.Tests.Perf_FileStream.GetLength(options: Write)

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.4911455499558217 > 2.434785547005789.
IsChangePoint: Marked as a change because one of 6/27/2023 10:31:09 PM, 7/3/2023 9:53:34 PM falls between 6/24/2023 6:56:43 PM and 7/3/2023 9:53:34 PM.
IsRegressionStdDev: Marked as regression because -11.788307930975892 (T) = (0 -2567.4656424690647) / Math.Sqrt((6035.394282970792 / (17)) + (4053.7202997522827 / (20))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (20) - 2, .025) and -0.12161732529010985 = (2289.074521744733 - 2567.4656424690647) / 2289.074521744733 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	a0964f9e3793cb36cc01d66c14a61e89ada5e7da
Compare	f6230892a0a0678c8e78163d13e4ea231584044c
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Collections.Tests.Perf_BitArray

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
BitArrayBoolArrayCtor - Duration of single invocation	62.69 ns	65.95 ns	1.05	0.45	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Perf_BitArray*'

Payloads

Baseline
Compare

System.Collections.Tests.Perf_BitArray.BitArrayBoolArrayCtor(Size: 512)

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 65.9549758830144 > 63.18059078746802.
IsChangePoint: Marked as a change because one of 6/26/2023 9:59:38 AM, 7/3/2023 9:53:34 PM falls between 6/24/2023 6:56:43 PM and 7/3/2023 9:53:34 PM.
IsRegressionStdDev: Marked as regression because -6.310158784990773 (T) = (0 -67.26076064727503) / Math.Sqrt((8.134672771419867 / (16)) + (4.509510531884336 / (18))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (18) - 2, .025) and -0.08900483310745369 = (61.7635097682237 - 67.26076064727503) / 61.7635097682237 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	a0964f9e3793cb36cc01d66c14a61e89ada5e7da
Compare	f6230892a0a0678c8e78163d13e4ea231584044c
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in Span.Sorting

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
BubbleSortSpan - Duration of single invocation	319.76 μs	339.38 μs	1.06	0.00	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Span.Sorting*'

Payloads

Baseline
Compare

Span.Sorting.BubbleSortSpan(Size: 512)

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 339.37786673333335 > 338.82943496499996.
IsChangePoint: Marked as a change because one of 6/26/2023 9:59:38 AM, 7/3/2023 9:53:34 PM falls between 6/24/2023 6:56:43 PM and 7/3/2023 9:53:34 PM.
IsRegressionStdDev: Marked as regression because -29.448550955135087 (T) = (0 -340148.28190226754) / Math.Sqrt((5795259.652475279 / (16)) + (357768.9449136072 / (21))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (21) - 2, .025) and -0.05631805086317552 = (322013.1300646748 - 340148.28190226754) / 322013.1300646748 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[radekdoulik]

@stephentoub looks like dotnet/runtime#87989 do you know what might be causing it?

[stephentoub]

@stephentoub looks like dotnet/runtime#87989 do you know what might be causing it?

Specifically the FrozenDictionary ones (the others are unrelated)? That would be dotnet/runtime#87988, and it's expected.