[Perf] Windows/arm64: 6 Improvements on 7/13/2023 7:10:54 AM

[performanceautofiler]

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	d3d910ea88b00c7af2f69df07cbfbbe86da56596
Compare	c88dbd3f943b8206b4821aaf10ddf3c216100942
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.ContainsTrue<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
ImmutableList - Duration of single invocation 📝 - Benchmark Source	643.07 μs	500.64 μs	0.78	0.08	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsTrue<Int32>*'

Payloads

Baseline
Compare

System.Collections.ContainsTrue<Int32>.ImmutableList(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 500.64404296875 < 608.9838529861111.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/12/2023 7:14:29 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsImprovementStdDev: Marked as improvement because 84.58540959860686 (T) = (0 -502534.99877041584) / Math.Sqrt((1199668659.6802623 / (299)) + (867775.2586783826 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.2550209876929505 = (674562.6258841393 - 502534.99877041584) / 674562.6258841393 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	d3d910ea88b00c7af2f69df07cbfbbe86da56596
Compare	c88dbd3f943b8206b4821aaf10ddf3c216100942
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_TimeSpan

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
FromSeconds - Duration of single invocation 📝 - Benchmark Source	1.31 ns	0.04 ns	0.03	0.51	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_TimeSpan*'

Payloads

Baseline
Compare

System.Tests.Perf_TimeSpan.FromSeconds

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.041575057740858554 < 1.3056746650491324.
IsChangePoint: Marked as a change because one of 6/22/2023 12:06:43 AM, 7/12/2023 7:14:29 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsImprovementStdDev: Marked as improvement because 7.083900514893958 (T) = (0 -0.0060102120361565725) / Math.Sqrt((0.23100420900511429 / (299)) + (0.00024597888740014094 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.971012596161294 = (0.20733874856813864 - 0.0060102120361565725) / 0.20733874856813864 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	d3d910ea88b00c7af2f69df07cbfbbe86da56596
Compare	c88dbd3f943b8206b4821aaf10ddf3c216100942
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.ContainsFalse<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
ImmutableList - Duration of single invocation 📝 - Benchmark Source	1.30 ms	1.00 ms	0.77	0.04	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsFalse<Int32>*'

Payloads

Baseline
Compare

System.Collections.ContainsFalse<Int32>.ImmutableList(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0031732572115386 < 1.2190552678571427.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/12/2023 7:14:29 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsImprovementStdDev: Marked as improvement because 50.49840525849263 (T) = (0 -1015637.619660845) / Math.Sqrt((2827949175.7267923 / (299)) + (228063957.72245052 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.24378553452702714 = (1343055.0009720013 - 1015637.619660845) / 1343055.0009720013 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	d3d910ea88b00c7af2f69df07cbfbbe86da56596
Compare	c88dbd3f943b8206b4821aaf10ddf3c216100942
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in Benchstone.BenchI.EightQueens

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
Test - Duration of single invocation 📝 - Benchmark Source	2.35 μs	2.21 μs	0.94	0.01	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchI.EightQueens*'

Payloads

Baseline
Compare

Benchstone.BenchI.EightQueens.Test

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.2051762323943667 < 2.2344798308994434.
IsChangePoint: Marked as a change because one of 4/9/2023 1:58:52 PM, 5/19/2023 4:22:12 AM, 7/12/2023 7:14:29 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsImprovementStdDev: Marked as improvement because 52.839902044167864 (T) = (0 -2206.451821443796) / Math.Sqrt((4446.5021930595185 / (299)) + (5.985734686650365 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.08673207285059421 = (2415.996177957131 - 2206.451821443796) / 2415.996177957131 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	d3d910ea88b00c7af2f69df07cbfbbe86da56596
Compare	c88dbd3f943b8206b4821aaf10ddf3c216100942
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in Benchstone.BenchI.Fib

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
Test - Duration of single invocation 📝 - Benchmark Source	142.13 μs	100.96 μs	0.71	0.03	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchI.Fib*'

Payloads

Baseline
Compare

Benchstone.BenchI.Fib.Test

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 100.95863213399504 < 134.97357102272727.
IsChangePoint: Marked as a change because one of 7/12/2023 7:14:29 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsImprovementStdDev: Marked as improvement because 12.23198791249768 (T) = (0 -105689.23284240132) / Math.Sqrt((6773038.551047063 / (299)) + (63332805.40158709 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.2584670057084211 = (142528.02458691833 - 105689.23284240132) / 142528.02458691833 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	d3d910ea88b00c7af2f69df07cbfbbe86da56596
Compare	c88dbd3f943b8206b4821aaf10ddf3c216100942
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.CreateAddAndClear<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
ImmutableList - Duration of single invocation 📝 - Benchmark Source	109.51 μs	100.44 μs	0.92	0.09	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CreateAddAndClear<Int32>*'

Payloads

Baseline
Compare

System.Collections.CreateAddAndClear<Int32>.ImmutableList(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 100.43511618589746 < 104.22740541910012.
IsChangePoint: Marked as a change because one of 5/16/2023 5:40:18 PM, 7/12/2023 7:14:29 PM, 7/19/2023 6:59:29 PM falls between 7/10/2023 2:17:11 PM and 7/19/2023 6:59:29 PM.
IsImprovementStdDev: Marked as improvement because 20.952164096846392 (T) = (0 -100788.64254392893) / Math.Sqrt((20227468.214931507 / (299)) + (2226408.622436969 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.1143437052840191 = (113801.07965726209 - 100788.64254392893) / 113801.07965726209 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[cincuranet]

Same as in #19970 and #19961.