[Perf] Windows/arm64: 6 Regressions on 10/14/2022 6:08:00 AM

[performanceautofiler]

Run Information

Architecture	arm64
OS	Windows 10.0.19041
Baseline	f58cee20c39591835a01c2ffd5f2b3900b279c38
Compare	65d20644b7852307fe076225a4f78395287bdb93
Diff	Diff

Regressions in System.IO.Tests.StreamReaderReadLineTests

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ReadLineAsync - Duration of single invocation	33.27 μs	38.43 μs	1.16	0.01	False
ReadLineAsync - Duration of single invocation	42.92 μs	46.69 μs	1.09	0.03	False
ReadLineAsync - Duration of single invocation	35.65 μs	41.05 μs	1.15	0.01	False
ReadLineAsync - Duration of single invocation	33.31 μs	38.80 μs	1.16	0.01	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.StreamReaderReadLineTests*'

Related Issues

Regressions

Improvements

• [Perf] Windows/x64: 1 Improvement on 10/14/2022 6:08:00 AM

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.IO.Tests.StreamReaderReadLineTests.ReadLineAsync(LineLengthRange: [ 0, 1024])

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 8/17/2022 3:53:31 PM, 10/13/2022 10:51:15 PM, 10/19/2022 4:51:05 PM falls between 10/10/2022 11:56:05 PM and 10/19/2022 4:51:05 PM.
IsRegressionStdDev: Marked as regression because -165.96934016961987 (T) = (0 -38425.70319791642) / Math.Sqrt((10507.106090648025 / (14)) + (3923.5972768755455 / (18))) is less than -2.0422724562973107 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (18) - 2, .025) and -0.15529002598093586 = (33260.65518940999 - 38425.70319791642) / 33260.65518940999 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StreamReaderReadLineTests.ReadLineAsync(LineLengthRange: [  33,  128])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 8/17/2022 3:53:31 PM, 10/13/2022 10:51:15 PM, 10/19/2022 4:51:05 PM falls between 10/10/2022 11:56:05 PM and 10/19/2022 4:51:05 PM.
IsRegressionStdDev: Marked as regression because -22.74286031119024 (T) = (0 -47086.13323950621) / Math.Sqrt((137003.14743574773 / (14)) + (737051.5002919621 / (18))) is less than -2.0422724562973107 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (18) - 2, .025) and -0.12207295919073562 = (41963.521938418155 - 47086.13323950621) / 41963.521938418155 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StreamReaderReadLineTests.ReadLineAsync(LineLengthRange: [1025, 2048])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 8/17/2022 3:53:31 PM, 10/13/2022 10:51:15 PM, 10/19/2022 4:51:05 PM falls between 10/10/2022 11:56:05 PM and 10/19/2022 4:51:05 PM.
IsRegressionStdDev: Marked as regression because -266.90112755820957 (T) = (0 -41148.310623741905) / Math.Sqrt((3665.7040913558403 / (14)) + (2951.1661314950748 / (18))) is less than -2.0422724562973107 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (18) - 2, .025) and -0.1545252275708303 = (35640.89345221125 - 41148.310623741905) / 35640.89345221125 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StreamReaderReadLineTests.ReadLineAsync(LineLengthRange: [ 129, 1024])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 8/17/2022 3:53:31 PM, 10/13/2022 10:51:15 PM, 10/19/2022 4:51:05 PM falls between 10/10/2022 11:56:05 PM and 10/19/2022 4:51:05 PM.
IsRegressionStdDev: Marked as regression because -101.28813670578724 (T) = (0 -38755.007310817884) / Math.Sqrt((30179.82778790027 / (14)) + (11776.349878686373 / (18))) is less than -2.0422724562973107 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (18) - 2, .025) and -0.16082188621647234 = (33385.83444281371 - 38755.007310817884) / 33385.83444281371 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information

Architecture	arm64
OS	Windows 10.0.19041
Baseline	f58cee20c39591835a01c2ffd5f2b3900b279c38
Compare	65d20644b7852307fe076225a4f78395287bdb93
Diff	Diff

Regressions in System.Collections.ContainsTrue<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
SortedSet - Duration of single invocation	30.15 μs	31.68 μs	1.05	0.19	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsTrue<Int32>*'

Related Issues

Regressions

• [Perf] Linux/arm64: 5 Regressions on 10/14/2022 6:08:00 AM

Improvements

• [Perf] Windows/x64: 1 Improvement on 10/14/2022 6:08:00 AM

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.ContainsTrue<Int32>.SortedSet(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 7/27/2022 5:17:06 PM, 8/4/2022 10:02:23 AM, 10/13/2022 10:51:15 PM, 10/19/2022 4:51:05 PM falls between 10/10/2022 11:56:05 PM and 10/19/2022 4:51:05 PM.
IsRegressionStdDev: Marked as regression because -3.843536546770683 (T) = (0 -32500.695041861636) / Math.Sqrt((158786.52650183512 / (14)) + (5561408.635084958 / (18))) is less than -2.0422724562973107 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (18) - 2, .025) and -0.07173133147176837 = (30325.412803999723 - 32500.695041861636) / 30325.412803999723 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	arm64
OS	Windows 10.0.19041
Baseline	f58cee20c39591835a01c2ffd5f2b3900b279c38
Compare	65d20644b7852307fe076225a4f78395287bdb93
Diff	Diff

Regressions in System.Reflection.Invoke

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Field_Get_int - Duration of single invocation	54.48 ns	64.50 ns	1.18	0.14	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Reflection.Invoke*'

Related Issues

Regressions

• [Perf] Linux/arm64: 5 Regressions on 10/14/2022 6:08:00 AM

Improvements

• [Perf] Windows/x64: 1 Improvement on 10/14/2022 6:08:00 AM

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Reflection.Invoke.Field_Get_int

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 9/1/2022 8:04:48 PM, 10/13/2022 10:51:15 PM, 10/19/2022 4:51:05 PM falls between 10/10/2022 11:56:05 PM and 10/19/2022 4:51:05 PM.
IsRegressionStdDev: Marked as regression because -10.816195254539386 (T) = (0 -77.97662589934215) / Math.Sqrt((10.576191551263214 / (14)) + (65.09754190703869 / (18))) is less than -2.0422724562973107 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (18) - 2, .025) and -0.40851798364857467 = (55.36075989413659 - 77.97662589934215) / 55.36075989413659 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[EgorBo]

PGO update, the improvement in Field_get_int is likely dotnet/runtime#76730