[Perf] Linux/arm64: 5 Regressions on 10/14/2022 6:08:00 AM

[performanceautofiler]

Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	f58cee20c39591835a01c2ffd5f2b3900b279c38
Compare	65d20644b7852307fe076225a4f78395287bdb93
Diff	Diff

Regressions in System.Buffers.Text.Tests.Utf8ParserTests

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
TryParseUInt64 - Duration of single invocation	1.12 ns	2.13 ns	1.90	0.69	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Text.Tests.Utf8ParserTests*'

Related Issues

Regressions

Improvements

• [Perf] Windows/x64: 1 Improvement on 10/14/2022 6:08:00 AM

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt64(value: 0)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 2.134521268709774 > 1.1449492208514764.
IsChangePoint: Marked as a change because one of 8/30/2022 6:17:03 PM, 10/13/2022 10:51:15 PM, 10/19/2022 4:51:05 PM falls between 10/7/2022 4:12:34 PM and 10/19/2022 4:51:05 PM.
IsRegressionStdDev: Marked as regression because -15.185131835870473 (T) = (0 -2.0494139053820137) / Math.Sqrt((0.06659524596110658 / (21)) + (0.02206510636395989 / (17))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (17) - 2, .025) and -0.9815228757559231 = (1.0342620468614026 - 2.0494139053820137) / 1.0342620468614026 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information

Architecture	arm64
OS	ubuntu 20.04
Baseline	f58cee20c39591835a01c2ffd5f2b3900b279c38
Compare	65d20644b7852307fe076225a4f78395287bdb93
Diff	Diff

Regressions in System.IO.Tests.StreamReaderReadLineTests

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ReadLineAsync - Duration of single invocation	35.15 μs	40.75 μs	1.16	0.01	False
ReadLineAsync - Duration of single invocation	45.42 μs	50.73 μs	1.12	0.05	False
ReadLineAsync - Duration of single invocation	37.03 μs	43.04 μs	1.16	0.01	False
ReadLineAsync - Duration of single invocation	34.98 μs	40.95 μs	1.17	0.01	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.StreamReaderReadLineTests*'

Related Issues

Regressions

Improvements

• [Perf] Windows/x64: 1 Improvement on 10/14/2022 6:08:00 AM

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.IO.Tests.StreamReaderReadLineTests.ReadLineAsync(LineLengthRange: [ 0, 1024])

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 40.752312512520035 > 36.94492272730752.
IsChangePoint: Marked as a change because one of 8/17/2022 3:53:31 PM, 10/13/2022 10:51:15 PM, 10/19/2022 4:51:05 PM falls between 10/7/2022 4:12:34 PM and 10/19/2022 4:51:05 PM.
IsRegressionStdDev: Marked as regression because -66.69494001973807 (T) = (0 -40573.90835906385) / Math.Sqrt((47351.88201649678 / (21)) + (72420.08712083002 / (17))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (17) - 2, .025) and -0.15297414840761955 = (35190.64882339361 - 40573.90835906385) / 35190.64882339361 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StreamReaderReadLineTests.ReadLineAsync(LineLengthRange: [  33,  128])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 50.73494815782922 > 47.71432393319908.
IsChangePoint: Marked as a change because one of 8/17/2022 3:53:31 PM, 10/13/2022 10:51:15 PM, 10/19/2022 4:51:05 PM falls between 10/7/2022 4:12:34 PM and 10/19/2022 4:51:05 PM.
IsRegressionStdDev: Marked as regression because -18.412108406073738 (T) = (0 -49817.48026571846) / Math.Sqrt((861515.1955845427 / (21)) + (589248.210916688 / (17))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (17) - 2, .025) and -0.11318748502909709 = (44752.10235085989 - 49817.48026571846) / 44752.10235085989 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StreamReaderReadLineTests.ReadLineAsync(LineLengthRange: [1025, 2048])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 43.038771491529296 > 38.76491286178186.
IsChangePoint: Marked as a change because one of 8/17/2022 3:53:31 PM, 10/13/2022 10:51:15 PM, 10/19/2022 4:51:05 PM falls between 10/7/2022 4:12:34 PM and 10/19/2022 4:51:05 PM.
IsRegressionStdDev: Marked as regression because -87.83037655772551 (T) = (0 -43051.58248153989) / Math.Sqrt((66288.38290810505 / (21)) + (26467.548715342782 / (17))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (17) - 2, .025) and -0.16287769681500675 = (37021.591006047674 - 43051.58248153989) / 37021.591006047674 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StreamReaderReadLineTests.ReadLineAsync(LineLengthRange: [ 129, 1024])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 40.95138072404097 > 36.72071263177544.
IsChangePoint: Marked as a change because one of 8/17/2022 3:53:31 PM, 10/13/2022 10:51:15 PM, 10/19/2022 4:51:05 PM falls between 10/7/2022 4:12:34 PM and 10/19/2022 4:51:05 PM.
IsRegressionStdDev: Marked as regression because -79.74532963872964 (T) = (0 -40802.14254523552) / Math.Sqrt((30393.040178183386 / (21)) + (62659.1312864996 / (17))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (17) - 2, .025) and -0.1628277132220444 = (35088.72559648418 - 40802.14254523552) / 35088.72559648418 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[EgorBo]

PGO, cc @AndyAyersMS