[Perf] Linux/x64: 3 Regressions on 11/5/2022 3:05:12 PM

[performanceautofiler]

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	ae1ada9ec291e24749cfdce8f4cad03c624bbf58
Compare	d3ab95d3be895a1950a46c559397780dbb3e9807
Diff	Diff

Regressions in System.IO.Tests.BinaryWriterTests

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
WriteNonAsciiChar - Duration of single invocation	22.59 ns	31.08 ns	1.38	0.64	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.BinaryWriterTests*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.IO.Tests.BinaryWriterTests.WriteNonAsciiChar

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 31.07720332520468 > 24.080739655076236.
IsChangePoint: Marked as a change because one of 9/23/2022 10:31:05 PM, 11/5/2022 1:28:30 PM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -3.531833865313926 (T) = (0 -29.46730582874035) / Math.Sqrt((11.234504408801987 / (29)) + (13.341747260336112 / (8))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (8) - 2, .025) and -0.20746939708202788 = (24.40418440413569 - 29.46730582874035) / 24.40418440413569 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	a20b31ab0c8b9b0aad8fbf72d1c5e91a2095adf0
Compare	d3ab95d3be895a1950a46c559397780dbb3e9807
Diff	Diff

Regressions in System.Buffers.Binary.Tests.BinaryReadAndWriteTests

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ReadStructAndReverseLE - Duration of single invocation	9.32 ns	13.60 ns	1.46	0.61	False
ReadStructFieldByFieldBE - Duration of single invocation	31.22 ns	34.99 ns	1.12	0.50	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Binary.Tests.BinaryReadAndWriteTests*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Buffers.Binary.Tests.BinaryReadAndWriteTests.ReadStructAndReverseLE

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 13.602822849988591 > 10.213792960872157.
IsChangePoint: Marked as a change because one of 9/16/2022 12:02:07 PM, 9/23/2022 4:51:48 AM, 11/3/2022 8:45:56 PM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -6.080708498967696 (T) = (0 -14.045544522004981) / Math.Sqrt((3.4814547112647634 / (27)) + (1.1835777602762014 / (8))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (8) - 2, .025) and -0.2950149383110965 = (10.845855214861524 - 14.045544522004981) / 10.845855214861524 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Binary.Tests.BinaryReadAndWriteTests.ReadStructFieldByFieldBE

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 34.987769801752634 > 30.610675718097777.
IsChangePoint: Marked as a change because one of 9/7/2022 3:19:02 AM, 9/23/2022 10:31:05 PM, 11/3/2022 8:45:56 PM, 11/8/2022 6:56:37 AM falls between 10/30/2022 5:36:44 PM and 11/8/2022 6:56:37 AM.
IsRegressionStdDev: Marked as regression because -6.695647096178333 (T) = (0 -33.77458128857416) / Math.Sqrt((4.63921388309199 / (28)) + (1.2644919688081449 / (8))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (8) - 2, .025) and -0.12714052368267928 = (29.964836308274393 - 33.77458128857416) / 29.964836308274393 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[radekdoulik]

Looks like dotnet/runtime#76339. @stephentoub do you see similar regressions in coreclr or is it mono specific?

[stephentoub]

Looks like dotnet/runtime#76339.

That PR only added new APIs. Other than changing a few existing methods from internal to public, it didn't touch any existing code.