[Perf] Linux/x64: 3 Regressions on 4/25/2023 8:51:41 AM

[performanceautofiler]

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	620b0db3a987bab2694a0043cacb3e2a86673df5
Compare	2e0033cf25cee5457ee326dc1eaff013b939d00d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Double

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Parse - Duration of single invocation	106.15 ns	114.73 ns	1.08	0.04	False
Parse - Duration of single invocation	110.88 ns	117.34 ns	1.06	0.02	False
TryParse - Duration of single invocation	103.78 ns	117.34 ns	1.13	0.02	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Double*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Double.Parse(value: "1.7976931348623157e+308")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 114.72776452606185 > 112.64234511867348.
IsChangePoint: Marked as a change because one of 2/16/2023 9:36:08 AM, 3/13/2023 2:14:21 PM, 4/17/2023 6:40:18 PM, 4/25/2023 3:46:39 AM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsRegressionStdDev: Marked as regression because -32.884648198755364 (T) = (0 -113.99280178975746) / Math.Sqrt((0.4901504597440322 / (20)) + (0.3947803557275337 / (30))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (30) - 2, .025) and -0.05930865116405167 = (107.61056436619602 - 113.99280178975746) / 107.61056436619602 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Double.Parse(value: "-1.7976931348623157e+308")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 117.33712036188545 > 116.35218278092339.
IsChangePoint: Marked as a change because one of 2/16/2023 9:36:08 AM, 3/13/2023 2:14:21 PM, 4/17/2023 6:40:18 PM, 4/25/2023 6:46:56 PM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsRegressionStdDev: Marked as regression because -25.09315015042158 (T) = (0 -117.44066061049307) / Math.Sqrt((1.5561279971824895 / (23)) + (0.11046915556988329 / (27))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (27) - 2, .025) and -0.060707276085276124 = (110.71919959286804 - 117.44066061049307) / 110.71919959286804 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Double.TryParse(value: "-1.7976931348623157e+308")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 117.33776759573571 > 110.26771743548994.
IsChangePoint: Marked as a change because one of 2/16/2023 1:21:52 PM, 3/13/2023 2:14:21 PM, 4/25/2023 3:46:39 AM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsRegressionStdDev: Marked as regression because -36.70681844748822 (T) = (0 -116.9210158086416) / Math.Sqrt((1.6630322126402004 / (20)) + (0.6535415888474102 / (30))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (30) - 2, .025) and -0.11321273031426488 = (105.03025398895167 - 116.9210158086416) / 105.03025398895167 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[cincuranet]

Interesting commit ranges are dotnet/runtime@f74138f...19d545c and dotnet/runtime@0fc8978...455150f, but nothing brought my attention...

[kunalspathak]

@jakobbotsch - can you check if the change in dotnet/runtime#85251 affected TryParse()? Don't think we will fix it because #85251 is bringing overall goodness, but we want to just confirm the theory.

[cincuranet]

Similar stuff for Int16 and Double: #17321