[Perf] Linux/x64: 26 Regressions on 5/2/2023 9:04:55 PM #85989

performanceautofiler · 2023-05-09T15:23:48Z

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	4772b5d2b5facba2bbfcb6b25c4410c1a93d1557
Compare	442141d696ee7d4ae54420974559e99738d753e6
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_UInt64

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
TryParse - Duration of single invocation	23.76 ns	34.01 ns	1.43	0.03	False
ParseSpan - Duration of single invocation	23.26 ns	34.00 ns	1.46	0.03	False
Parse - Duration of single invocation	11.86 ns	15.23 ns	1.28	0.09	False
TryParse - Duration of single invocation	12.63 ns	17.11 ns	1.35	0.05	False
Parse - Duration of single invocation	23.17 ns	35.23 ns	1.52	0.03	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_UInt64*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_UInt64.TryParse(value: "18446744073709551615")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 34.01435809036666 > 25.3743801477778.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/25/2023 6:46:56 PM, 5/2/2023 3:18:41 PM, 5/8/2023 3:52:55 PM falls between 4/29/2023 8:32:21 PM and 5/8/2023 3:52:55 PM.
IsRegressionStdDev: Marked as regression because -47.04559135093131 (T) = (0 -34.297548225823824) / Math.Sqrt((0.8005490307930864 / (23)) + (0.31126691835768755 / (21))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (21) - 2, .025) and -0.44004695426090074 = (23.816965220711797 - 34.297548225823824) / 23.816965220711797 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt64.ParseSpan(value: "18446744073709551615")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 34.0025681010384 > 24.611768631477045.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/25/2023 6:46:56 PM, 5/2/2023 3:18:41 PM, 5/8/2023 3:52:55 PM falls between 4/29/2023 8:32:21 PM and 5/8/2023 3:52:55 PM.
IsRegressionStdDev: Marked as regression because -46.84627991199859 (T) = (0 -34.268259827484265) / Math.Sqrt((0.8428960124932353 / (23)) + (0.27599784156844087 / (21))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (21) - 2, .025) and -0.43893043423743877 = (23.815091412424486 - 34.268259827484265) / 23.815091412424486 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt64.Parse(value: "12345")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.233663593779918 > 12.463789074068497.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/25/2023 6:46:56 PM, 5/2/2023 3:18:41 PM, 5/8/2023 3:52:55 PM falls between 4/29/2023 8:32:21 PM and 5/8/2023 3:52:55 PM.
IsRegressionStdDev: Marked as regression because -17.55416495146352 (T) = (0 -15.950257694152228) / Math.Sqrt((0.56611465688463 / (23)) + (0.459656587047386 / (21))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (21) - 2, .025) and -0.3111793848142055 = (12.164817323155507 - 15.950257694152228) / 12.164817323155507 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt64.TryParse(value: "12345")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 17.110386249177225 > 13.017706441037802.
IsChangePoint: Marked as a change because one of 2/24/2023 8:06:06 PM, 4/17/2023 6:40:18 PM, 4/25/2023 6:46:56 PM, 5/2/2023 3:18:41 PM, 5/8/2023 3:52:55 PM falls between 4/29/2023 8:32:21 PM and 5/8/2023 3:52:55 PM.
IsRegressionStdDev: Marked as regression because -21.476102037711637 (T) = (0 -16.274352469310102) / Math.Sqrt((0.20947989456445193 / (23)) + (0.46853402978353403 / (21))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (21) - 2, .025) and -0.3053286466629373 = (12.467628371533376 - 16.274352469310102) / 12.467628371533376 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt64.Parse(value: "18446744073709551615")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 35.228765884344334 > 24.448686000325765.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/25/2023 6:46:56 PM, 5/2/2023 3:18:41 PM, 5/8/2023 3:52:55 PM falls between 4/29/2023 8:32:21 PM and 5/8/2023 3:52:55 PM.
IsRegressionStdDev: Marked as regression because -30.66097569867656 (T) = (0 -34.29792558643127) / Math.Sqrt((1.8703417116844718 / (23)) + (0.6842109628104064 / (21))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (21) - 2, .025) and -0.4320577592674471 = (23.9500993339654 - 34.29792558643127) / 23.9500993339654 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	4772b5d2b5facba2bbfcb6b25c4410c1a93d1557
Compare	442141d696ee7d4ae54420974559e99738d753e6
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Globalization.Tests.StringEquality

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Compare_Same - Duration of single invocation	538.46 ns	686.08 ns	1.27	0.00	True
Compare_Same_Upper - Duration of single invocation	1.15 μs	2.28 μs	1.98	0.36	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.StringEquality*'

Payloads

Baseline
Compare

Histogram

System.Globalization.Tests.StringEquality.Compare_Same(Count: 1024, Options: (en-US, OrdinalIgnoreCase))

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 686.0821531424299 > 564.3787270447281.
IsChangePoint: Marked as a change because one of 5/2/2023 3:18:41 PM, 5/8/2023 3:52:55 PM falls between 4/29/2023 8:32:21 PM and 5/8/2023 3:52:55 PM.
IsRegressionStdDev: Marked as regression because -579.3427571726128 (T) = (0 -685.6942626411378) / Math.Sqrt((0.725414759068659 / (23)) + (0.714626546587564 / (21))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (21) - 2, .025) and -0.2760794439870314 = (537.3444936145421 - 685.6942626411378) / 537.3444936145421 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Globalization.Tests.StringEquality.Compare_Same_Upper(Count: 1024, Options: (en-US, OrdinalIgnoreCase))

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.27794682493587 > 1.1776218656975188.
IsChangePoint: Marked as a change because one of 5/2/2023 3:18:41 PM, 5/8/2023 3:52:55 PM falls between 4/29/2023 8:32:21 PM and 5/8/2023 3:52:55 PM.
IsRegressionStdDev: Marked as regression because -93.48272309886674 (T) = (0 -2250.1618006414624) / Math.Sqrt((2809.1135448678656 / (23)) + (636.5590553633673 / (21))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (21) - 2, .025) and -1.0531923371231457 = (1095.9332742271495 - 2250.1618006414624) / 1095.9332742271495 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	4772b5d2b5facba2bbfcb6b25c4410c1a93d1557
Compare	442141d696ee7d4ae54420974559e99738d753e6
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.AddGivenSize<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Stack - Duration of single invocation	1.57 μs	1.67 μs	1.06	0.38	False
HashSet - Duration of single invocation	8.88 μs	9.66 μs	1.09	0.01	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.AddGivenSize&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.AddGivenSize<String>.Stack(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.6693939692714146 > 1.6459481093363364.
IsChangePoint: Marked as a change because one of 3/2/2023 4:26:55 AM, 4/9/2023 1:58:52 PM, 4/25/2023 6:46:56 PM, 5/2/2023 3:18:41 PM, 5/8/2023 3:52:55 PM falls between 4/29/2023 8:32:21 PM and 5/8/2023 3:52:55 PM.
IsRegressionStdDev: Marked as regression because -16.106220302535547 (T) = (0 -1729.8309426630458) / Math.Sqrt((50.09530921594631 / (23)) + (2164.523623082486 / (21))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (21) - 2, .025) and -0.10560978515321734 = (1564.5944580920318 - 1729.8309426630458) / 1564.5944580920318 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.AddGivenSize<String>.HashSet(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.659812924644845 > 9.415499566867954.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 5/2/2023 3:18:41 PM, 5/8/2023 3:52:55 PM falls between 4/29/2023 8:32:21 PM and 5/8/2023 3:52:55 PM.
IsRegressionStdDev: Marked as regression because -14.906857164051546 (T) = (0 -9967.163168512167) / Math.Sqrt((5329.6980621772145 / (23)) + (89724.65834760114 / (21))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (21) - 2, .025) and -0.11157541090470617 = (8966.699938423375 - 9967.163168512167) / 8966.699938423375 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	4772b5d2b5facba2bbfcb6b25c4410c1a93d1557
Compare	442141d696ee7d4ae54420974559e99738d753e6
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_UInt16

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
Parse - Duration of single invocation	13.44 ns	16.83 ns	1.25	0.08	False
Parse - Duration of single invocation	13.24 ns	16.24 ns	1.23	0.12	False
Parse - Duration of single invocation	11.03 ns	12.15 ns	1.10	0.29	False
TryParse - Duration of single invocation	14.55 ns	15.74 ns	1.08	0.08	False
TryParse - Duration of single invocation	10.57 ns	12.09 ns	1.14	0.32	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_UInt16*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_UInt16.Parse(value: "65535")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.826146143964596 > 14.119530080196824.
IsChangePoint: Marked as a change because one of 3/3/2023 4:59:02 PM, 3/13/2023 6:11:08 PM, 4/17/2023 1:54:35 PM, 5/2/2023 3:18:41 PM, 5/8/2023 3:52:55 PM falls between 4/29/2023 8:32:21 PM and 5/8/2023 3:52:55 PM.
IsRegressionStdDev: Marked as regression because -46.76185571127924 (T) = (0 -16.518261848600194) / Math.Sqrt((0.02085848296423869 / (23)) + (0.07074501304336984 / (21))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (21) - 2, .025) and -0.2271602380239204 = (13.460558235817132 - 16.518261848600194) / 13.460558235817132 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt16.Parse(value: "12345")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.23877566364392 > 13.90532514864634.
IsChangePoint: Marked as a change because one of 3/3/2023 4:59:02 PM, 3/13/2023 12:04:12 PM, 4/17/2023 6:40:18 PM, 5/2/2023 3:18:41 PM, 5/8/2023 3:52:55 PM falls between 4/29/2023 8:32:21 PM and 5/8/2023 3:52:55 PM.
IsRegressionStdDev: Marked as regression because -48.36895243368514 (T) = (0 -16.200777365552934) / Math.Sqrt((0.005068269745208753 / (23)) + (0.07270830373925728 / (21))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (21) - 2, .025) and -0.22127048370204414 = (13.265511270233459 - 16.200777365552934) / 13.265511270233459 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt16.Parse(value: "0")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.15398815823343 > 10.635719684883263.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/2/2023 3:18:41 PM, 5/8/2023 3:52:55 PM falls between 4/29/2023 8:32:21 PM and 5/8/2023 3:52:55 PM.
IsRegressionStdDev: Marked as regression because -18.676969945448686 (T) = (0 -12.135074364221301) / Math.Sqrt((0.14678046554799734 / (23)) + (0.06144337488222862 / (21))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (21) - 2, .025) and -0.17437792912437924 = (10.333193483352723 - 12.135074364221301) / 10.333193483352723 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt16.TryParse(value: "65535")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.739083277852119 > 15.320294620178725.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/25/2023 6:46:56 PM, 5/2/2023 3:18:41 PM, 5/8/2023 3:52:55 PM falls between 4/29/2023 8:32:21 PM and 5/8/2023 3:52:55 PM.
IsRegressionStdDev: Marked as regression because -16.26890816159469 (T) = (0 -15.79932298951513) / Math.Sqrt((0.034046559937457296 / (23)) + (0.0682911959451056 / (21))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (21) - 2, .025) and -0.07623619696671498 = (14.680163177975475 - 15.79932298951513) / 14.680163177975475 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt16.TryParse(value: "0")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.085731777599758 > 10.746377702684738.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/2/2023 3:18:41 PM, 5/8/2023 3:52:55 PM falls between 4/29/2023 8:32:21 PM and 5/8/2023 3:52:55 PM.
IsRegressionStdDev: Marked as regression because -8.268523248929515 (T) = (0 -11.909829426312127) / Math.Sqrt((0.5631650447733183 / (23)) + (0.09868101589272804 / (21))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (21) - 2, .025) and -0.13456361094530517 = (10.49727781802291 - 11.909829426312127) / 10.49727781802291 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	4772b5d2b5facba2bbfcb6b25c4410c1a93d1557
Compare	442141d696ee7d4ae54420974559e99738d753e6
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Buffers.Text.Tests.Utf8FormatterTests

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
FormatterInt32 - Duration of single invocation	5.68 ns	10.58 ns	1.86	0.04	False
FormatterInt32 - Duration of single invocation	1.55 ns	7.09 ns	4.56	0.17	False
FormatterInt32 - Duration of single invocation	4.45 ns	8.83 ns	1.99	0.07	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Text.Tests.Utf8FormatterTests*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt32(value: 2147483647)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.575831831229376 > 5.957003043233291.
IsChangePoint: Marked as a change because one of 4/9/2023 4:09:45 AM, 4/25/2023 12:29:35 AM, 5/2/2023 3:18:41 PM, 5/8/2023 3:52:55 PM falls between 4/29/2023 8:32:21 PM and 5/8/2023 3:52:55 PM.
IsRegressionStdDev: Marked as regression because -1574.8660868034437 (T) = (0 -10.562699249205188) / Math.Sqrt((0.00012342313623343488 / (23)) + (8.982280015006975E-05 / (21))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (21) - 2, .025) and -0.8622160891109447 = (5.672112549649386 - 10.562699249205188) / 5.672112549649386 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt32(value: 4)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.08956926702956 > 1.6324580612478683.
IsChangePoint: Marked as a change because one of 4/12/2023 4:44:56 PM, 4/25/2023 12:29:35 AM, 5/2/2023 3:18:41 PM, 5/8/2023 3:52:55 PM falls between 4/29/2023 8:32:21 PM and 5/8/2023 3:52:55 PM.
IsRegressionStdDev: Marked as regression because -3087.9906465206955 (T) = (0 -7.088097218182055) / Math.Sqrt((4.623066496265888E-06 / (23)) + (6.317721837668212E-05 / (21))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (21) - 2, .025) and -3.5553670501149837 = (1.555988165213414 - 7.088097218182055) / 1.555988165213414 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt32(value: 12345)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.830698653458 > 4.6845527687024315.
IsChangePoint: Marked as a change because one of 4/9/2023 4:09:45 AM, 4/25/2023 12:29:35 AM, 5/2/2023 3:18:41 PM, 5/8/2023 3:52:55 PM falls between 4/29/2023 8:32:21 PM and 5/8/2023 3:52:55 PM.
IsRegressionStdDev: Marked as regression because -118.10554486920714 (T) = (0 -8.822609142330025) / Math.Sqrt((0.03070856775007168 / (23)) + (0.00021011967589717185 / (21))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (21) - 2, .025) and -0.9645442995820404 = (4.490918908882354 - 8.822609142330025) / 4.490918908882354 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	4772b5d2b5facba2bbfcb6b25c4410c1a93d1557
Compare	442141d696ee7d4ae54420974559e99738d753e6
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.ContainsKeyFalse<String, String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ImmutableDictionary - Duration of single invocation	27.24 μs	30.06 μs	1.10	0.04	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsKeyFalse&lt;String, String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsKeyFalse<String, String>.ImmutableDictionary(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 30.06372096072114 > 28.678544994103074.
IsChangePoint: Marked as a change because one of 4/5/2023 11:16:13 AM, 4/25/2023 6:46:56 PM, 5/2/2023 3:18:41 PM, 5/8/2023 3:52:55 PM falls between 4/29/2023 8:32:21 PM and 5/8/2023 3:52:55 PM.
IsRegressionStdDev: Marked as regression because -30.840691194083934 (T) = (0 -29983.887024653) / Math.Sqrt((58502.37431372103 / (23)) + (101199.03982157727 / (21))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (21) - 2, .025) and -0.09680082329814221 = (27337.58617584709 - 29983.887024653) / 27337.58617584709 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	4772b5d2b5facba2bbfcb6b25c4410c1a93d1557
Compare	442141d696ee7d4ae54420974559e99738d753e6
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.ContainsFalse<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ImmutableHashSet - Duration of single invocation	30.32 μs	32.73 μs	1.08	0.03	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsFalse&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsFalse<String>.ImmutableHashSet(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 32.72961513211054 > 32.118814469121496.
IsChangePoint: Marked as a change because one of 4/5/2023 6:15:03 PM, 4/25/2023 6:46:56 PM, 5/2/2023 3:18:41 PM, 5/8/2023 3:52:55 PM falls between 4/29/2023 8:32:21 PM and 5/8/2023 3:52:55 PM.
IsRegressionStdDev: Marked as regression because -17.91965283205727 (T) = (0 -32618.762840887095) / Math.Sqrt((139903.52594900012 / (23)) + (149967.47578337672 / (21))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (21) - 2, .025) and -0.06743512581233914 = (30558.07519549591 - 32618.762840887095) / 30558.07519549591 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	4772b5d2b5facba2bbfcb6b25c4410c1a93d1557
Compare	442141d696ee7d4ae54420974559e99738d753e6
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Perf_Utf8Encoding

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
GetString - Duration of single invocation	185.21 μs	219.41 μs	1.18	0.41	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Perf_Utf8Encoding*'

Payloads

Baseline
Compare

Histogram

System.Text.Perf_Utf8Encoding.GetString(Input: Chinese)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 219.4075287078373 > 195.10829571693026.
IsChangePoint: Marked as a change because one of 4/6/2023 1:55:21 PM, 4/25/2023 6:46:56 PM, 5/2/2023 3:18:41 PM, 5/8/2023 3:52:55 PM falls between 4/29/2023 8:32:21 PM and 5/8/2023 3:52:55 PM.
IsRegressionStdDev: Marked as regression because -58.20746821828837 (T) = (0 -219761.94657888898) / Math.Sqrt((860357.8866407182 / (23)) + (6260099.308236786 / (21))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (21) - 2, .025) and -0.1812207718516285 = (186046.46296085705 - 219761.94657888898) / 186046.46296085705 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

The text was updated successfully, but these errors were encountered:

performanceautofiler · 2023-05-09T15:23:53Z

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	3e8f17a65a068fca3d19fa5cd43a7e1cd414a5ae
Compare	442141d696ee7d4ae54420974559e99738d753e6
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in Benchmark.GetChildKeysTests

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
AddChainedConfigurationNoDelimiter - Duration of single invocation	614.56 μs	691.19 μs	1.12	0.02	False
AddChainedConfigurationEmpty - Duration of single invocation	10.43 ms	28.76 ms	2.76	0.40	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchmark.GetChildKeysTests*'

Payloads

Baseline
Compare

Histogram

Benchmark.GetChildKeysTests.AddChainedConfigurationNoDelimiter

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 691.1870628881987 > 648.0551650379069.
IsChangePoint: Marked as a change because one of 4/19/2023 1:39:34 AM, 5/2/2023 3:18:41 PM, 5/8/2023 3:52:55 PM falls between 4/29/2023 8:32:21 PM and 5/8/2023 3:52:55 PM.
IsRegressionStdDev: Marked as regression because -9.538408082504391 (T) = (0 -669053.6250722164) / Math.Sqrt((194146205.87499458 / (22)) + (208740830.1342201 / (21))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (21) - 2, .025) and -0.06582221373880205 = (627734.7351630442 - 669053.6250722164) / 627734.7351630442 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Benchmark.GetChildKeysTests.AddChainedConfigurationEmpty

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 28.761574888888887 > 10.922525795416668.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 5/5/2023 3:18:41 PM, 5/8/2023 3:52:55 PM falls between 4/29/2023 8:32:21 PM and 5/8/2023 3:52:55 PM.
IsRegressionStdDev: Marked as regression because -36.91362847216741 (T) = (0 -28747999.36971001) / Math.Sqrt((1232956503218.2644 / (34)) + (1960290041084.2651 / (10))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (10) - 2, .025) and -1.6237492826160578 = (10956839.344438538 - 28747999.36971001) / 10956839.344438538 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	4772b5d2b5facba2bbfcb6b25c4410c1a93d1557
Compare	442141d696ee7d4ae54420974559e99738d753e6
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.CtorFromCollection<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
HashSet - Duration of single invocation	11.27 μs	12.31 μs	1.09	0.02	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorFromCollection&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.CtorFromCollection<String>.HashSet(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.314636443983911 > 11.881346597006926.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 5/2/2023 3:18:41 PM, 5/8/2023 3:52:55 PM falls between 4/29/2023 8:32:21 PM and 5/8/2023 3:52:55 PM.
IsRegressionStdDev: Marked as regression because -13.79665625480297 (T) = (0 -12617.95000391291) / Math.Sqrt((6061.447485116022 / (23)) + (174431.12459801493 / (21))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (21) - 2, .025) and -0.11262038947747587 = (11340.750289358553 - 12617.95000391291) / 11340.750289358553 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	4772b5d2b5facba2bbfcb6b25c4410c1a93d1557
Compare	442141d696ee7d4ae54420974559e99738d753e6
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.TryGetValueFalse<String, String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ImmutableDictionary - Duration of single invocation	27.24 μs	29.58 μs	1.09	0.01	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryGetValueFalse&lt;String, String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.TryGetValueFalse<String, String>.ImmutableDictionary(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 29.5817267845912 > 29.01065883294753.
IsChangePoint: Marked as a change because one of 4/5/2023 6:15:03 PM, 4/25/2023 6:46:56 PM, 5/2/2023 3:18:41 PM, 5/8/2023 3:52:55 PM falls between 4/29/2023 8:32:21 PM and 5/8/2023 3:52:55 PM.
IsRegressionStdDev: Marked as regression because -18.133331424962236 (T) = (0 -29676.86438856725) / Math.Sqrt((88931.83283086984 / (23)) + (199080.3953755346 / (21))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (21) - 2, .025) and -0.07595179712559556 = (27581.964608311424 - 29676.86438856725) / 27581.964608311424 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	4772b5d2b5facba2bbfcb6b25c4410c1a93d1557
Compare	442141d696ee7d4ae54420974559e99738d753e6
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.ContainsTrue<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
HashSet - Duration of single invocation	7.93 μs	9.53 μs	1.20	0.02	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsTrue&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsTrue<String>.HashSet(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.53326783809959 > 8.427870448664828.
IsChangePoint: Marked as a change because one of 2/25/2023 3:45:56 AM, 4/25/2023 6:46:56 PM, 5/2/2023 3:18:41 PM, 5/8/2023 3:52:55 PM falls between 4/29/2023 8:32:21 PM and 5/8/2023 3:52:55 PM.
IsRegressionStdDev: Marked as regression because -40.61725093443036 (T) = (0 -9588.879345410225) / Math.Sqrt((10545.336482174154 / (23)) + (20410.437866570424 / (21))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (21) - 2, .025) and -0.1907658346753327 = (8052.699419298231 - 9588.879345410225) / 8052.699419298231 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	4772b5d2b5facba2bbfcb6b25c4410c1a93d1557
Compare	442141d696ee7d4ae54420974559e99738d753e6
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in PerfLabTests.CastingPerf2.CastingPerf

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
IFooObjIsIFooInterAlia - Duration of single invocation	327.20 μs	375.00 μs	1.15	0.20	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.CastingPerf2.CastingPerf*'

Payloads

Baseline
Compare

Histogram

PerfLabTests.CastingPerf2.CastingPerf.IFooObjIsIFooInterAlia

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 374.99534365079364 > 342.9615589029948.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/2/2023 3:18:41 PM, 5/8/2023 3:52:55 PM falls between 4/29/2023 8:32:21 PM and 5/8/2023 3:52:55 PM.
IsRegressionStdDev: Marked as regression because -27.208384159897836 (T) = (0 -380677.6481669433) / Math.Sqrt((94391.71942150916 / (23)) + (82173026.93236075 / (21))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (21) - 2, .025) and -0.16476559400606464 = (326827.68973081565 - 380677.6481669433) / 326827.68973081565 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

cincuranet · 2023-05-09T16:42:01Z

Commit range is c62f69b...2a216ab. Probably PGO...

AndyAyersMS · 2023-05-09T18:29:21Z

Likely a PGO update, but we usually don't see perf swings quite this large. So need to see if there's somethimg amiss with the PGO data, or the way the JIT interacts with it...

ghost · 2023-05-09T18:29:50Z

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.

Issue Details

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	4772b5d2b5facba2bbfcb6b25c4410c1a93d1557
Compare	442141d696ee7d4ae54420974559e99738d753e6
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_UInt64

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
TryParse - Duration of single invocation	23.76 ns	34.01 ns	1.43	0.03	False
ParseSpan - Duration of single invocation	23.26 ns	34.00 ns	1.46	0.03	False
Parse - Duration of single invocation	11.86 ns	15.23 ns	1.28	0.09	False
TryParse - Duration of single invocation	12.63 ns	17.11 ns	1.35	0.05	False
Parse - Duration of single invocation	23.17 ns	35.23 ns	1.52	0.03	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_UInt64*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_UInt64.TryParse(value: "18446744073709551615")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 34.01435809036666 > 25.3743801477778.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/25/2023 6:46:56 PM, 5/2/2023 3:18:41 PM, 5/8/2023 3:52:55 PM falls between 4/29/2023 8:32:21 PM and 5/8/2023 3:52:55 PM.
IsRegressionStdDev: Marked as regression because -47.04559135093131 (T) = (0 -34.297548225823824) / Math.Sqrt((0.8005490307930864 / (23)) + (0.31126691835768755 / (21))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (21) - 2, .025) and -0.44004695426090074 = (23.816965220711797 - 34.297548225823824) / 23.816965220711797 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt64.ParseSpan(value: "18446744073709551615")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 34.0025681010384 > 24.611768631477045.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/25/2023 6:46:56 PM, 5/2/2023 3:18:41 PM, 5/8/2023 3:52:55 PM falls between 4/29/2023 8:32:21 PM and 5/8/2023 3:52:55 PM.
IsRegressionStdDev: Marked as regression because -46.84627991199859 (T) = (0 -34.268259827484265) / Math.Sqrt((0.8428960124932353 / (23)) + (0.27599784156844087 / (21))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (21) - 2, .025) and -0.43893043423743877 = (23.815091412424486 - 34.268259827484265) / 23.815091412424486 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt64.Parse(value: "12345")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.233663593779918 > 12.463789074068497.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/25/2023 6:46:56 PM, 5/2/2023 3:18:41 PM, 5/8/2023 3:52:55 PM falls between 4/29/2023 8:32:21 PM and 5/8/2023 3:52:55 PM.
IsRegressionStdDev: Marked as regression because -17.55416495146352 (T) = (0 -15.950257694152228) / Math.Sqrt((0.56611465688463 / (23)) + (0.459656587047386 / (21))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (21) - 2, .025) and -0.3111793848142055 = (12.164817323155507 - 15.950257694152228) / 12.164817323155507 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt64.TryParse(value: "12345")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 17.110386249177225 > 13.017706441037802.
IsChangePoint: Marked as a change because one of 2/24/2023 8:06:06 PM, 4/17/2023 6:40:18 PM, 4/25/2023 6:46:56 PM, 5/2/2023 3:18:41 PM, 5/8/2023 3:52:55 PM falls between 4/29/2023 8:32:21 PM and 5/8/2023 3:52:55 PM.
IsRegressionStdDev: Marked as regression because -21.476102037711637 (T) = (0 -16.274352469310102) / Math.Sqrt((0.20947989456445193 / (23)) + (0.46853402978353403 / (21))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (21) - 2, .025) and -0.3053286466629373 = (12.467628371533376 - 16.274352469310102) / 12.467628371533376 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt64.Parse(value: "18446744073709551615")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 35.228765884344334 > 24.448686000325765.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/25/2023 6:46:56 PM, 5/2/2023 3:18:41 PM, 5/8/2023 3:52:55 PM falls between 4/29/2023 8:32:21 PM and 5/8/2023 3:52:55 PM.
IsRegressionStdDev: Marked as regression because -30.66097569867656 (T) = (0 -34.29792558643127) / Math.Sqrt((1.8703417116844718 / (23)) + (0.6842109628104064 / (21))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (21) - 2, .025) and -0.4320577592674471 = (23.9500993339654 - 34.29792558643127) / 23.9500993339654 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	4772b5d2b5facba2bbfcb6b25c4410c1a93d1557
Compare	442141d696ee7d4ae54420974559e99738d753e6
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Globalization.Tests.StringEquality

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Compare_Same - Duration of single invocation	538.46 ns	686.08 ns	1.27	0.00	True
Compare_Same_Upper - Duration of single invocation	1.15 μs	2.28 μs	1.98	0.36	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.StringEquality*'

Payloads

Baseline
Compare

Histogram

System.Globalization.Tests.StringEquality.Compare_Same(Count: 1024, Options: (en-US, OrdinalIgnoreCase))

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 686.0821531424299 > 564.3787270447281.
IsChangePoint: Marked as a change because one of 5/2/2023 3:18:41 PM, 5/8/2023 3:52:55 PM falls between 4/29/2023 8:32:21 PM and 5/8/2023 3:52:55 PM.
IsRegressionStdDev: Marked as regression because -579.3427571726128 (T) = (0 -685.6942626411378) / Math.Sqrt((0.725414759068659 / (23)) + (0.714626546587564 / (21))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (21) - 2, .025) and -0.2760794439870314 = (537.3444936145421 - 685.6942626411378) / 537.3444936145421 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Globalization.Tests.StringEquality.Compare_Same_Upper(Count: 1024, Options: (en-US, OrdinalIgnoreCase))

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.27794682493587 > 1.1776218656975188.
IsChangePoint: Marked as a change because one of 5/2/2023 3:18:41 PM, 5/8/2023 3:52:55 PM falls between 4/29/2023 8:32:21 PM and 5/8/2023 3:52:55 PM.
IsRegressionStdDev: Marked as regression because -93.48272309886674 (T) = (0 -2250.1618006414624) / Math.Sqrt((2809.1135448678656 / (23)) + (636.5590553633673 / (21))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (21) - 2, .025) and -1.0531923371231457 = (1095.9332742271495 - 2250.1618006414624) / 1095.9332742271495 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	4772b5d2b5facba2bbfcb6b25c4410c1a93d1557
Compare	442141d696ee7d4ae54420974559e99738d753e6
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.AddGivenSize<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Stack - Duration of single invocation	1.57 μs	1.67 μs	1.06	0.38	False
HashSet - Duration of single invocation	8.88 μs	9.66 μs	1.09	0.01	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.AddGivenSize&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.AddGivenSize<String>.Stack(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.6693939692714146 > 1.6459481093363364.
IsChangePoint: Marked as a change because one of 3/2/2023 4:26:55 AM, 4/9/2023 1:58:52 PM, 4/25/2023 6:46:56 PM, 5/2/2023 3:18:41 PM, 5/8/2023 3:52:55 PM falls between 4/29/2023 8:32:21 PM and 5/8/2023 3:52:55 PM.
IsRegressionStdDev: Marked as regression because -16.106220302535547 (T) = (0 -1729.8309426630458) / Math.Sqrt((50.09530921594631 / (23)) + (2164.523623082486 / (21))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (21) - 2, .025) and -0.10560978515321734 = (1564.5944580920318 - 1729.8309426630458) / 1564.5944580920318 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.AddGivenSize<String>.HashSet(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.659812924644845 > 9.415499566867954.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 5/2/2023 3:18:41 PM, 5/8/2023 3:52:55 PM falls between 4/29/2023 8:32:21 PM and 5/8/2023 3:52:55 PM.
IsRegressionStdDev: Marked as regression because -14.906857164051546 (T) = (0 -9967.163168512167) / Math.Sqrt((5329.6980621772145 / (23)) + (89724.65834760114 / (21))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (21) - 2, .025) and -0.11157541090470617 = (8966.699938423375 - 9967.163168512167) / 8966.699938423375 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	4772b5d2b5facba2bbfcb6b25c4410c1a93d1557
Compare	442141d696ee7d4ae54420974559e99738d753e6
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_UInt16

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
Parse - Duration of single invocation	13.44 ns	16.83 ns	1.25	0.08	False
Parse - Duration of single invocation	13.24 ns	16.24 ns	1.23	0.12	False
Parse - Duration of single invocation	11.03 ns	12.15 ns	1.10	0.29	False
TryParse - Duration of single invocation	14.55 ns	15.74 ns	1.08	0.08	False
TryParse - Duration of single invocation	10.57 ns	12.09 ns	1.14	0.32	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_UInt16*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_UInt16.Parse(value: "65535")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.826146143964596 > 14.119530080196824.
IsChangePoint: Marked as a change because one of 3/3/2023 4:59:02 PM, 3/13/2023 6:11:08 PM, 4/17/2023 1:54:35 PM, 5/2/2023 3:18:41 PM, 5/8/2023 3:52:55 PM falls between 4/29/2023 8:32:21 PM and 5/8/2023 3:52:55 PM.
IsRegressionStdDev: Marked as regression because -46.76185571127924 (T) = (0 -16.518261848600194) / Math.Sqrt((0.02085848296423869 / (23)) + (0.07074501304336984 / (21))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (21) - 2, .025) and -0.2271602380239204 = (13.460558235817132 - 16.518261848600194) / 13.460558235817132 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt16.Parse(value: "12345")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.23877566364392 > 13.90532514864634.
IsChangePoint: Marked as a change because one of 3/3/2023 4:59:02 PM, 3/13/2023 12:04:12 PM, 4/17/2023 6:40:18 PM, 5/2/2023 3:18:41 PM, 5/8/2023 3:52:55 PM falls between 4/29/2023 8:32:21 PM and 5/8/2023 3:52:55 PM.
IsRegressionStdDev: Marked as regression because -48.36895243368514 (T) = (0 -16.200777365552934) / Math.Sqrt((0.005068269745208753 / (23)) + (0.07270830373925728 / (21))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (21) - 2, .025) and -0.22127048370204414 = (13.265511270233459 - 16.200777365552934) / 13.265511270233459 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt16.Parse(value: "0")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.15398815823343 > 10.635719684883263.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/2/2023 3:18:41 PM, 5/8/2023 3:52:55 PM falls between 4/29/2023 8:32:21 PM and 5/8/2023 3:52:55 PM.
IsRegressionStdDev: Marked as regression because -18.676969945448686 (T) = (0 -12.135074364221301) / Math.Sqrt((0.14678046554799734 / (23)) + (0.06144337488222862 / (21))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (21) - 2, .025) and -0.17437792912437924 = (10.333193483352723 - 12.135074364221301) / 10.333193483352723 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt16.TryParse(value: "65535")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.739083277852119 > 15.320294620178725.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/25/2023 6:46:56 PM, 5/2/2023 3:18:41 PM, 5/8/2023 3:52:55 PM falls between 4/29/2023 8:32:21 PM and 5/8/2023 3:52:55 PM.
IsRegressionStdDev: Marked as regression because -16.26890816159469 (T) = (0 -15.79932298951513) / Math.Sqrt((0.034046559937457296 / (23)) + (0.0682911959451056 / (21))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (21) - 2, .025) and -0.07623619696671498 = (14.680163177975475 - 15.79932298951513) / 14.680163177975475 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt16.TryParse(value: "0")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.085731777599758 > 10.746377702684738.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/2/2023 3:18:41 PM, 5/8/2023 3:52:55 PM falls between 4/29/2023 8:32:21 PM and 5/8/2023 3:52:55 PM.
IsRegressionStdDev: Marked as regression because -8.268523248929515 (T) = (0 -11.909829426312127) / Math.Sqrt((0.5631650447733183 / (23)) + (0.09868101589272804 / (21))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (21) - 2, .025) and -0.13456361094530517 = (10.49727781802291 - 11.909829426312127) / 10.49727781802291 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	4772b5d2b5facba2bbfcb6b25c4410c1a93d1557
Compare	442141d696ee7d4ae54420974559e99738d753e6
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Buffers.Text.Tests.Utf8FormatterTests

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
FormatterInt32 - Duration of single invocation	5.68 ns	10.58 ns	1.86	0.04	False
FormatterInt32 - Duration of single invocation	1.55 ns	7.09 ns	4.56	0.17	False
FormatterInt32 - Duration of single invocation	4.45 ns	8.83 ns	1.99	0.07	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Text.Tests.Utf8FormatterTests*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt32(value: 2147483647)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.575831831229376 > 5.957003043233291.
IsChangePoint: Marked as a change because one of 4/9/2023 4:09:45 AM, 4/25/2023 12:29:35 AM, 5/2/2023 3:18:41 PM, 5/8/2023 3:52:55 PM falls between 4/29/2023 8:32:21 PM and 5/8/2023 3:52:55 PM.
IsRegressionStdDev: Marked as regression because -1574.8660868034437 (T) = (0 -10.562699249205188) / Math.Sqrt((0.00012342313623343488 / (23)) + (8.982280015006975E-05 / (21))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (21) - 2, .025) and -0.8622160891109447 = (5.672112549649386 - 10.562699249205188) / 5.672112549649386 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt32(value: 4)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.08956926702956 > 1.6324580612478683.
IsChangePoint: Marked as a change because one of 4/12/2023 4:44:56 PM, 4/25/2023 12:29:35 AM, 5/2/2023 3:18:41 PM, 5/8/2023 3:52:55 PM falls between 4/29/2023 8:32:21 PM and 5/8/2023 3:52:55 PM.
IsRegressionStdDev: Marked as regression because -3087.9906465206955 (T) = (0 -7.088097218182055) / Math.Sqrt((4.623066496265888E-06 / (23)) + (6.317721837668212E-05 / (21))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (21) - 2, .025) and -3.5553670501149837 = (1.555988165213414 - 7.088097218182055) / 1.555988165213414 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt32(value: 12345)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.830698653458 > 4.6845527687024315.
IsChangePoint: Marked as a change because one of 4/9/2023 4:09:45 AM, 4/25/2023 12:29:35 AM, 5/2/2023 3:18:41 PM, 5/8/2023 3:52:55 PM falls between 4/29/2023 8:32:21 PM and 5/8/2023 3:52:55 PM.
IsRegressionStdDev: Marked as regression because -118.10554486920714 (T) = (0 -8.822609142330025) / Math.Sqrt((0.03070856775007168 / (23)) + (0.00021011967589717185 / (21))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (21) - 2, .025) and -0.9645442995820404 = (4.490918908882354 - 8.822609142330025) / 4.490918908882354 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	4772b5d2b5facba2bbfcb6b25c4410c1a93d1557
Compare	442141d696ee7d4ae54420974559e99738d753e6
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.ContainsKeyFalse<String, String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ImmutableDictionary - Duration of single invocation	27.24 μs	30.06 μs	1.10	0.04	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsKeyFalse&lt;String, String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsKeyFalse<String, String>.ImmutableDictionary(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 30.06372096072114 > 28.678544994103074.
IsChangePoint: Marked as a change because one of 4/5/2023 11:16:13 AM, 4/25/2023 6:46:56 PM, 5/2/2023 3:18:41 PM, 5/8/2023 3:52:55 PM falls between 4/29/2023 8:32:21 PM and 5/8/2023 3:52:55 PM.
IsRegressionStdDev: Marked as regression because -30.840691194083934 (T) = (0 -29983.887024653) / Math.Sqrt((58502.37431372103 / (23)) + (101199.03982157727 / (21))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (21) - 2, .025) and -0.09680082329814221 = (27337.58617584709 - 29983.887024653) / 27337.58617584709 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	4772b5d2b5facba2bbfcb6b25c4410c1a93d1557
Compare	442141d696ee7d4ae54420974559e99738d753e6
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.ContainsFalse<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ImmutableHashSet - Duration of single invocation	30.32 μs	32.73 μs	1.08	0.03	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsFalse&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsFalse<String>.ImmutableHashSet(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 32.72961513211054 > 32.118814469121496.
IsChangePoint: Marked as a change because one of 4/5/2023 6:15:03 PM, 4/25/2023 6:46:56 PM, 5/2/2023 3:18:41 PM, 5/8/2023 3:52:55 PM falls between 4/29/2023 8:32:21 PM and 5/8/2023 3:52:55 PM.
IsRegressionStdDev: Marked as regression because -17.91965283205727 (T) = (0 -32618.762840887095) / Math.Sqrt((139903.52594900012 / (23)) + (149967.47578337672 / (21))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (21) - 2, .025) and -0.06743512581233914 = (30558.07519549591 - 32618.762840887095) / 30558.07519549591 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	4772b5d2b5facba2bbfcb6b25c4410c1a93d1557
Compare	442141d696ee7d4ae54420974559e99738d753e6
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Perf_Utf8Encoding

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
GetString - Duration of single invocation	185.21 μs	219.41 μs	1.18	0.41	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Perf_Utf8Encoding*'

Payloads

Baseline
Compare

Histogram

System.Text.Perf_Utf8Encoding.GetString(Input: Chinese)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 219.4075287078373 > 195.10829571693026.
IsChangePoint: Marked as a change because one of 4/6/2023 1:55:21 PM, 4/25/2023 6:46:56 PM, 5/2/2023 3:18:41 PM, 5/8/2023 3:52:55 PM falls between 4/29/2023 8:32:21 PM and 5/8/2023 3:52:55 PM.
IsRegressionStdDev: Marked as regression because -58.20746821828837 (T) = (0 -219761.94657888898) / Math.Sqrt((860357.8866407182 / (23)) + (6260099.308236786 / (21))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (21) - 2, .025) and -0.1812207718516285 = (186046.46296085705 - 219761.94657888898) / 186046.46296085705 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Author:	performanceautofiler[bot]
Assignees:	AndyAyersMS
Labels:	`os-linux`, `arch-x64`, `area-CodeGen-coreclr`, `runtime-coreclr`, `PGO`, `needs-area-label`
Milestone:	-

AndyAyersMS · 2023-05-10T17:04:41Z

Benchmark.GetChildKeysTests.AddChainedConfigurationEmpty

[link]

Looking at the most dramatic slowdown:

The big improvement on April 17 seems to fall in the range 223d152...9aefa9d -- the only possible candidate is #84582

AndyAyersMS · 2023-05-10T19:22:05Z

Can repro

Method	Job	Toolchain	Mean	Error	StdDev	Median	Min	Max	Ratio	RatioSD	Allocated	Alloc Ratio
AddChainedConfigurationEmpty	Job-JFZJCV	\base-rel\corerun.exe	12.61 ms	0.236 ms	0.221 ms	12.66 ms	12.29 ms	13.03 ms	1.00	0.00	143.24 KB	1.00
AddChainedConfigurationEmpty	Job-UJTJCB	\diff-rel\corerun.exe	25.80 ms	0.427 ms	0.379 ms	25.90 ms	25.01 ms	26.32 ms	2.05	0.04	143.32 KB	1.00

;; base 0535b03e15fe8c6751a99ff3f64cf08ae812e99b

50.26%   1.827E+07   Tier-1   [System.Private.CoreLib]Number.TryParseBinaryIntegerStyle(value class System.ReadOnlySpan`1<wchar>,value class System.Globalization.NumberStyles,class System.Globalization.NumberFormatInfo,!!0&)
27.68%   1.006E+07   Tier-1   [System.Private.CoreLib]Ordinal.CompareStringIgnoreCase(wchar&,int32,wchar&,int32)
08.25%   3E+06       Tier-1   [System.Private.CoreLib]OrdinalCasing.LastIndexOf(value class System.ReadOnlySpan`1<wchar>,value class System.ReadOnlySpan`1<wchar>)
05.20%   1.89E+06    Tier-1   [System.Private.CoreLib]PackedSpanHelpers.IndexOf(int16&,int16,int32)
01.90%   6.9E+05     native   coreclr.dll
01.07%   3.9E+05     Tier-1   [Microsoft.Extensions.Configuration]ConfigurationKeyComparer.Compare(class System.String,class System.String)
01.02%   3.7E+05     Tier-1   [System.Private.CoreLib]String.GetNonRandomizedHashCodeOrdinalIgnoreCase()

;; diff 52b81afe83ba93ea786708a66abf3985419bbea7

51.29%   3.705E+07   Tier-1   [System.Private.CoreLib]Number.TryParseBinaryIntegerStyle(value class System.ReadOnlySpan`1<wchar>,value class System.Globalization.NumberStyles,class System.Globalization.NumberFormatInfo,!!0&)
24.61%   1.778E+07   Tier-1   [System.Private.CoreLib]Number.IsWhite(int32)
13.12%   9.48E+06    Tier-1   [System.Private.CoreLib]Ordinal.CompareStringIgnoreCase(wchar&,int32,wchar&,int32)
04.11%   2.97E+06    Tier-1   [System.Private.CoreLib]OrdinalCasing.LastIndexOf(value class System.ReadOnlySpan`1<wchar>,value class System.ReadOnlySpan`1<wchar>)
02.99%   2.16E+06    Tier-1   [System.Private.CoreLib]PackedSpanHelpers.IndexOf(int16&,int16,int32)

Note this could be stale PGO data -- so might resolve itself with another PGO update. We took one with #85866 and just took another one with #85968.

But let's drill in and see....

AndyAyersMS · 2023-05-10T19:41:06Z

In case you're wondering how AddChainedConfigurationEmpty ends up spending basically all of its time parsing integers, it is sorting on strings using a custom comparer that eventually bottoms out here:

runtime/src/libraries/Microsoft.Extensions.Configuration/src/ConfigurationKeyComparer.cs

Lines 70 to 98 in d80ba2f

    
                       static int Compare(ReadOnlySpan<char> a, ReadOnlySpan<char> b) 
        
                       { 
        
           #if NETCOREAPP 
        
                           bool aIsInt = int.TryParse(a, out int value1); 
        
                           bool bIsInt = int.TryParse(b, out int value2); 
        
           #else 
        
                           bool aIsInt = int.TryParse(a.ToString(), out int value1); 
        
                           bool bIsInt = int.TryParse(b.ToString(), out int value2); 
        
           #endif 
        
                           int result; 
        
                           if (!aIsInt && !bIsInt) 
        
                           { 
        
                               // Both are strings 
        
                               result = a.CompareTo(b, StringComparison.OrdinalIgnoreCase); 
        
                           } 
        
                           else if (aIsInt && bIsInt) 
        
                           { 
        
                               // Both are int 
        
                               result = value1 - value2; 
        
                           } 
        
                           else 
        
                           { 
        
                               // Only one of them is int 
        
                               result = aIsInt ? -1 : 1; 
        
                           } 
        
                           return result; 
        
                       }

So not too surprising that it gets lumped in with parsing benchmarks like we see here.

AndyAyersMS · 2023-05-10T22:36:14Z

Comparing jit dumps/disasm shows that the diff version has static pgo. This impacts inlining; some of the calls to IsWhite

;; base

Inlines into 06001907 [via ExtendedDefaultPolicy] System.Number:TryParseBinaryIntegerStyle[int](System.ReadOnlySpan`1[ushort],int,System.Globalization.NumberFormatInfo,byref):int:
  [INL01 IL=0002 TR=000001 06001A8B] [INLINED: callee: below ALWAYS_INLINE size] System.ReadOnlySpan`1[ushort]:get_IsEmpty():bool:this
  [INL02 IL=0030 TR=000762 06001919] [INLINED: call site: profitable inline] System.Number:IsWhite(int):bool
  [INL03 IL=0065 TR=000793 06001919] [INLINED: call site: profitable inline] System.Number:IsWhite(int):bool
...
  [INL65 IL=1067 TR=000119 06001919] [INLINED: call site: profitable inline] System.Number:IsWhite(int):bool
  [INL66 IL=1094 TR=000213 06001919] [INLINED: call site: profitable inline] System.Number:IsWhite(int):bool

;; diff

Inlines into 06001905 [via ExtendedDefaultPolicy] System.Number:TryParseBinaryIntegerStyle[int](System.ReadOnlySpan`1[ushort],int,System.Globalization.NumberFormatInfo,byref):int:
  [INL01 IL=0002 TR=000001 06001A89] [INLINED: callee: below ALWAYS_INLINE size] System.ReadOnlySpan`1[ushort]:get_IsEmpty():bool:this
  [INL02 IL=0030 TR=000798 06001917] [INLINED: call site: profitable inline] System.Number:IsWhite(int):bool
  [INL00 IL=0065 TR=000832 06001917] [FAILED: call site: unprofitable inline] System.Number:IsWhite(int):bool
...
  [INL00 IL=1067 TR=000124 06001917] [FAILED: call site: unprofitable inline] System.Number:IsWhite(int):bool
  [INL00 IL=1094 TR=000221 06001917] [FAILED: call site: unprofitable inline] System.Number:IsWhite(int):bool

So this suggests that the static PGO data is not a good match for this test case, since we see IsWhite prominent in the diff profiles.

IsWhite is called in a loop at the very start of TryParseBinaryIntegerStyle. As you can see from the profile data, we never observe this loop to execute and so the in loop call in BB05 is considered cold and not inlined.

AndyAyersMS · 2023-05-10T22:42:22Z

This particular test case fills a dictionary with "empty keys" which are just whitespace:

https://github.com/dotnet/performance/blob/d5e733fe3416e9402556ffe5c4a4e54a99b06d38/src/benchmarks/micro/libraries/Microsoft.Extensions.Configuration/GetChildKeysTests.cs#L29

so the implication is that we never try parsing any integers with leading or trailing whitespace allowed and actual leading or trailing whitespace in our training scenarios, and when we rely on this static PGO data we make some decisions that adversely impact this test.

AndyAyersMS · 2023-05-10T22:57:56Z

I realized the above is actually looking at perf on Windows and this report was about a regression in Linux. So I pulled Linux (and arm64) data into my data query and surprisingly Linux perf (orange line below) on this test has returned to normal

Not sure if this is telling us that Linux PGO data is different, or that the jit responds differently on Linux, or that all the PGO data has been updated but the Windows runs haven't seen it yet for some reason.

At any rate I'm going to look into some of the other regressions here and come back to this one.

AndyAyersMS · 2023-05-10T23:20:35Z

System.Tests.Perf_UInt64.TryParse(value: "12345")

[link]

This one seems linux-x64 specific.

Actually, if you check out the ampere results there seems to be a regression there too. I think those two may share the same MIBC? So am going to take a look at how that data differs from the windows data.

AndyAyersMS · 2023-05-11T03:05:45Z

For the cases above, on windows there is no static PGO for System.Number:TryParseBinaryIntegerStyle[ulong](System.ReadOnlySpan`1[ushort],int,System.Globalization.NumberFormatInfo,byref):int (note this is a different method than in the previous benchmark, that one had [int] as its first generic param).

On Linux there is static PGO, but it is very "thin" -- only 4 calls to the method. This leads to cascading diffs but as far as I can tell the main impact is to block layout and some flow opts; notably inlining is the same.

This lack of coverage for uint parsing likely explains the bulk of the regressions above.

One mitigation we might contemplate is to toss out static PGO data if there's not a lot of coverage in a method. Another might be to try and use profile blend to avoid over-fitting to what is likely a very small and not representative sample. Blend is perhaps not quite ready for general use, but I will try enabling it here and see what it does.

AndyAyersMS · 2023-06-09T01:28:05Z

Updated and collated results

Recent Score	Orig Score	x64-lin-intel	Benchmark
1.68	2.76	1.68 2.76	Benchmark.GetChildKeysTests.AddChainedConfigurationEmpty
1.44	4.56	1.44 4.56	System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt32(value: 4)
1.28	1.31	1.28 1.31	System.Tests.Perf_Int32.TryParseSpan(value: "2147483647")
1.26	1.27	1.26 1.27	System.Globalization.Tests.StringEquality.Compare_Same(Count: 1024, Options: (en-US, OrdinalIgnoreCase))
1.24	1.25	1.24 1.25	System.Tests.Perf_Int32.ParseSpan(value: "2147483647")
1.22	1.26	1.22 1.26	System.Tests.Perf_Int32.TryParse(value: "2147483647")
1.22	1.24	1.22 1.24	System.Tests.Perf_Int32.Parse(value: "2147483647")
1.21	1.27	1.21 1.27	System.Tests.Perf_Int32.Parse(value: "-2147483648")
1.18	1.27	1.18 1.27	System.Tests.Perf_Int32.TryParseSpan(value: "-2147483648")
1.15	1.18	1.15 1.18	System.Text.Perf_Utf8Encoding.GetString(Input: Chinese)
1.14	1.15	1.14 1.15	System.Tests.Perf_Int32.TryParse(value: "-2147483648")
1.12	1.98	1.12 1.98	System.Globalization.Tests.StringEquality.Compare_Same_Upper(Count: 1024, Options: (en-US, OrdinalIgnoreCase))
1.10	1.23	1.10 1.23	System.Tests.Perf_UInt16.Parse(value: "12345")
1.09	1.20	1.09 1.20	System.Collections.ContainsTrue(String).HashSet(Size: 512)
1.09	1.25	1.09 1.25	System.Tests.Perf_UInt16.Parse(value: "65535")
1.08	1.19	1.08 1.19	System.Tests.Perf_Int32.TryParse(value: "12345")
1.06	1.15	1.06 1.15	PerfLabTests.CastingPerf2.CastingPerf.IFooObjIsIFooInterAlia
1.04	1.10	1.04 1.10	System.Collections.ContainsKeyFalse(String, String).ImmutableDictionary(Size: 512)
1.03	1.21	1.03 1.21	System.Tests.Perf_Int32.TryParseSpan(value: "12345")
1.03	1.16	1.03 1.16	System.Tests.Perf_Int32.Parse(value: "12345")
1.03	1.09	1.03 1.09	System.Collections.TryGetValueFalse(String, String).ImmutableDictionary(Size: 512)
1.01	1.14	1.01 1.14	System.Tests.Perf_Int32.TryParseSpan(value: "4")
1.01	1.08	1.01 1.08	System.Collections.ContainsFalse(String).ImmutableHashSet(Size: 512)
0.99	1.08	0.99 1.08	System.Tests.Perf_UInt16.TryParse(value: "65535")
0.99	1.28	0.99 1.28	System.Tests.Perf_UInt64.Parse(value: "12345")
0.99	1.46	0.99 1.46	System.Tests.Perf_UInt64.ParseSpan(value: "18446744073709551615")
0.98	1.35	0.98 1.35	System.Tests.Perf_UInt64.TryParse(value: "12345")
0.98	1.43	0.98 1.43	System.Tests.Perf_UInt64.TryParse(value: "18446744073709551615")
0.98	1.86	0.98 1.86	System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt32(value: 2147483647)
0.98	1.09	0.98 1.09	System.Collections.AddGivenSize(String).HashSet(Size: 512)
0.97	1.52	0.97 1.52	System.Tests.Perf_UInt64.Parse(value: "18446744073709551615")
0.97	1.06	0.97 1.06	System.Collections.AddGivenSize(String).Stack(Size: 512)
0.96	1.12	0.96 1.12	System.Tests.Perf_Int64.Parse(value: "12345")
0.96	1.20	0.96 1.20	System.Tests.Perf_Int64.ParseSpan(value: "12345")
0.94	1.13	0.94 1.13	System.Tests.Perf_Int64.TryParseSpan(value: "12345")
0.93	1.17	0.93 1.17	System.Tests.Perf_Int32.Parse(value: "4")
0.92	1.13	0.92 1.13	System.Tests.Perf_Int32.ParseSpan(value: "4")
0.91	1.32	0.91 1.32	System.Tests.Perf_Int64.Parse(value: "-9223372036854775808")
0.91	1.14	0.91 1.14	System.Tests.Perf_Int32.TryFormat(value: -2147483648)
0.90	1.18	0.90 1.18	System.Tests.Perf_Int64.TryParse(value: "-9223372036854775808")
0.90	1.29	0.90 1.29	System.Tests.Perf_Int64.TryParseSpan(value: "-9223372036854775808")
0.90	1.21	0.90 1.21	System.Tests.Perf_Int64.ParseSpan(value: "-9223372036854775808")
0.87	1.14	0.87 1.14	System.Tests.Perf_UInt16.TryParse(value: "0")
0.87	1.09	0.87 1.09	System.Linq.Tests.Perf_Enumerable.ToArray(input: ICollection)
0.86	1.22	0.86 1.22	System.Tests.Perf_Int64.Parse(value: "9223372036854775807")
0.86	1.99	0.86 1.99	System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt32(value: 12345)
0.85	1.10	0.85 1.10	System.Tests.Perf_UInt16.Parse(value: "0")
0.85	1.17	0.85 1.17	System.Tests.Perf_Int64.ParseSpan(value: "9223372036854775807")
0.83	1.16	0.83 1.16	System.Tests.Perf_Int64.TryParse(value: "9223372036854775807")
0.81	1.12	0.81 1.12	Benchmark.GetChildKeysTests.AddChainedConfigurationNoDelimiter
0.80	1.14	0.80 1.14	System.Tests.Perf_Int64.TryParseSpan(value: "9223372036854775807")
0.79	1.31	0.79 1.31	System.Linq.Tests.Perf_Enumerable.ElementAt(input: IList)
0.77	1.09	0.77 1.09	System.Collections.CtorFromCollection(String).HashSet(Size: 512)
0.63	1.45	0.63 1.45	System.Linq.Tests.Perf_Enumerable.Count(input: ICollection)
0.45	1.09	0.45 1.09	System.Linq.Tests.Perf_Enumerable.SelectToList(input: Array)
0.21	1.13	0.21 1.13	System.Collections.IterateForEach(Int32).ImmutableQueue(Size: 512)

AndyAyersMS · 2023-07-25T17:30:35Z

Looks like many of these are now dramatically improved, eg

Let me rerun the classifier and see what's left to investigate.

AndyAyersMS · 2023-07-25T17:42:54Z

Updated classification

Notes	Recent Score	Orig Score	main_x64_ubuntu%2018.04/	Benchmark
0.7ns slower overall with recent data	2.07	4.56	2.07 4.56	System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt32(value: 4)
since improved	1.32	2.76	1.32 2.76	Benchmark.GetChildKeysTests.AddChainedConfigurationEmpty
stabilized & improved w/dynamic PGO	1.31	1.99	1.31 1.99	System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt32(value: 12345)
complicated, but original regression is gone	1.17	1.18	1.17 1.18	System.Text.Perf_Utf8Encoding.GetString(Input: Chinese)
stablized w/dynamic pgo; only slightly regressed	1.16	1.86	1.16 1.86	System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt32(value: 2147483647)
	1.14	1.24	1.14 1.24	System.Tests.Perf_Int32.Parse(value: "2147483647")
	1.13	1.26	1.13 1.26	System.Tests.Perf_Int32.TryParse(value: "2147483647")
	1.13	1.98	1.13 1.98	System.Globalization.Tests.StringEquality.Compare_Same_Upper(Count: 1024, Options: (en-US, OrdinalIgnoreCase))
	1.09	1.25	1.09 1.25	System.Tests.Perf_Int32.ParseSpan(value: "2147483647")
	1.08	1.15	1.08 1.15	System.Tests.Perf_Int32.TryParse(value: "-2147483648")
	1.08	1.27	1.08 1.27	System.Tests.Perf_Int32.Parse(value: "-2147483648")
	1.07	1.31	1.07 1.31	System.Tests.Perf_Int32.TryParseSpan(value: "2147483647")
	1.07	1.27	1.07 1.27	System.Tests.Perf_Int32.TryParseSpan(value: "-2147483648")
	1.05	1.09	1.05 1.09	System.Collections.AddGivenSize(String).HashSet(Size: 512)
	1.03	1.08	1.03 1.08	System.Collections.ContainsFalse(String).ImmutableHashSet(Size: 512)
	1.00	1.20	1.00 1.20	System.Collections.ContainsTrue(String).HashSet(Size: 512)
	1.00	1.15	1.00 1.15	PerfLabTests.CastingPerf2.CastingPerf.IFooObjIsIFooInterAlia
	0.99	1.06	0.99 1.06	System.Collections.AddGivenSize(String).Stack(Size: 512)
	0.99	1.10	0.99 1.10	System.Collections.ContainsKeyFalse(String, String).ImmutableDictionary(Size: 512)
	0.99	1.27	0.99 1.27	System.Globalization.Tests.StringEquality.Compare_Same(Count: 1024, Options: (en-US, OrdinalIgnoreCase))
	0.96	1.09	0.96 1.09	System.Collections.TryGetValueFalse(String, String).ImmutableDictionary(Size: 512)
	0.93	1.28	0.93 1.28	System.Tests.Perf_UInt64.Parse(value: "12345")
	0.93	1.25	0.93 1.25	System.Tests.Perf_UInt16.Parse(value: "65535")
	0.92	1.23	0.92 1.23	System.Tests.Perf_UInt16.Parse(value: "12345")
	0.92	1.32	0.92 1.32	System.Tests.Perf_Int64.Parse(value: "-9223372036854775808")
	0.90	1.21	0.90 1.21	System.Tests.Perf_Int64.ParseSpan(value: "-9223372036854775808")
	0.90	1.21	0.90 1.21	System.Tests.Perf_Int32.TryParseSpan(value: "12345")
	0.87	1.09	0.87 1.09	System.Linq.Tests.Perf_Enumerable.ToArray(input: ICollection)
	0.87	1.09	0.87 1.09	System.Collections.CtorFromCollection(String).HashSet(Size: 512)
	0.86	1.46	0.86 1.46	System.Tests.Perf_UInt64.ParseSpan(value: "18446744073709551615")
	0.85	1.52	0.85 1.52	System.Tests.Perf_UInt64.Parse(value: "18446744073709551615")
	0.85	1.43	0.85 1.43	System.Tests.Perf_UInt64.TryParse(value: "18446744073709551615")
	0.84	1.20	0.84 1.20	System.Tests.Perf_Int64.ParseSpan(value: "12345")
	0.84	1.19	0.84 1.19	System.Tests.Perf_Int32.TryParse(value: "12345")
	0.82	1.31	0.82 1.31	System.Linq.Tests.Perf_Enumerable.ElementAt(input: IList)
	0.80	1.12	0.80 1.12	System.Tests.Perf_Int64.Parse(value: "12345")
	0.80	1.29	0.80 1.29	System.Tests.Perf_Int64.TryParseSpan(value: "-9223372036854775808")
	0.80	1.13	0.80 1.13	System.Tests.Perf_Int64.TryParseSpan(value: "12345")
	0.80	1.08	0.80 1.08	System.Tests.Perf_UInt16.TryParse(value: "65535")
	0.80	1.16	0.80 1.16	System.Tests.Perf_Int32.Parse(value: "12345")
	0.79	1.35	0.79 1.35	System.Tests.Perf_UInt64.TryParse(value: "12345")
	0.79	1.14	0.79 1.14	System.Tests.Perf_Int32.TryFormat(value: -2147483648)
	0.78	1.18	0.78 1.18	System.Tests.Perf_Int64.TryParse(value: "-9223372036854775808")
	0.75	1.17	0.75 1.17	System.Tests.Perf_Int64.ParseSpan(value: "9223372036854775807")
	0.75	1.22	0.75 1.22	System.Tests.Perf_Int64.Parse(value: "9223372036854775807")
	0.73	1.14	0.73 1.14	System.Tests.Perf_Int64.TryParseSpan(value: "9223372036854775807")
	0.72	1.17	0.72 1.17	System.Tests.Perf_Int32.Parse(value: "4")
	0.71	1.16	0.71 1.16	System.Tests.Perf_Int64.TryParse(value: "9223372036854775807")
	0.70	1.13	0.70 1.13	System.Tests.Perf_Int32.ParseSpan(value: "4")
	0.70	1.12	0.70 1.12	Benchmark.GetChildKeysTests.AddChainedConfigurationNoDelimiter
	0.70	1.14	0.70 1.14	System.Tests.Perf_Int32.TryParseSpan(value: "4")
	0.61	1.45	0.61 1.45	System.Linq.Tests.Perf_Enumerable.Count(input: ICollection)
	0.61	1.14	0.61 1.14	System.Tests.Perf_UInt16.TryParse(value: "0")
	0.57	1.10	0.57 1.10	System.Tests.Perf_UInt16.Parse(value: "0")
	0.54	1.09	0.54 1.09	System.Linq.Tests.Perf_Enumerable.SelectToList(input: Array)
	0.21	1.13	0.21 1.13	System.Collections.IterateForEach(Int32).ImmutableQueue(Size: 512)

AndyAyersMS · 2023-08-01T18:44:32Z

Benchmark.GetChildKeysTests.AddChainedConfigurationEmpty

This one is interesting -- while the report here is for ubuntu x64 on 02-May, the data shows more substantial hits to windows platforms(win-x64-intel highlighted below). However, things have since improved across the board.

AndyAyersMS · 2023-08-04T17:18:04Z

For the most part the issues here have been resolved (some by enabling Dynamic PGO).

There is still too much sensitivity to static PGO in some places, typically these are tests where a cold inline can improve hot code. Not actionable for .NET 8, but something to keep in mind for the future.

performanceautofiler bot assigned AndyAyersMS May 9, 2023

performanceautofiler bot added arch-x64 os-linux Linux OS (any supported distro) PGO runtime-coreclr specific to the CoreCLR runtime untriaged New issue has not been triaged by the area owner labels May 9, 2023

cincuranet removed the untriaged New issue has not been triaged by the area owner label May 9, 2023

dotnet-issue-labeler bot added the needs-area-label An area label is needed to ensure this gets routed to the appropriate area owners label May 9, 2023

cincuranet transferred this issue from dotnet/perf-autofiling-issues May 9, 2023

ghost added the untriaged New issue has not been triaged by the area owner label May 9, 2023

This was referenced May 9, 2023

[Perf] Linux/x64: 11 Regressions on 5/3/2023 2:49:09 AM dotnet/perf-autofiling-issues#17577

Closed

[Perf] Linux/x64: 19 Regressions on 5/5/2023 3:18:41 PM dotnet/perf-autofiling-issues#17581

Closed

AndyAyersMS added area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI and removed untriaged New issue has not been triaged by the area owner labels May 9, 2023

AndyAyersMS removed the needs-area-label An area label is needed to ensure this gets routed to the appropriate area owners label May 9, 2023

AndyAyersMS added this to the 8.0.0 milestone May 9, 2023

AndyAyersMS added the Priority:2 Work that is important, but not critical for the release label Jun 9, 2023

AndyAyersMS closed this as completed Aug 4, 2023

ghost locked as resolved and limited conversation to collaborators Sep 3, 2023

[Perf] Linux/x64: 26 Regressions on 5/2/2023 9:04:55 PM #85989

[Perf] Linux/x64: 26 Regressions on 5/2/2023 9:04:55 PM #85989

Comments

performanceautofiler bot commented May 9, 2023

Run Information

Regressions in System.Tests.Perf_UInt64

Repro

Payloads

Payloads

Histogram

System.Tests.Perf_UInt64.TryParse(value: "18446744073709551615")

Description of detection logic

JIT Disasms

System.Tests.Perf_UInt64.ParseSpan(value: "18446744073709551615")

Description of detection logic

JIT Disasms

System.Tests.Perf_UInt64.Parse(value: "12345")

Description of detection logic

JIT Disasms

System.Tests.Perf_UInt64.TryParse(value: "12345")

Description of detection logic

JIT Disasms

System.Tests.Perf_UInt64.Parse(value: "18446744073709551615")

Description of detection logic

JIT Disasms

Docs

Run Information

Regressions in System.Globalization.Tests.StringEquality

Repro

Payloads

Payloads

Histogram

System.Globalization.Tests.StringEquality.Compare_Same(Count: 1024, Options: (en-US, OrdinalIgnoreCase))

Description of detection logic

JIT Disasms

System.Globalization.Tests.StringEquality.Compare_Same_Upper(Count: 1024, Options: (en-US, OrdinalIgnoreCase))

Description of detection logic

JIT Disasms

Docs

Run Information

Regressions in System.Collections.AddGivenSize<String>

Repro

Payloads

Payloads

Histogram

System.Collections.AddGivenSize<String>.Stack(Size: 512)

Description of detection logic

JIT Disasms

System.Collections.AddGivenSize<String>.HashSet(Size: 512)

Description of detection logic

JIT Disasms

Docs

Run Information

Regressions in System.Tests.Perf_UInt16

Repro

Payloads

Payloads

Histogram

System.Tests.Perf_UInt16.Parse(value: "65535")

Description of detection logic

JIT Disasms

System.Tests.Perf_UInt16.Parse(value: "12345")

Description of detection logic

JIT Disasms

System.Tests.Perf_UInt16.Parse(value: "0")

Description of detection logic

JIT Disasms

System.Tests.Perf_UInt16.TryParse(value: "65535")

Description of detection logic

JIT Disasms

System.Tests.Perf_UInt16.TryParse(value: "0")

Description of detection logic

JIT Disasms

Docs

Run Information

Regressions in System.Buffers.Text.Tests.Utf8FormatterTests

Repro

Payloads

Payloads

Histogram