[Perf] Linux/x64: 3 Regressions on 6/19/2023 11:51:49 PM

[performanceautofiler]

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	4dc137c1d1369926b82a9f412aa66568e3befb13
Compare	0c77cbef02e1713d2b7cd01969b92e552acbdb6b
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Tests.Perf_Encoding

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
GetChars - Duration of single invocation	279.54 ns	310.57 ns	1.11	0.12	False
GetString - Duration of single invocation	242.26 ns	275.79 ns	1.14	0.14	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Tests.Perf_Encoding*'

Payloads

Baseline
Compare

Histogram

System.Text.Tests.Perf_Encoding.GetChars(size: 512, encName: "utf-8")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 310.56757333041094 > 296.0617993513515.
IsChangePoint: Marked as a change because one of 6/19/2023 6:37:24 PM, 6/26/2023 9:59:38 AM falls between 6/17/2023 6:37:57 PM and 6/26/2023 9:59:38 AM.
IsRegressionStdDev: Marked as regression because -10.596992419576516 (T) = (0 -301.1261235872364) / Math.Sqrt((28.010236336036286 / (13)) + (39.23498831414148 / (22))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (13) + (22) - 2, .025) and -0.07507830639958464 = (280.09692112168244 - 301.1261235872364) / 280.09692112168244 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Tests.Perf_Encoding.GetString(size: 512, encName: "ascii")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 275.7923218711697 > 257.4541133321917.
IsChangePoint: Marked as a change because one of 6/19/2023 6:37:24 PM, 6/26/2023 9:59:38 AM falls between 6/17/2023 6:37:57 PM and 6/26/2023 9:59:38 AM.
IsRegressionStdDev: Marked as regression because -17.44252451118468 (T) = (0 -274.20346146514845) / Math.Sqrt((12.156998915309643 / (13)) + (36.549109913036 / (21))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (13) + (21) - 2, .025) and -0.11613487957244636 = (245.67233448540426 - 274.20346146514845) / 245.67233448540426 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	4dc137c1d1369926b82a9f412aa66568e3befb13
Compare	0c77cbef02e1713d2b7cd01969b92e552acbdb6b
Diff	Diff
Configs	AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Perf_Ascii

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ToUtf16 - Duration of single invocation	23.28 ns	32.61 ns	1.40	0.37	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Perf_Ascii*'

Payloads

Baseline
Compare

Histogram

System.Text.Perf_Ascii.ToUtf16(Size: 128)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 32.6068615116673 > 24.47019216187988.
IsChangePoint: Marked as a change because one of 6/19/2023 6:37:24 PM, 6/26/2023 9:59:38 AM falls between 6/17/2023 6:37:57 PM and 6/26/2023 9:59:38 AM.
IsRegressionStdDev: Marked as regression because -17.219163995062726 (T) = (0 -31.2437002682462) / Math.Sqrt((0.4507876090402956 / (13)) + (3.760689514120004 / (22))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (13) + (22) - 2, .025) and -0.33316807824051925 = (23.435679850271264 - 31.2437002682462) / 23.435679850271264 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[radekdoulik]

@xtqqczze, looks like this is caused by dotnet/runtime#87737