Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Regressions from Roslyn update #80744

Open
performanceautofiler bot opened this issue Jan 17, 2023 · 21 comments
Open

Regressions from Roslyn update #80744

performanceautofiler bot opened this issue Jan 17, 2023 · 21 comments
Assignees
@EgorBo
Labels
arch-x64 area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI os-linux Linux OS (any supported distro) tenet-performance Performance related issue tenet-performance-benchmarks Issue from performance benchmark
Milestone
Future

Comments

@performanceautofiler
Copy link

performanceautofiler bot commented Jan 17, 2023 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline a8549002b2df6bf64c7c882d8f117630beba762d
Compare 8a2b72e3140b14a2ec915345b2747eecdecff943
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetHashCodeBenchmark - Duration of single invocation 79.04 ns 105.44 ns 1.33 0.06 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<Byte>.GetHashCodeBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 105.4351294570456 > 83.15154770429102.
IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -38.87873117596192 (T) = (0 -105.35273354069511) / Math.Sqrt((3.992587969600447 / (23)) + (7.861876182668589 / (31))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (31) - 2, .025) and -0.31787460021963265 = (79.94139466921767 - 105.35273354069511) / 79.94139466921767 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline a8549002b2df6bf64c7c882d8f117630beba762d
Compare 8a2b72e3140b14a2ec915345b2747eecdecff943
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<UInt16>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetHashCodeBenchmark - Duration of single invocation 39.22 ns 51.70 ns 1.32 0.05 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<UInt16>.GetHashCodeBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 51.69693024643556 > 41.1064538719314.
IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -36.663764455094125 (T) = (0 -51.68067070574212) / Math.Sqrt((1.65991615061608 / (22)) + (1.9274338623056948 / (31))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (31) - 2, .025) and -0.35719061117778034 = (38.07915430603609 - 51.68067070574212) / 38.07915430603609 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline a8549002b2df6bf64c7c882d8f117630beba762d
Compare 8a2b72e3140b14a2ec915345b2747eecdecff943
Diff Diff

Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetHashCodeBenchmark - Duration of single invocation 9.48 ns 12.98 ns 1.37 0.07 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.GetHashCodeBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.97857567164073 > 9.912366475799981.
IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -93.87222615864435 (T) = (0 -13.031390688994902) / Math.Sqrt((0.011639596703320161 / (23)) + (0.027995569909519624 / (31))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (31) - 2, .025) and -0.3706358320006545 = (9.50755144783686 - 13.031390688994902) / 9.50755144783686 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline a8549002b2df6bf64c7c882d8f117630beba762d
Compare 8a2b72e3140b14a2ec915345b2747eecdecff943
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<Double>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetHashCodeBenchmark - Duration of single invocation 13.38 ns 14.95 ns 1.12 0.03 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<Double>.GetHashCodeBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.948886232048702 > 14.071781964405073.
IsChangePoint: Marked as a change because one of 11/2/2022 4:25:40 PM, 12/14/2022 5:20:21 PM, 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -23.26476081715157 (T) = (0 -14.987631064522386) / Math.Sqrt((0.05384720602955832 / (23)) + (0.045076784078112886 / (31))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (31) - 2, .025) and -0.10574021657368117 = (13.554387223939488 - 14.987631064522386) / 13.554387223939488 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline a8549002b2df6bf64c7c882d8f117630beba762d
Compare 8a2b72e3140b14a2ec915345b2747eecdecff943
Diff Diff

Regressions in System.Tests.Perf_HashCode

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Combine_7 - Duration of single invocation 96.03 μs 105.47 μs 1.10 0.00 False
Combine_2 - Duration of single invocation 56.55 μs 63.40 μs 1.12 0.00 False
Combine_1 - Duration of single invocation 46.17 μs 52.16 μs 1.13 0.00 False
Combine_5 - Duration of single invocation 74.27 μs 82.59 μs 1.11 0.00 False
Combine_4 - Duration of single invocation 63.99 μs 71.89 μs 1.12 0.00 False
Combine_6 - Duration of single invocation 82.93 μs 93.13 μs 1.12 0.00 False
Combine_8 - Duration of single invocation 82.16 μs 90.71 μs 1.10 0.00 False
Combine_3 - Duration of single invocation 69.56 μs 74.71 μs 1.07 0.00 False

graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_HashCode*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_HashCode.Combine_7

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 105.4663142677373 > 100.81468016679449.
IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -161.18619752295376 (T) = (0 -105545.67733509371) / Math.Sqrt((6585.849273667598 / (23)) + (98379.09306130768 / (31))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (31) - 2, .025) and -0.09869467137568116 = (96064.61202086239 - 105545.67733509371) / 96064.61202086239 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_HashCode.Combine_2

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 63.40146640519568 > 59.350448427346585.
IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -521.8008790934781 (T) = (0 -63448.79334557399) / Math.Sqrt((1550.9166089246503 / (23)) + (3334.649210823573 / (31))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (31) - 2, .025) and -0.12207380790148624 = (56546.006954958306 - 63448.79334557399) / 56546.006954958306 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_HashCode.Combine_1

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 52.161184375000005 > 48.49616808904867.
IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -474.7956288486027 (T) = (0 -52200.563625643365) / Math.Sqrt((2370.9048049895378 / (22)) + (1632.5099591971323 / (31))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (31) - 2, .025) and -0.13020631808142258 = (46186.756161703495 - 52200.563625643365) / 46186.756161703495 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_HashCode.Combine_5

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 82.59137885338346 > 77.98060776806872.
IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -73.22615233862166 (T) = (0 -82505.62178616505) / Math.Sqrt((104660.75114867605 / (23)) + (244552.67765220546 / (31))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (31) - 2, .025) and -0.10986241775819691 = (74338.6031151659 - 82505.62178616505) / 74338.6031151659 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_HashCode.Combine_4

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 71.89388650884665 > 67.18613039001559.
IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -281.6459962191387 (T) = (0 -71670.10661699313) / Math.Sqrt((709.6575365608301 / (23)) + (22155.12236267368 / (31))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (31) - 2, .025) and -0.12019719537860528 = (63979.90185359284 - 71670.10661699313) / 63979.90185359284 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_HashCode.Combine_6

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 93.12999140625 > 87.02870333416007.
IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -115.48185935902607 (T) = (0 -93069.88297768412) / Math.Sqrt((3333.509569601689 / (23)) + (235566.862937844 / (31))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (31) - 2, .025) and -0.12257404511481351 = (82907.56710677843 - 93069.88297768412) / 82907.56710677843 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_HashCode.Combine_8

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 90.71372703516377 > 86.27665725675548.
IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -535.3374364970878 (T) = (0 -90757.96576528928) / Math.Sqrt((3413.288835681361 / (23)) + (3346.5812596985775 / (31))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (31) - 2, .025) and -0.10429181728794486 = (82186.57817114303 - 90757.96576528928) / 82186.57817114303 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_HashCode.Combine_3

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 74.7060919059011 > 73.07507285416666.
IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -140.11060272122677 (T) = (0 -74609.6146257151) / Math.Sqrt((1174.072760930738 / (23)) + (38213.510624717346 / (31))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (31) - 2, .025) and -0.07213825342420208 = (69589.54629911434 - 74609.6146257151) / 69589.54629911434 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline a8549002b2df6bf64c7c882d8f117630beba762d
Compare 8a2b72e3140b14a2ec915345b2747eecdecff943
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<Int64>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetHashCodeBenchmark - Duration of single invocation 10.81 ns 13.50 ns 1.25 0.03 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<Int64>.GetHashCodeBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.501432980336178 > 11.349420695138868.
IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -53.84020803276331 (T) = (0 -13.53650726544601) / Math.Sqrt((0.008029975980635195 / (24)) + (0.06952606554272087 / (31))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (31) - 2, .025) and -0.2530132020379094 = (10.803164119444345 - 13.53650726544601) / 10.803164119444345 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline a8549002b2df6bf64c7c882d8f117630beba762d
Compare 8a2b72e3140b14a2ec915345b2747eecdecff943
Diff Diff

Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetHashCodeBenchmark - Duration of single invocation 38.33 ns 54.55 ns 1.42 0.02 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;*'

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.GetHashCodeBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 54.54619767782453 > 40.21929684002488.
IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -82.21235737703599 (T) = (0 -54.1115179051457) / Math.Sqrt((0.06252101293171532 / (24)) + (1.0797309196087996 / (31))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (31) - 2, .025) and -0.4163482868152744 = (38.20495171199592 - 54.1115179051457) / 38.20495171199592 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline a8549002b2df6bf64c7c882d8f117630beba762d
Compare 8a2b72e3140b14a2ec915345b2747eecdecff943
Diff Diff

Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetHashCodeBenchmark - Duration of single invocation 19.54 ns 24.49 ns 1.25 0.03 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt16&gt;*'

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.GetHashCodeBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 24.489060082989262 > 20.74241711124787.
IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -24.00227051035366 (T) = (0 -24.845609585691953) / Math.Sqrt((0.07136336798950783 / (23)) + (1.2121066048338698 / (31))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (31) - 2, .025) and -0.2475992129108641 = (19.91473650237632 - 24.845609585691953) / 19.91473650237632 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline a8549002b2df6bf64c7c882d8f117630beba762d
Compare 8a2b72e3140b14a2ec915345b2747eecdecff943
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<UInt32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetHashCodeBenchmark - Duration of single invocation 20.29 ns 23.33 ns 1.15 0.04 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<UInt32>.GetHashCodeBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 23.331260832845107 > 21.056982512327213.
IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -25.472944182933528 (T) = (0 -23.645774694088463) / Math.Sqrt((0.1302430376724827 / (23)) + (0.4615801379454354 / (31))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (31) - 2, .025) and -0.1826468503281096 = (19.993943828225863 - 23.645774694088463) / 19.993943828225863 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline a8549002b2df6bf64c7c882d8f117630beba762d
Compare 8a2b72e3140b14a2ec915345b2747eecdecff943
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<UInt64>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetHashCodeBenchmark - Duration of single invocation 10.81 ns 13.54 ns 1.25 0.05 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<UInt64>.GetHashCodeBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.536802932034956 > 11.35145753448431.
IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -8.574082760730478 (T) = (0 -13.506688811973898) / Math.Sqrt((1.7762903502862928 / (23)) + (0.10097248716469229 / (31))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (31) - 2, .025) and -0.21965381121373115 = (11.074198832316851 - 13.506688811973898) / 11.074198832316851 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline a8549002b2df6bf64c7c882d8f117630beba762d
Compare 8a2b72e3140b14a2ec915345b2747eecdecff943
Diff Diff

Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetHashCodeBenchmark - Duration of single invocation 6.14 ns 8.24 ns 1.34 0.20 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;*'

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.GetHashCodeBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.243436857714752 > 6.447982991414564.
IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -8.389320644851091 (T) = (0 -8.063495790669068) / Math.Sqrt((0.03524392530140337 / (24)) + (1.7697574079138925 / (31))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (31) - 2, .025) and -0.33647761884757654 = (6.033393808436607 - 8.063495790669068) / 6.033393808436607 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline a8549002b2df6bf64c7c882d8f117630beba762d
Compare 8a2b72e3140b14a2ec915345b2747eecdecff943
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<Int16>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetHashCodeBenchmark - Duration of single invocation 37.85 ns 51.66 ns 1.37 0.07 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<Int16>.GetHashCodeBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 51.66120195005749 > 39.72074575315014.
IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -39.559854435578245 (T) = (0 -51.881876605118066) / Math.Sqrt((0.8104291630112155 / (23)) + (2.8603275361064218 / (31))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (31) - 2, .025) and -0.37413871248681874 = (37.75592386246504 - 51.881876605118066) / 37.75592386246504 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
@performanceautofiler
Copy link
Author

performanceautofiler bot commented Jan 17, 2023 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline a8549002b2df6bf64c7c882d8f117630beba762d
Compare 8a2b72e3140b14a2ec915345b2747eecdecff943
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetHashCodeBenchmark - Duration of single invocation 20.03 ns 23.32 ns 1.16 0.02 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<Int32>.GetHashCodeBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 23.320096686065867 > 21.10859220137375.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -27.41942871555706 (T) = (0 -23.556103451989134) / Math.Sqrt((0.045964928710745 / (23)) + (0.4321694720681641 / (31))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (31) - 2, .025) and -0.17227431687018183 = (20.094361117524805 - 23.556103451989134) / 20.094361117524805 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline a8549002b2df6bf64c7c882d8f117630beba762d
Compare 8a2b72e3140b14a2ec915345b2747eecdecff943
Diff Diff

Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetHashCodeBenchmark - Duration of single invocation 38.11 ns 54.62 ns 1.43 0.01 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;SByte&gt;*'

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>.GetHashCodeBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 54.619012798368786 > 40.02805183024191.
IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -94.84377908284456 (T) = (0 -54.19152874424385) / Math.Sqrt((0.03709778719896129 / (24)) + (0.848027256877123 / (31))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (31) - 2, .025) and -0.42355745765139086 = (38.06767928682692 - 54.19152874424385) / 38.06767928682692 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline a8549002b2df6bf64c7c882d8f117630beba762d
Compare 8a2b72e3140b14a2ec915345b2747eecdecff943
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<SByte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetHashCodeBenchmark - Duration of single invocation 79.26 ns 105.48 ns 1.33 0.06 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<SByte>.GetHashCodeBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 105.48442802319262 > 83.21140817384584.
IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -38.47455636359471 (T) = (0 -105.38424496933936) / Math.Sqrt((4.656914615225362 / (23)) + (7.935677422340858 / (31))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (31) - 2, .025) and -0.32837623290392604 = (79.333130448263 - 105.38424496933936) / 79.333130448263 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline a8549002b2df6bf64c7c882d8f117630beba762d
Compare 8a2b72e3140b14a2ec915345b2747eecdecff943
Diff Diff

Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetHashCodeBenchmark - Duration of single invocation 5.77 ns 7.44 ns 1.29 0.16 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;*'

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.GetHashCodeBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.437152732708328 > 6.263572225291969.
IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -18.5845985828341 (T) = (0 -7.6594885997405475) / Math.Sqrt((0.10212762220395216 / (24)) + (0.10276806239422005 / (31))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (31) - 2, .025) and -0.26760735431885063 = (6.042477249476339 - 7.6594885997405475) / 6.042477249476339 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline a8549002b2df6bf64c7c882d8f117630beba762d
Compare 8a2b72e3140b14a2ec915345b2747eecdecff943
Diff Diff

Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetHashCodeBenchmark - Duration of single invocation 19.68 ns 24.73 ns 1.26 0.02 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int16&gt;*'

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.GetHashCodeBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 24.72658991657417 > 20.91314539556719.
IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -19.432827146762506 (T) = (0 -25.219160386806106) / Math.Sqrt((0.04376875150126903 / (23)) + (2.1488050817985904 / (31))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (31) - 2, .025) and -0.2588724915109568 = (20.033133265575536 - 25.219160386806106) / 20.033133265575536 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline a8549002b2df6bf64c7c882d8f117630beba762d
Compare 8a2b72e3140b14a2ec915345b2747eecdecff943
Diff Diff

Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetHashCodeBenchmark - Duration of single invocation 9.41 ns 13.56 ns 1.44 0.12 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>.GetHashCodeBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.561207618701866 > 9.87549907306126.
IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -11.216464759854135 (T) = (0 -13.204830589160034) / Math.Sqrt((2.0464586540799097 / (23)) + (0.11329716342766691 / (31))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (31) - 2, .025) and -0.3486623636520238 = (9.7910573802941 - 13.204830589160034) / 9.7910573802941 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler performanceautofiler bot added PGO untriaged New issue has not been triaged by the area owner labels Jan 17, 2023
@EgorBo EgorBo changed the title [Perf] Linux/x64: 25 Regressions on 1/10/2023 12:53:48 PM Regressions from Roslyn update Jan 17, 2023
@EgorBo EgorBo removed PGO untriaged New issue has not been triaged by the area owner labels Jan 17, 2023
@dotnet-issue-labeler
Copy link

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

@EgorBo EgorBo transferred this issue from dotnet/perf-autofiling-issues Jan 17, 2023
@ghost ghost added the untriaged New issue has not been triaged by the area owner label Jan 17, 2023
@EgorBo EgorBo added os-linux Linux OS (any supported distro) tenet-performance Performance related issue tenet-performance-benchmarks Issue from performance benchmark arch-x64 labels Jan 17, 2023
@EgorBo
Copy link
Member

EgorBo commented Jan 17, 2023

Windows-x64 dotnet/perf-autofiling-issues#11738

@EgorBo EgorBo mentioned this issue Jan 17, 2023
Closed
@EgorBo
Copy link
Member

EgorBo commented Jan 17, 2023

IL diff for HashCode.Combine:

image

@EgorBo EgorBo self-assigned this Jan 17, 2023
@EgorBo EgorBo added area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI and removed untriaged New issue has not been triaged by the area owner labels Jan 17, 2023
@ghost
Copy link

ghost commented Jan 17, 2023

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch, @kunalspathak
See info in area-owners.md if you want to be subscribed.

Issue Details

Run Information

Architecture x64
OS ubuntu 18.04
Baseline a8549002b2df6bf64c7c882d8f117630beba762d
Compare 8a2b72e3140b14a2ec915345b2747eecdecff943
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetHashCodeBenchmark - Duration of single invocation 79.04 ns 105.44 ns 1.33 0.06 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<Byte>.GetHashCodeBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 105.4351294570456 > 83.15154770429102.
IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -38.87873117596192 (T) = (0 -105.35273354069511) / Math.Sqrt((3.992587969600447 / (23)) + (7.861876182668589 / (31))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (31) - 2, .025) and -0.31787460021963265 = (79.94139466921767 - 105.35273354069511) / 79.94139466921767 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline a8549002b2df6bf64c7c882d8f117630beba762d
Compare 8a2b72e3140b14a2ec915345b2747eecdecff943
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<UInt16>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetHashCodeBenchmark - Duration of single invocation 39.22 ns 51.70 ns 1.32 0.05 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<UInt16>.GetHashCodeBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 51.69693024643556 > 41.1064538719314.
IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -36.663764455094125 (T) = (0 -51.68067070574212) / Math.Sqrt((1.65991615061608 / (22)) + (1.9274338623056948 / (31))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (31) - 2, .025) and -0.35719061117778034 = (38.07915430603609 - 51.68067070574212) / 38.07915430603609 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline a8549002b2df6bf64c7c882d8f117630beba762d
Compare 8a2b72e3140b14a2ec915345b2747eecdecff943
Diff Diff

Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetHashCodeBenchmark - Duration of single invocation 9.48 ns 12.98 ns 1.37 0.07 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.GetHashCodeBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.97857567164073 > 9.912366475799981.
IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -93.87222615864435 (T) = (0 -13.031390688994902) / Math.Sqrt((0.011639596703320161 / (23)) + (0.027995569909519624 / (31))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (31) - 2, .025) and -0.3706358320006545 = (9.50755144783686 - 13.031390688994902) / 9.50755144783686 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline a8549002b2df6bf64c7c882d8f117630beba762d
Compare 8a2b72e3140b14a2ec915345b2747eecdecff943
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<Double>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetHashCodeBenchmark - Duration of single invocation 13.38 ns 14.95 ns 1.12 0.03 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<Double>.GetHashCodeBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.948886232048702 > 14.071781964405073.
IsChangePoint: Marked as a change because one of 11/2/2022 4:25:40 PM, 12/14/2022 5:20:21 PM, 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -23.26476081715157 (T) = (0 -14.987631064522386) / Math.Sqrt((0.05384720602955832 / (23)) + (0.045076784078112886 / (31))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (31) - 2, .025) and -0.10574021657368117 = (13.554387223939488 - 14.987631064522386) / 13.554387223939488 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline a8549002b2df6bf64c7c882d8f117630beba762d
Compare 8a2b72e3140b14a2ec915345b2747eecdecff943
Diff Diff

Regressions in System.Tests.Perf_HashCode

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Combine_7 - Duration of single invocation 96.03 μs 105.47 μs 1.10 0.00 False
Combine_2 - Duration of single invocation 56.55 μs 63.40 μs 1.12 0.00 False
Combine_1 - Duration of single invocation 46.17 μs 52.16 μs 1.13 0.00 False
Combine_5 - Duration of single invocation 74.27 μs 82.59 μs 1.11 0.00 False
Combine_4 - Duration of single invocation 63.99 μs 71.89 μs 1.12 0.00 False
Combine_6 - Duration of single invocation 82.93 μs 93.13 μs 1.12 0.00 False
Combine_8 - Duration of single invocation 82.16 μs 90.71 μs 1.10 0.00 False
Combine_3 - Duration of single invocation 69.56 μs 74.71 μs 1.07 0.00 False

graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_HashCode*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_HashCode.Combine_7

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 105.4663142677373 > 100.81468016679449.
IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -161.18619752295376 (T) = (0 -105545.67733509371) / Math.Sqrt((6585.849273667598 / (23)) + (98379.09306130768 / (31))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (31) - 2, .025) and -0.09869467137568116 = (96064.61202086239 - 105545.67733509371) / 96064.61202086239 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_HashCode.Combine_2

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 63.40146640519568 > 59.350448427346585.
IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -521.8008790934781 (T) = (0 -63448.79334557399) / Math.Sqrt((1550.9166089246503 / (23)) + (3334.649210823573 / (31))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (31) - 2, .025) and -0.12207380790148624 = (56546.006954958306 - 63448.79334557399) / 56546.006954958306 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_HashCode.Combine_1

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 52.161184375000005 > 48.49616808904867.
IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -474.7956288486027 (T) = (0 -52200.563625643365) / Math.Sqrt((2370.9048049895378 / (22)) + (1632.5099591971323 / (31))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (31) - 2, .025) and -0.13020631808142258 = (46186.756161703495 - 52200.563625643365) / 46186.756161703495 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_HashCode.Combine_5

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 82.59137885338346 > 77.98060776806872.
IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -73.22615233862166 (T) = (0 -82505.62178616505) / Math.Sqrt((104660.75114867605 / (23)) + (244552.67765220546 / (31))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (31) - 2, .025) and -0.10986241775819691 = (74338.6031151659 - 82505.62178616505) / 74338.6031151659 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_HashCode.Combine_4

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 71.89388650884665 > 67.18613039001559.
IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -281.6459962191387 (T) = (0 -71670.10661699313) / Math.Sqrt((709.6575365608301 / (23)) + (22155.12236267368 / (31))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (31) - 2, .025) and -0.12019719537860528 = (63979.90185359284 - 71670.10661699313) / 63979.90185359284 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_HashCode.Combine_6

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 93.12999140625 > 87.02870333416007.
IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -115.48185935902607 (T) = (0 -93069.88297768412) / Math.Sqrt((3333.509569601689 / (23)) + (235566.862937844 / (31))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (31) - 2, .025) and -0.12257404511481351 = (82907.56710677843 - 93069.88297768412) / 82907.56710677843 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_HashCode.Combine_8

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 90.71372703516377 > 86.27665725675548.
IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -535.3374364970878 (T) = (0 -90757.96576528928) / Math.Sqrt((3413.288835681361 / (23)) + (3346.5812596985775 / (31))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (31) - 2, .025) and -0.10429181728794486 = (82186.57817114303 - 90757.96576528928) / 82186.57817114303 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_HashCode.Combine_3

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 74.7060919059011 > 73.07507285416666.
IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -140.11060272122677 (T) = (0 -74609.6146257151) / Math.Sqrt((1174.072760930738 / (23)) + (38213.510624717346 / (31))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (31) - 2, .025) and -0.07213825342420208 = (69589.54629911434 - 74609.6146257151) / 69589.54629911434 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline a8549002b2df6bf64c7c882d8f117630beba762d
Compare 8a2b72e3140b14a2ec915345b2747eecdecff943
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<Int64>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetHashCodeBenchmark - Duration of single invocation 10.81 ns 13.50 ns 1.25 0.03 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<Int64>.GetHashCodeBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.501432980336178 > 11.349420695138868.
IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -53.84020803276331 (T) = (0 -13.53650726544601) / Math.Sqrt((0.008029975980635195 / (24)) + (0.06952606554272087 / (31))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (31) - 2, .025) and -0.2530132020379094 = (10.803164119444345 - 13.53650726544601) / 10.803164119444345 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline a8549002b2df6bf64c7c882d8f117630beba762d
Compare 8a2b72e3140b14a2ec915345b2747eecdecff943
Diff Diff

Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetHashCodeBenchmark - Duration of single invocation 38.33 ns 54.55 ns 1.42 0.02 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;*'

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.GetHashCodeBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 54.54619767782453 > 40.21929684002488.
IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -82.21235737703599 (T) = (0 -54.1115179051457) / Math.Sqrt((0.06252101293171532 / (24)) + (1.0797309196087996 / (31))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (31) - 2, .025) and -0.4163482868152744 = (38.20495171199592 - 54.1115179051457) / 38.20495171199592 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline a8549002b2df6bf64c7c882d8f117630beba762d
Compare 8a2b72e3140b14a2ec915345b2747eecdecff943
Diff Diff

Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetHashCodeBenchmark - Duration of single invocation 19.54 ns 24.49 ns 1.25 0.03 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt16&gt;*'

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.GetHashCodeBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 24.489060082989262 > 20.74241711124787.
IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -24.00227051035366 (T) = (0 -24.845609585691953) / Math.Sqrt((0.07136336798950783 / (23)) + (1.2121066048338698 / (31))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (31) - 2, .025) and -0.2475992129108641 = (19.91473650237632 - 24.845609585691953) / 19.91473650237632 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline a8549002b2df6bf64c7c882d8f117630beba762d
Compare 8a2b72e3140b14a2ec915345b2747eecdecff943
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<UInt32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetHashCodeBenchmark - Duration of single invocation 20.29 ns 23.33 ns 1.15 0.04 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<UInt32>.GetHashCodeBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 23.331260832845107 > 21.056982512327213.
IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -25.472944182933528 (T) = (0 -23.645774694088463) / Math.Sqrt((0.1302430376724827 / (23)) + (0.4615801379454354 / (31))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (31) - 2, .025) and -0.1826468503281096 = (19.993943828225863 - 23.645774694088463) / 19.993943828225863 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline a8549002b2df6bf64c7c882d8f117630beba762d
Compare 8a2b72e3140b14a2ec915345b2747eecdecff943
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<UInt64>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetHashCodeBenchmark - Duration of single invocation 10.81 ns 13.54 ns 1.25 0.05 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<UInt64>.GetHashCodeBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.536802932034956 > 11.35145753448431.
IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -8.574082760730478 (T) = (0 -13.506688811973898) / Math.Sqrt((1.7762903502862928 / (23)) + (0.10097248716469229 / (31))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (31) - 2, .025) and -0.21965381121373115 = (11.074198832316851 - 13.506688811973898) / 11.074198832316851 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline a8549002b2df6bf64c7c882d8f117630beba762d
Compare 8a2b72e3140b14a2ec915345b2747eecdecff943
Diff Diff

Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetHashCodeBenchmark - Duration of single invocation 6.14 ns 8.24 ns 1.34 0.20 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;*'

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.GetHashCodeBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.243436857714752 > 6.447982991414564.
IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -8.389320644851091 (T) = (0 -8.063495790669068) / Math.Sqrt((0.03524392530140337 / (24)) + (1.7697574079138925 / (31))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (31) - 2, .025) and -0.33647761884757654 = (6.033393808436607 - 8.063495790669068) / 6.033393808436607 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline a8549002b2df6bf64c7c882d8f117630beba762d
Compare 8a2b72e3140b14a2ec915345b2747eecdecff943
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<Int16>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetHashCodeBenchmark - Duration of single invocation 37.85 ns 51.66 ns 1.37 0.07 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<Int16>.GetHashCodeBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 51.66120195005749 > 39.72074575315014.
IsChangePoint: Marked as a change because one of 1/10/2023 7:48:18 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM.
IsRegressionStdDev: Marked as regression because -39.559854435578245 (T) = (0 -51.881876605118066) / Math.Sqrt((0.8104291630112155 / (23)) + (2.8603275361064218 / (31))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (31) - 2, .025) and -0.37413871248681874 = (37.75592386246504 - 51.881876605118066) / 37.75592386246504 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Author: performanceautofiler[bot]
Assignees: EgorBo
Labels:

os-linux, tenet-performance, tenet-performance-benchmarks, arch-x64, area-CodeGen-coreclr
Milestone: -

@EgorBo
Copy link
Member

EgorBo commented Jan 17, 2023 •
edited
Loading

So it is no longer inlined:

multiplier in methods of struct increased to 3.
Inline candidate is generic and caller is not.  Multiplier increased to 5.
Inline has 2 foldable BOX ops.  Multiplier increased to 8.
Inline candidate callsite is in a loop.  Multiplier increased to 11.
calleeNativeSizeEstimate=1251
callsiteNativeSizeEstimate=85
benefit multiplier=11
threshold=935
Native estimate for function size exceeds threshold for inlining 125.1 > 93.5 (multiplier = 11)

@EgorBo
Copy link
Member

EgorBo commented Jan 17, 2023

So I assume Roslyn has fixed some IL bug or something like that (e.g. dotnet/roslyn#64789) so this is not actionable

@jakobbotsch
Copy link
Member

The codegen difference in the screenshot above looks like the result of dotnet/roslyn#65642, related issue in dotnet/runtime is #73606.

@EgorBo
Copy link
Member

EgorBo commented Jan 17, 2023

Performance impact of that change, FYI @jaredpar @AlekseyTs

@jaredpar
Copy link
Member

Is there a suggestion on how we can emit this code better? The runtime team asked us to fix a bug which to our knowledge requires the new codegen. Until we properly spill these values codegen can't fix their bug.

@EgorBo
Copy link
Member

EgorBo commented Jan 17, 2023 •
edited
Loading

Is there a suggestion on how we can emit this code better? The runtime team asked us to fix a bug which to our knowledge requires the new codegen. Until we properly spill these values codegen can't fix their bug.

I just wanted to let you know that we also try to track perf issues after Roslyn updates 🙂 - these regressions don't look bad to me at all honestly, the HashCode.Combine is perf critical, but the big performance difference is purely because of inlining. So I assume we can close this. Unless somebody thinks it should be improved.

@jakobbotsch
Copy link
Member

Is there a suggestion on how we can emit this code better? The runtime team asked us to fix a bug which to our knowledge requires the new codegen. Until we properly spill these values codegen can't fix their bug.

A copy is only needed when the receiver could actually be changed between loading its address and the constrained call. It is not the case in the above screenshot (and probably in most cases). However, it would of course require some analysis to determine whether it was safe to omit the copy.

@AlekseyTs
Copy link
Contributor

A copy is only needed when the receiver could actually be changed between loading its address and the constrained call. It is not the case in the above screenshot (and probably in most cases). However, it would of course require some analysis to determine whether it was safe to omit the copy.

We do detect some arguments "safe" for reordering, but it was not our goal to be able to accurately detect "safety" for non-trivial scenarios. I assume that the argument in the scenario is not trivial, perhaps another method call or something similar.

@jakobbotsch
Copy link
Member

@AlekseyTs maybe I am missing something, but there are no args in the above screenshot -- the IL emitted before is

ldarga.s value1
constrained. !!T1
callvirt instance int32 System.Object::GetHashCode()

@AlekseyTs
Copy link
Contributor

maybe I am missing something, but there are no args in the above screenshot

What C# code looks like? Perhaps the IL is different due to a different reason. Due to dotnet/roslyn#66250, for example.

@jakobbotsch
Copy link
Member

What C# code looks like? Perhaps the IL is different due to a different reason

runtime/src/libraries/System.Private.CoreLib/src/System/HashCode.cs

Line 87 in 63f46d2

uint hc1 = (uint)(value1?.GetHashCode() ?? 0);

So actually dotnet/roslyn#66250 might be the cause after all?

@AlekseyTs
Copy link
Contributor

I think so.

@jakobbotsch
Copy link
Member

We can change the runtime implementation here to avoid the conditional access to fix this regression. However, it kind of puts us down a path of never using conditional access on generically typed values that may be structs.

@tannergooding
Copy link
Member

tannergooding commented Jan 18, 2023 •
edited
Loading

@jakobbotsch what about putting some level of “trust” in the C# language features such as “readonly members” as a way to determine “this method mutates vs not”?

That is, rather than doing the complex analysis on the JIT side to determine if a non inlined method does mutate x, and would therefore require a copy to be preserved, we can check for the IsReadOnlyAttribute and use that as a much cheaper way of determining the same thing.

There are of course unsafe scenarios where that may change the observed output, but it will be correct in the vast majority of cases and match how C# will itself emit code in other scenarios involving the same (minus generics because no “readonly” concept exists for reference types, hence the general regression here)

That should allow us to much more trivially fix this case and improve codegen in a number of other places while also taking advantage of C# metadata that already tracks the info we care about

@jakobbotsch
Copy link
Member

That should allow us to much more trivially fix this case and improve codegen in a number of other places while also taking advantage of C# metadata that already tracks the info we care about

We likely have all information we need to eliminate this copy in the vast majority of cases even today (e.g. in the above case we can see that the copy is the last use, so nothing fundamentally prevents us from removing it). But actually doing so requires #8887.

@jakobbotsch
Copy link
Member

Once we have #8887 I think it makes good sense to experiment with using IsReadOnlyAttribute too as an input to whether reverse copy propagation can be done safely.

This was referenced Jan 19, 2023
Closed
Closed
Closed
@JulieLeeMSFT JulieLeeMSFT added this to the 8.0.0 milestone Apr 13, 2023
@EgorBo EgorBo modified the milestones: 8.0.0, Future Apr 24, 2023
@DrewScoggins DrewScoggins mentioned this issue Sep 5, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@EgorBo EgorBo

Labels
arch-x64 area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI os-linux Linux OS (any supported distro) tenet-performance Performance related issue tenet-performance-benchmarks Issue from performance benchmark
Projects
None yet
Milestone
Future
Development

No branches or pull requests
7 participants
@jaredpar @EgorBo @jakobbotsch @tannergooding @AlekseyTs @kunalspathak @JulieLeeMSFT