Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Perf] Windows/x64: 31 Regressions on 11/2/2022 4:25:40 PM #9560

Closed
performanceautofiler bot opened this issue Nov 8, 2022 · 2 comments
Closed

[Perf] Windows/x64: 31 Regressions on 11/2/2022 4:25:40 PM #9560

performanceautofiler bot opened this issue Nov 8, 2022 · 2 comments
Labels
arch-x64 branch-refs/heads/main kind-micro os-windows perf-regression PGO Applied if there were any profile guided optimization updates in the observed interval. runtime-coreclr untriaged

Comments

@performanceautofiler
Copy link

performanceautofiler bot commented Nov 8, 2022 •
edited
Loading

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<Int64>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
MultiplyBenchmark - Duration of single invocation 8.92 ns 9.92 ns 1.11 0.23 False
OnesComplementBenchmark - Duration of single invocation 0.04 ns 6.79 ns 174.26 0.11 False
MultiplyOperatorBenchmark - Duration of single invocation 8.63 ns 10.24 ns 1.19 0.01 False
OnesComplementOperatorBenchmark - Duration of single invocation 0.33 ns 6.79 ns 20.49 0.06 False

graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_VectorOf<Int64>.MultiplyBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 9.924407954873866 > 9.380941657231439.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsRegressionStdDev: Marked as regression because -23.208770162382724 (T) = (0 -9.949550129920484) / Math.Sqrt((0.07281614088964912 / (30)) + (0.018208059231963614 / (18))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (18) - 2, .025) and -0.15846474736975388 = (8.588565299471153 - 9.949550129920484) / 8.588565299471153 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.OnesComplementBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 6.790125047461315 > 0.04099140963554434.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsRegressionStdDev: Marked as regression because -212.96884720120684 (T) = (0 -6.844331234224709) / Math.Sqrt((0.0018059361770770984 / (30)) + (0.017239124658091548 / (18))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (18) - 2, .025) and -137.0828216697104 = (0.04956685525007685 - 6.844331234224709) / 0.04956685525007685 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.MultiplyOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 10.236456522195983 > 9.065207145886507.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsRegressionStdDev: Marked as regression because -36.2015427790211 (T) = (0 -9.980998539879657) / Math.Sqrt((0.020362441300664755 / (30)) + (0.021486694478803945 / (18))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (18) - 2, .025) and -0.1861678749032711 = (8.414490689771531 - 9.980998539879657) / 8.414490689771531 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;.OnesComplementOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 6.790334859271105 > 0.3479015176104111.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsRegressionStdDev: Marked as regression because -207.78737488047406 (T) = (0 -6.843048401163635) / Math.Sqrt((1.3702087561900537E-07 / (30)) + (0.017677082838601494 / (18))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (18) - 2, .025) and -19.64729248543175 = (0.3314259439096885 - 6.843048401163635) / 0.3314259439096885 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS Windows 10.0.18362
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<Double>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetHashCodeBenchmark - Duration of single invocation 16.43 ns 17.56 ns 1.07 0.01 False
EqualsBenchmark - Duration of single invocation 1.09 ns 9.92 ns 9.09 0.06 False
OnesComplementBenchmark - Duration of single invocation 0.04 ns 6.76 ns 173.61 0.07 False
OnesComplementOperatorBenchmark - Duration of single invocation 0.33 ns 7.07 ns 21.35 0.07 False

graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_VectorOf<Double>.GetHashCodeBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 17.560073445810733 > 17.250479851255434.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsRegressionStdDev: Marked as regression because -23.582288421770908 (T) = (0 -18.90565415739771) / Math.Sqrt((0.008344080178715707 / (30)) + (0.19849130009527532 / (18))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (18) - 2, .025) and -0.15290890679227098 = (16.398220228863316 - 18.90565415739771) / 16.398220228863316 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.EqualsBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 9.916796277306377 > 1.1464668517791967.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsRegressionStdDev: Marked as regression because -245.53570666253717 (T) = (0 -9.66710454991381) / Math.Sqrt((4.800430275461183E-06 / (30)) + (0.021947944209935098 / (18))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (18) - 2, .025) and -7.846965266188278 = (1.0927028940488754 - 9.66710454991381) / 1.0927028940488754 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.OnesComplementBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 6.762453476481473 > 0.04096144948226001.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsRegressionStdDev: Marked as regression because -188.44743599748554 (T) = (0 -6.892148838918826) / Math.Sqrt((1.7930513468871228E-07 / (30)) + (0.02380271417031148 / (18))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (18) - 2, .025) and -174.1652921129036 = (0.03934654380318939 - 6.892148838918826) / 0.03934654380318939 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;.OnesComplementOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 7.074197786627925 > 0.3478596896493219.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsRegressionStdDev: Marked as regression because -191.90712827726512 (T) = (0 -6.873482388076457) / Math.Sqrt((2.8774168902265765E-06 / (30)) + (0.020912956670522335 / (18))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (18) - 2, .025) and -19.707320670078307 = (0.3319348986568075 - 6.873482388076457) / 0.3319348986568075 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<Int16>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
OnesComplementBenchmark - Duration of single invocation 0.04 ns 6.76 ns 173.20 0.12 False
OnesComplementOperatorBenchmark - Duration of single invocation 0.33 ns 7.07 ns 21.35 0.08 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_VectorOf<Int16>.OnesComplementBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 6.762356191648517 > 0.04098585548342089.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsRegressionStdDev: Marked as regression because -246.2123796691402 (T) = (0 -6.803847075677239) / Math.Sqrt((0.005569301653433481 / (30)) + (0.010130357988170583 / (18))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (18) - 2, .025) and -98.97603098156782 = (0.06805478282021055 - 6.803847075677239) / 0.06805478282021055 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;.OnesComplementOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 7.0743562144771115 > 0.34798288190913307.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsRegressionStdDev: Marked as regression because -185.25012869009166 (T) = (0 -6.872086951893583) / Math.Sqrt((3.647115587399701E-07 / (30)) + (0.02243678616453442 / (18))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (18) - 2, .025) and -19.71896758071754 = (0.3316809549086417 - 6.872086951893583) / 0.3316809549086417 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<Single>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
OnesComplementOperatorBenchmark - Duration of single invocation 0.33 ns 6.79 ns 20.50 0.05 False
OnesComplementBenchmark - Duration of single invocation 0.04 ns 7.10 ns 182.03 0.11 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_VectorOf<Single>.OnesComplementOperatorBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 6.789964343972225 > 0.34800797248407855.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsRegressionStdDev: Marked as regression because -195.1479355296793 (T) = (0 -6.868918769338474) / Math.Sqrt((8.728154017387623E-07 / (30)) + (0.020197289398664026 / (18))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (18) - 2, .025) and -19.69588993964407 = (0.33189772410707974 - 6.868918769338474) / 0.33189772410707974 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;.OnesComplementBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 7.102869990397697 > 0.04099045380051155.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsRegressionStdDev: Marked as regression because -189.89346123155667 (T) = (0 -6.8715755277887896) / Math.Sqrt((2.2344891594832825E-07 / (30)) + (0.023301216908863144 / (18))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (18) - 2, .025) and -173.77486303968678 = (0.039316726720755237 - 6.8715755277887896) / 0.039316726720755237 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetHashCodeBenchmark - Duration of single invocation 22.09 ns 25.09 ns 1.14 0.05 False
OnesComplementOperatorBenchmark - Duration of single invocation 0.33 ns 6.76 ns 20.32 0.08 False
OnesComplementBenchmark - Duration of single invocation 0.04 ns 7.07 ns 176.78 0.11 False
SquareRootBenchmark - Duration of single invocation 15.97 ns 20.15 ns 1.26 0.03 False

graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_VectorOf<Int32>.GetHashCodeBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 25.094238532381475 > 22.731608185712.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsRegressionStdDev: Marked as regression because -22.860848103936604 (T) = (0 -28.187360743702683) / Math.Sqrt((0.06488344055135822 / (30)) + (1.3055237876915176 / (18))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (18) - 2, .025) and -0.2847748638915731 = (21.939533171067332 - 28.187360743702683) / 21.939533171067332 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.OnesComplementOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 6.762199820746956 > 0.34793111550063816.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsRegressionStdDev: Marked as regression because -190.97901211834562 (T) = (0 -6.868464373322808) / Math.Sqrt((7.525058263943504E-05 / (30)) + (0.021052162528292437 / (18))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (18) - 2, .025) and -19.80138135691388 = (0.3301927047763054 - 6.868464373322808) / 0.3301927047763054 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.OnesComplementBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 7.074192824273745 > 0.040967883778766194.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsRegressionStdDev: Marked as regression because -173.32298187355443 (T) = (0 -6.882654893301064) / Math.Sqrt((0.0058263305873151525 / (30)) + (0.024325119827798874 / (18))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (18) - 2, .025) and -99.33027311865584 = (0.06859998163426981 - 6.882654893301064) / 0.06859998163426981 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;.SquareRootBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 20.150429506374497 > 16.5950319188554.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsRegressionStdDev: Marked as regression because -53.10131479620377 (T) = (0 -19.742467201747278) / Math.Sqrt((0.012171968635715569 / (30)) + (0.08733722776229622 / (18))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (18) - 2, .025) and -0.24228542593188718 = (15.892054104182762 - 19.742467201747278) / 15.892054104182762 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<UInt32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SquareRootBenchmark - Duration of single invocation 16.43 ns 20.23 ns 1.23 0.04 False
OnesComplementBenchmark - Duration of single invocation 0.04 ns 6.76 ns 169.99 0.10 False
GetHashCodeBenchmark - Duration of single invocation 22.11 ns 25.09 ns 1.13 0.04 False
OnesComplementOperatorBenchmark - Duration of single invocation 0.33 ns 7.07 ns 21.27 0.10 False

graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_VectorOf<UInt32>.SquareRootBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 20.23223062282262 > 17.249276934990636.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsRegressionStdDev: Marked as regression because -38.137920068117026 (T) = (0 -20.47810251128054) / Math.Sqrt((0.13675615010944966 / (30)) + (0.10711260536042357 / (18))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (18) - 2, .025) and -0.23597272338091077 = (16.568409742299348 - 20.47810251128054) / 16.568409742299348 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.OnesComplementBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 6.7623034307219285 > 0.04096626356066246.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsRegressionStdDev: Marked as regression because -186.6948559252825 (T) = (0 -6.833324830686189) / Math.Sqrt((0.009810141816136294 / (30)) + (0.01759798205502751 / (18))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (18) - 2, .025) and -75.03964352032591 = (0.08986529281741827 - 6.833324830686189) / 0.08986529281741827 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.GetHashCodeBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 25.09356805342821 > 23.220890438686673.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsRegressionStdDev: Marked as regression because -22.56344293897268 (T) = (0 -28.16184364095539) / Math.Sqrt((0.044578923947614035 / (30)) + (1.3005925371796858 / (18))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (18) - 2, .025) and -0.2780696816117584 = (22.03466997624169 - 28.16184364095539) / 22.03466997624169 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;.OnesComplementOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 7.074306219108055 > 0.3480012809835305.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsRegressionStdDev: Marked as regression because -175.90452726654104 (T) = (0 -6.8840709878875) / Math.Sqrt((0.00037441175922029634 / (30)) + (0.02471750401949549 / (18))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (18) - 2, .025) and -19.483102568034024 = (0.3360853642665832 - 6.8840709878875) / 0.3360853642665832 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
@performanceautofiler
Copy link
Author

performanceautofiler bot commented Nov 8, 2022 •
edited
Loading

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<UInt16>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
OnesComplementOperatorBenchmark - Duration of single invocation 0.33 ns 6.76 ns 20.40 0.07 False
OnesComplementBenchmark - Duration of single invocation 0.04 ns 6.76 ns 173.54 0.08 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_VectorOf<UInt16>.OnesComplementOperatorBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 6.762506489222711 > 0.3479844738747354.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsRegressionStdDev: Marked as regression because -186.11431701471665 (T) = (0 -6.884950196957978) / Math.Sqrt((0.00012420550539667322 / (30)) + (0.022227272995050375 / (18))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (18) - 2, .025) and -19.622149801021834 = (0.3338619039910585 - 6.884950196957978) / 0.3338619039910585 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;.OnesComplementBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 6.762612157535205 > 0.04129620945884716.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsRegressionStdDev: Marked as regression because -592.1447438019158 (T) = (0 -6.767256089911493) / Math.Sqrt((0.0036510188507173813 / (30)) + (0.00011923348342629186 / (18))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (18) - 2, .025) and -112.90580997949093 = (0.059410982557693566 - 6.767256089911493) / 0.059410982557693566 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS Windows 10.0.18362
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<UInt64>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
MultiplyBenchmark - Duration of single invocation 8.67 ns 9.88 ns 1.14 0.20 False
OnesComplementBenchmark - Duration of single invocation 0.04 ns 6.76 ns 169.24 0.09 False
OnesComplementOperatorBenchmark - Duration of single invocation 0.33 ns 6.77 ns 20.33 0.04 False

graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_VectorOf<UInt64>.MultiplyBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 9.88254998255454 > 8.92004189596878.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsRegressionStdDev: Marked as regression because -23.090949185315928 (T) = (0 -9.980114163644972) / Math.Sqrt((0.07324236880664972 / (30)) + (0.023049272091450385 / (18))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (18) - 2, .025) and -0.16435179102897315 = (8.571390743363946 - 9.980114163644972) / 8.571390743363946 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.OnesComplementBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 6.762488394528183 > 0.04126335483401254.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsRegressionStdDev: Marked as regression because -215.06538626756142 (T) = (0 -6.8233502122940735) / Math.Sqrt((0.004218233850694481 / (30)) + (0.01526817007131273 / (18))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (18) - 2, .025) and -111.88857413337385 = (0.06044323143130966 - 6.8233502122940735) / 0.06044323143130966 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;.OnesComplementOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 6.766268761431905 > 0.347935010864776.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsRegressionStdDev: Marked as regression because -186.1593070849445 (T) = (0 -6.893584976237646) / Math.Sqrt((3.2237387307015748E-06 / (30)) + (0.02236095972826655 / (18))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (18) - 2, .025) and -19.767350447645583 = (0.33194340287252094 - 6.893584976237646) / 0.33194340287252094 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<SByte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
OnesComplementBenchmark - Duration of single invocation 0.04 ns 6.76 ns 173.64 0.09 False
OnesComplementOperatorBenchmark - Duration of single invocation 0.33 ns 6.76 ns 20.40 0.04 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_VectorOf<SByte>.OnesComplementBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 6.76249010647671 > 0.04096103821871357.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsRegressionStdDev: Marked as regression because -193.70893185697489 (T) = (0 -6.850972619275364) / Math.Sqrt((0.004075440677512255 / (30)) + (0.0196791268566774 / (18))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (18) - 2, .025) and -113.69340378149678 = (0.05973292616136147 - 6.850972619275364) / 0.05973292616136147 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;.OnesComplementOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 6.762332158180505 > 0.3479870868340348.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsRegressionStdDev: Marked as regression because -232.46857375901936 (T) = (0 -6.817433289350514) / Math.Sqrt((3.2775049131572877E-07 / (30)) + (0.014010719666229006 / (18))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (18) - 2, .025) and -19.555029672524565 = (0.3316674019917967 - 6.817433289350514) / 0.3316674019917967 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Regressions in System.Collections.IndexerSet<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Dictionary - Duration of single invocation 3.81 μs 4.39 μs 1.15 0.22 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IndexerSet&lt;Int32&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.IndexerSet<Int32>.Dictionary(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 4.388458920957691 > 4.023578055043763.
IsChangePoint: Marked as a change because one of 9/23/2022 1:16:35 AM, 9/26/2022 8:26:31 AM, 9/27/2022 11:22:50 AM, 9/29/2022 7:42:19 AM, 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsRegressionStdDev: Marked as regression because -11.705162970578977 (T) = (0 -4500.152776468205) / Math.Sqrt((2903.5423842743253 / (30)) + (57856.003437287865 / (18))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (18) - 2, .025) and -0.17601187712279534 = (3826.6218768794915 - 4500.152776468205) / 3826.6218768794915 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorConvert

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Convert_float_uint - Duration of single invocation 7.97 μs 12.27 μs 1.54 0.19 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorConvert*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_VectorConvert.Convert_float_uint

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 12.265712323077413 > 8.51185203476759.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsRegressionStdDev: Marked as regression because -87.45623541307633 (T) = (0 -12124.981505275406) / Math.Sqrt((34348.46404241066 / (30)) + (17259.07840399285 / (18))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (18) - 2, .025) and -0.4943981660816324 = (8113.621777968022 - 12124.981505275406) / 8113.621777968022 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline bdd67af67dbf43c0f5e9151b98dc100b9555898a
Compare fbc20578253e0deb7267eea5a02b680c3cfe0e2a
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
OnesComplementOperatorBenchmark - Duration of single invocation 0.33 ns 6.79 ns 20.39 0.08 False
OnesComplementBenchmark - Duration of single invocation 0.04 ns 6.79 ns 168.72 0.11 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_VectorOf<Byte>.OnesComplementOperatorBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 6.790681770048661 > 0.3479643192956441.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsRegressionStdDev: Marked as regression because -169.2576599932962 (T) = (0 -6.840202505658922) / Math.Sqrt((5.959203758596193E-07 / (30)) + (0.026614757852988363 / (18))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (18) - 2, .025) and -19.617164121898377 = (0.33177222945000706 - 6.840202505658922) / 0.33177222945000706 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;.OnesComplementBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 6.790797564396699 > 0.04093786377817169.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/8/2022 2:29:58 AM falls between 10/30/2022 12:44:54 AM and 11/8/2022 2:29:58 AM.
IsRegressionStdDev: Marked as regression because -312.19308654162023 (T) = (0 -6.79064570961086) / Math.Sqrt((0.00524677260275301 / (30)) + (0.005200570252663874 / (18))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (18) - 2, .025) and -100.11726580029136 = (0.0671561444612488 - 6.79064570961086) / 0.0671561444612488 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler performanceautofiler bot added CoreClr PGO Applied if there were any profile guided optimization updates in the observed interval. untriaged labels Nov 8, 2022
This was referenced Nov 8, 2022
Open
Closed
Open
Closed
Closed
Closed
Closed
Closed
Open
Closed
Open
Open
@EgorBo
Copy link
Member

EgorBo commented Nov 8, 2022

dotnet/runtime#78039

@EgorBo EgorBo closed this as completed Nov 8, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
arch-x64 branch-refs/heads/main kind-micro os-windows perf-regression PGO Applied if there were any profile guided optimization updates in the observed interval. runtime-coreclr untriaged
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@EgorBo