Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Perf] Linux/x64: 36 Regressions on 10/11/2022 10:51:47 AM #9055

Open
performanceautofiler bot opened this issue Oct 19, 2022 · 4 comments
Open

[Perf] Linux/x64: 36 Regressions on 10/11/2022 10:51:47 AM #9055

performanceautofiler bot opened this issue Oct 19, 2022 · 4 comments
Labels
arch-wasm arch-x64 branch-refs/heads/main kind-micro os-linux perf-regression untriaged

Comments

@performanceautofiler
Copy link

performanceautofiler bot commented Oct 19, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline f745eb9faee1bd67886bf7f7b684821424aa014d
Compare 2b07c8c46523b816b1eac5c8af092e52b20a69d2
Diff Diff

Regressions in System.Numerics.Tests.Perf_Matrix3x2

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
MultiplyByScalarBenchmark - Duration of single invocation 115.56 ns 129.74 ns 1.12 0.08 False
AddBenchmark - Duration of single invocation 140.67 ns 154.43 ns 1.10 0.11 False
NegateBenchmark - Duration of single invocation 115.91 ns 126.23 ns 1.09 0.05 False
SubtractBenchmark - Duration of single invocation 141.40 ns 154.53 ns 1.09 0.05 False
LerpBenchmark - Duration of single invocation 167.90 ns 182.56 ns 1.09 0.12 False
MultiplyByMatrixOperatorBenchmark - Duration of single invocation 167.55 ns 215.44 ns 1.29 0.08 False
MultiplyByScalarOperatorBenchmark - Duration of single invocation 108.17 ns 118.13 ns 1.09 0.10 False
MultiplyByMatrixBenchmark - Duration of single invocation 184.82 ns 237.20 ns 1.28 0.09 False
SubtractOperatorBenchmark - Duration of single invocation 124.08 ns 137.04 ns 1.10 0.11 False
NegationOperatorBenchmark - Duration of single invocation 104.53 ns 117.00 ns 1.12 0.08 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_Matrix3x2*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_Matrix3x2.MultiplyByScalarBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 129.74148286370792 > 124.88805434587783.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 8:25:03 PM falls between 10/7/2022 9:25:17 PM and 10/18/2022 8:25:03 PM.
IsRegressionStdDev: Marked as regression because -19.256380288724614 (T) = (0 -130.45689106555028) / Math.Sqrt((5.560508179993891 / (20)) + (5.090975230658562 / (32))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (32) - 2, .025) and -0.10814425835847044 = (117.72554889090004 - 130.45689106555028) / 117.72554889090004 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix3x2.AddBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 154.43450555466262 > 147.97939508184018.
IsChangePoint: Marked as a change because one of 10/7/2022 9:25:17 PM, 10/18/2022 8:25:03 PM falls between 10/7/2022 9:25:17 PM and 10/18/2022 8:25:03 PM.
IsRegressionStdDev: Marked as regression because -8.942411509328059 (T) = (0 -154.832071937561) / Math.Sqrt((4.664677171683494 / (12)) + (66.30925719461884 / (36))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (36) - 2, .025) and -0.0944031475428082 = (141.47626702755318 - 154.832071937561) / 141.47626702755318 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix3x2.NegateBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 126.23078363748031 > 120.43156006512582.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 8:25:03 PM falls between 10/7/2022 9:25:17 PM and 10/18/2022 8:25:03 PM.
IsRegressionStdDev: Marked as regression because -19.66971668145645 (T) = (0 -127.66701822474711) / Math.Sqrt((3.2559875136647567 / (19)) + (7.636335244808278 / (31))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (31) - 2, .025) and -0.11058720876475564 = (114.95451885020721 - 127.66701822474711) / 114.95451885020721 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix3x2.SubtractBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 154.53127529800955 > 147.23469516010223.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 8:25:03 PM falls between 10/7/2022 9:25:17 PM and 10/18/2022 8:25:03 PM.
IsRegressionStdDev: Marked as regression because -26.17084769219833 (T) = (0 -154.74415374811835) / Math.Sqrt((2.1059206180053973 / (20)) + (4.751857642243014 / (28))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (28) - 2, .025) and -0.09732128180256758 = (141.01991487299 - 154.74415374811835) / 141.01991487299 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix3x2.LerpBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 182.55667188056026 > 176.26638177589157.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 8:25:03 PM falls between 10/7/2022 9:25:17 PM and 10/18/2022 8:25:03 PM.
IsRegressionStdDev: Marked as regression because -17.055005988124364 (T) = (0 -184.38699432573335) / Math.Sqrt((6.408309141373444 / (18)) + (3.44172027692764 / (7))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (7) - 2, .025) and -0.09308845276379991 = (168.684422435827 - 184.38699432573335) / 168.684422435827 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix3x2.MultiplyByMatrixOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 215.4419677445951 > 176.99680726949234.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 8:25:03 PM falls between 10/7/2022 9:25:17 PM and 10/18/2022 8:25:03 PM.
IsRegressionStdDev: Marked as regression because -37.079376838254674 (T) = (0 -215.36459987677867) / Math.Sqrt((21.43721172409544 / (17)) + (8.775378872216649 / (33))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (33) - 2, .025) and -0.27024336308991104 = (169.54593594797205 - 215.36459987677867) / 169.54593594797205 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix3x2.MultiplyByScalarOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 118.12907997070442 > 113.06535182962054.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 8:25:03 PM falls between 10/7/2022 9:25:17 PM and 10/18/2022 8:25:03 PM.
IsRegressionStdDev: Marked as regression because -15.790610821955553 (T) = (0 -120.46721933596935) / Math.Sqrt((1.0080339512197796 / (18)) + (17.24996927937572 / (31))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (31) - 2, .025) and -0.11430649693495223 = (108.10959073408475 - 120.46721933596935) / 108.10959073408475 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix3x2.MultiplyByMatrixBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 237.2002444048293 > 194.86109330733902.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 8:25:03 PM falls between 10/7/2022 9:25:17 PM and 10/18/2022 8:25:03 PM.
IsRegressionStdDev: Marked as regression because -54.48789002468813 (T) = (0 -233.93076546122245) / Math.Sqrt((9.050020263738675 / (19)) + (8.42323665793898 / (31))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (31) - 2, .025) and -0.2522741723298146 = (186.80475141158746 - 233.93076546122245) / 186.80475141158746 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix3x2.SubtractOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 137.0440175970456 > 128.04355314730603.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 8:25:03 PM falls between 10/7/2022 9:25:17 PM and 10/18/2022 8:25:03 PM.
IsRegressionStdDev: Marked as regression because -14.255761545654519 (T) = (0 -135.94250935623148) / Math.Sqrt((12.874890070181044 / (20)) + (5.63144638543853 / (31))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (31) - 2, .025) and -0.1053054244581151 = (122.99090038654103 - 135.94250935623148) / 122.99090038654103 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix3x2.NegationOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 116.99938951990501 > 109.97998387466033.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 8:25:03 PM falls between 10/7/2022 9:25:17 PM and 10/18/2022 8:25:03 PM.
IsRegressionStdDev: Marked as regression because -21.651014219994433 (T) = (0 -116.9271800708819) / Math.Sqrt((1.6316445486198163 / (20)) + (6.648168332493054 / (32))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (32) - 2, .025) and -0.1106191395512301 = (105.28107783027122 - 116.9271800708819) / 105.28107783027122 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline f745eb9faee1bd67886bf7f7b684821424aa014d
Compare 2b07c8c46523b816b1eac5c8af092e52b20a69d2
Diff Diff

Regressions in System.Numerics.Tests.Perf_Quaternion

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
CreateFromAxisAngleBenchmark - Duration of single invocation 139.39 ns 154.36 ns 1.11 0.16 False
ConcatenateBenchmark - Duration of single invocation 233.07 ns 260.81 ns 1.12 0.14 False
CreateFromRotationMatrixBenchmark - Duration of single invocation 125.39 ns 154.21 ns 1.23 0.14 False
MultiplyByQuaternionBenchmark - Duration of single invocation 269.28 ns 299.73 ns 1.11 0.06 False
MultiplyByScalarBenchmark - Duration of single invocation 136.74 ns 156.05 ns 1.14 0.10 False
NegationOperatorBenchmark - Duration of single invocation 122.34 ns 142.90 ns 1.17 0.12 False
InverseBenchmark - Duration of single invocation 156.67 ns 187.42 ns 1.20 0.12 False
NormalizeBenchmark - Duration of single invocation 159.10 ns 173.17 ns 1.09 0.12 False

graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_Quaternion*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_Quaternion.CreateFromAxisAngleBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 154.3554105431983 > 146.76816169940645.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 8:25:03 PM falls between 10/7/2022 9:25:17 PM and 10/18/2022 8:25:03 PM.
IsRegressionStdDev: Marked as regression because -7.834239149662726 (T) = (0 -157.45173392545556) / Math.Sqrt((21.297203572713723 / (19)) + (26.58867093631373 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.12404657087599788 = (140.07581002871567 - 157.45173392545556) / 140.07581002871567 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Quaternion.ConcatenateBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 260.8082658081093 > 244.81102414781375.
IsChangePoint: Marked as a change because one of 9/7/2022 8:46:47 AM, 9/23/2022 4:51:48 AM, 10/11/2022 6:23:40 AM, 10/18/2022 8:25:03 PM falls between 10/7/2022 9:25:17 PM and 10/18/2022 8:25:03 PM.
IsRegressionStdDev: Marked as regression because -14.463050944188934 (T) = (0 -260.40439124918123) / Math.Sqrt((18.360960870826705 / (20)) + (13.881691732528422 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.10448562301701106 = (235.76983332554482 - 260.40439124918123) / 235.76983332554482 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Quaternion.CreateFromRotationMatrixBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 154.2121876441484 > 131.419026912988.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 8:25:03 PM falls between 10/7/2022 9:25:17 PM and 10/18/2022 8:25:03 PM.
IsRegressionStdDev: Marked as regression because -16.577419424516528 (T) = (0 -154.9900212173521) / Math.Sqrt((6.156447806743617 / (20)) + (20.072170375617944 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.23547033559384747 = (125.45021661151728 - 154.9900212173521) / 125.45021661151728 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Quaternion.MultiplyByQuaternionBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 299.73083443124585 > 283.6012450923734.
IsChangePoint: Marked as a change because one of 9/7/2022 8:46:47 AM, 9/23/2022 4:51:48 AM, 10/11/2022 6:23:40 AM, 10/18/2022 8:25:03 PM falls between 10/7/2022 9:25:17 PM and 10/18/2022 8:25:03 PM.
IsRegressionStdDev: Marked as regression because -22.039728306541377 (T) = (0 -301.66807758270545) / Math.Sqrt((21.362007116800715 / (18)) + (20.11406186216976 / (33))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (33) - 2, .025) and -0.10854756005474932 = (272.12912503980124 - 301.66807758270545) / 272.12912503980124 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Quaternion.MultiplyByScalarBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 156.05383627749993 > 145.04405490025064.
IsChangePoint: Marked as a change because one of 7/19/2022 10:33:10 AM, 10/11/2022 6:23:40 AM, 10/18/2022 8:25:03 PM falls between 10/7/2022 9:25:17 PM and 10/18/2022 8:25:03 PM.
IsRegressionStdDev: Marked as regression because -24.903809466559263 (T) = (0 -155.46404618370437) / Math.Sqrt((5.353556616302134 / (19)) + (5.702807157269131 / (33))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (33) - 2, .025) and -0.12108138929301386 = (138.6732914028164 - 155.46404618370437) / 138.6732914028164 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Quaternion.NegationOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 142.9023008258486 > 127.90752965027916.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 8:25:03 PM falls between 10/7/2022 9:25:17 PM and 10/18/2022 8:25:03 PM.
IsRegressionStdDev: Marked as regression because -27.65573783918533 (T) = (0 -141.0404090669416) / Math.Sqrt((2.950986089014227 / (16)) + (7.908065341633604 / (29))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (29) - 2, .025) and -0.1528371057615229 = (122.34201029969049 - 141.0404090669416) / 122.34201029969049 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Quaternion.InverseBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 187.4199538506305 > 166.1469699664517.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 8:25:03 PM falls between 10/7/2022 9:25:17 PM and 10/18/2022 8:25:03 PM.
IsRegressionStdDev: Marked as regression because -23.71314757280847 (T) = (0 -185.17330379978824) / Math.Sqrt((16.971543508364523 / (19)) + (6.669130115863812 / (31))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (31) - 2, .025) and -0.15582857524637156 = (160.20827635302015 - 185.17330379978824) / 160.20827635302015 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Quaternion.NormalizeBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 173.16937642005112 > 165.86239097441103.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 8:25:03 PM falls between 10/7/2022 9:25:17 PM and 10/18/2022 8:25:03 PM.
IsRegressionStdDev: Marked as regression because -31.75562591142125 (T) = (0 -173.06692511321094) / Math.Sqrt((3.15966737722625 / (18)) + (2.7738596720745954 / (31))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (31) - 2, .025) and -0.10431213549750883 = (156.71920967819642 - 173.06692511321094) / 156.71920967819642 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
@performanceautofiler
Copy link
Author

performanceautofiler bot commented Oct 19, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline f745eb9faee1bd67886bf7f7b684821424aa014d
Compare 2b07c8c46523b816b1eac5c8af092e52b20a69d2
Diff Diff

Regressions in System.Numerics.Tests.Perf_Matrix4x4

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
AddBenchmark - Duration of single invocation 245.75 ns 288.96 ns 1.18 0.08 False
MultiplyByScalarBenchmark - Duration of single invocation 199.61 ns 243.98 ns 1.22 0.11 False
AddOperatorBenchmark - Duration of single invocation 222.40 ns 267.24 ns 1.20 0.07 False
CreatePerspectiveOffCenterBenchmark - Duration of single invocation 188.28 ns 284.99 ns 1.51 0.12 False
MultiplyByScalarOperatorBenchmark - Duration of single invocation 188.70 ns 229.68 ns 1.22 0.14 False
TransformBenchmark - Duration of single invocation 508.54 ns 543.42 ns 1.07 0.08 False
CreateOrthographicOffCenterBenchmark - Duration of single invocation 141.35 ns 180.46 ns 1.28 0.22 False
NegateBenchmark - Duration of single invocation 196.33 ns 240.05 ns 1.22 0.08 False
CreateFromQuaternionBenchmark - Duration of single invocation 290.33 ns 327.06 ns 1.13 0.10 False
CreateRotationXWithCenterBenchmark - Duration of single invocation 156.48 ns 176.85 ns 1.13 0.16 False
CreateFromYawPitchRollBenchmarkBenchmark - Duration of single invocation 474.13 ns 546.64 ns 1.15 0.08 False
CreatePerspectiveFieldOfViewBenchmark - Duration of single invocation 176.28 ns 285.65 ns 1.62 0.13 False
LerpBenchmark - Duration of single invocation 327.90 ns 377.25 ns 1.15 0.06 False
MultiplyByMatrixOperatorBenchmark - Duration of single invocation 544.75 ns 598.83 ns 1.10 0.02 False
Transpose - Duration of single invocation 156.88 ns 202.76 ns 1.29 0.13 False
CreatePerspectiveBenchmark - Duration of single invocation 161.19 ns 263.65 ns 1.64 0.18 False
MultiplyByMatrixBenchmark - Duration of single invocation 568.10 ns 620.76 ns 1.09 0.04 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_Matrix4x4*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_Matrix4x4.AddBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 288.95950674548607 > 257.18177245069717.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 8:25:03 PM falls between 10/7/2022 9:25:17 PM and 10/18/2022 8:25:03 PM.
IsRegressionStdDev: Marked as regression because -28.350922745202205 (T) = (0 -292.8134178922455) / Math.Sqrt((7.690504113654699 / (19)) + (78.59538172231366 / (33))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (33) - 2, .025) and -0.1927797725406829 = (245.48824907429284 - 292.8134178922455) / 245.48824907429284 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix4x4.MultiplyByScalarBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 243.97936782880572 > 210.34982954625454.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 8:25:03 PM falls between 10/7/2022 9:25:17 PM and 10/18/2022 8:25:03 PM.
IsRegressionStdDev: Marked as regression because -48.5093790167221 (T) = (0 -244.92838919093427) / Math.Sqrt((9.80809725639616 / (19)) + (8.34564306084625 / (32))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (32) - 2, .025) and -0.21150878721723637 = (202.168066608514 - 244.92838919093427) / 202.168066608514 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix4x4.AddOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 267.241279777649 > 236.29542016394498.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 8:25:03 PM falls between 10/7/2022 9:25:17 PM and 10/18/2022 8:25:03 PM.
IsRegressionStdDev: Marked as regression because -42.1928171321367 (T) = (0 -270.0636401256716) / Math.Sqrt((12.102943487501989 / (17)) + (14.803518172425749 / (33))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (33) - 2, .025) and -0.20236577746253917 = (224.61021861052237 - 270.0636401256716) / 224.61021861052237 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix4x4.CreatePerspectiveOffCenterBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 284.99251860551925 > 196.75194268858522.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 8:25:03 PM falls between 10/7/2022 9:25:17 PM and 10/18/2022 8:25:03 PM.
IsRegressionStdDev: Marked as regression because -85.29608339049827 (T) = (0 -283.9828265825797) / Math.Sqrt((13.592836282196336 / (19)) + (3.235717137754713 / (6))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (6) - 2, .025) and -0.5070211407440645 = (188.43984261718342 - 283.9828265825797) / 188.43984261718342 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix4x4.MultiplyByScalarOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 229.68357247229318 > 198.38207113084277.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 8:25:03 PM falls between 10/7/2022 9:25:17 PM and 10/18/2022 8:25:03 PM.
IsRegressionStdDev: Marked as regression because -33.855682880917904 (T) = (0 -232.63373034512352) / Math.Sqrt((13.68103726877884 / (16)) + (20.648561622611023 / (32))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (32) - 2, .025) and -0.2169291585759325 = (191.16456262528442 - 232.63373034512352) / 191.16456262528442 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix4x4.TransformBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 543.4186864567295 > 534.0781301369268.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 8:25:03 PM falls between 10/7/2022 9:25:17 PM and 10/18/2022 8:25:03 PM.
IsRegressionStdDev: Marked as regression because -13.096632410406832 (T) = (0 -545.4385176073663) / Math.Sqrt((32.91484631459209 / (19)) + (42.59972876476363 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.07196889731316375 = (508.81935005248806 - 545.4385176073663) / 508.81935005248806 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix4x4.CreateOrthographicOffCenterBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 180.45719333296967 > 146.98615470293345.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 8:25:03 PM falls between 10/7/2022 9:25:17 PM and 10/18/2022 8:25:03 PM.
IsRegressionStdDev: Marked as regression because -19.746533349328907 (T) = (0 -182.83145508173146) / Math.Sqrt((11.57837675473383 / (19)) + (26.178844850775192 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.29072252853436137 = (141.65047176277295 - 182.83145508173146) / 141.65047176277295 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix4x4.NegateBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 240.04517612000564 > 209.23254851746609.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 8:25:03 PM falls between 10/7/2022 9:25:17 PM and 10/18/2022 8:25:03 PM.
IsRegressionStdDev: Marked as regression because -45.26121249736586 (T) = (0 -242.4763334520819) / Math.Sqrt((5.222765121943637 / (20)) + (4.693272294681625 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.21975916266540615 = (198.790335726788 - 242.4763334520819) / 198.790335726788 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix4x4.CreateFromQuaternionBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 327.0643613211186 > 305.266565634454.
IsChangePoint: Marked as a change because one of 9/7/2022 8:46:47 AM, 9/23/2022 4:51:48 AM, 10/11/2022 6:23:40 AM, 10/18/2022 8:25:03 PM falls between 10/7/2022 9:25:17 PM and 10/18/2022 8:25:03 PM.
IsRegressionStdDev: Marked as regression because -23.47221565503901 (T) = (0 -333.62052822246494) / Math.Sqrt((20.858671651244272 / (16)) + (43.92257257888038 / (29))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (29) - 2, .025) and -0.1339294982871218 = (294.2162883375215 - 333.62052822246494) / 294.2162883375215 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix4x4.CreateRotationXWithCenterBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 176.85116049650875 > 163.34019318613935.
IsChangePoint: Marked as a change because one of 10/10/2022 3:36:43 PM, 10/18/2022 8:25:03 PM falls between 10/7/2022 9:25:17 PM and 10/18/2022 8:25:03 PM.
IsRegressionStdDev: Marked as regression because -14.975360762909405 (T) = (0 -178.86004972304144) / Math.Sqrt((11.844481435756062 / (16)) + (49.64862816003449 / (34))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (34) - 2, .025) and -0.1418155774624681 = (156.64530529574128 - 178.86004972304144) / 156.64530529574128 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix4x4.CreateFromYawPitchRollBenchmarkBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 546.6391109528228 > 474.318583323906.
IsChangePoint: Marked as a change because one of 9/7/2022 8:46:47 AM, 10/11/2022 6:23:40 AM, 10/18/2022 8:25:03 PM falls between 10/7/2022 9:25:17 PM and 10/18/2022 8:25:03 PM.
IsRegressionStdDev: Marked as regression because -15.965014868165776 (T) = (0 -531.9681929281132) / Math.Sqrt((202.15064443849943 / (19)) + (249.7732059896235 / (32))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (32) - 2, .025) and -0.14796175432829772 = (463.4023659084197 - 531.9681929281132) / 463.4023659084197 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix4x4.CreatePerspectiveFieldOfViewBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 285.6476298075618 > 186.50187410928876.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 8:25:03 PM falls between 10/7/2022 9:25:17 PM and 10/18/2022 8:25:03 PM.
IsRegressionStdDev: Marked as regression because -55.532966677905186 (T) = (0 -285.9158446678087) / Math.Sqrt((25.155085795904753 / (18)) + (15.765298885351044 / (7))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (7) - 2, .025) and -0.5899692680930262 = (179.82476165135557 - 285.9158446678087) / 179.82476165135557 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix4x4.LerpBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 377.25047580289515 > 345.3037924881254.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 8:25:03 PM falls between 10/7/2022 9:25:17 PM and 10/18/2022 8:25:03 PM.
IsRegressionStdDev: Marked as regression because -43.87082174751285 (T) = (0 -379.8216646444323) / Math.Sqrt((22.415857294533108 / (19)) + (5.981808737933685 / (31))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (31) - 2, .025) and -0.15650918348413417 = (328.4207942907727 - 379.8216646444323) / 328.4207942907727 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix4x4.MultiplyByMatrixOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 598.8301777416322 > 574.5809171931131.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 8:25:03 PM falls between 10/7/2022 9:25:17 PM and 10/18/2022 8:25:03 PM.
IsRegressionStdDev: Marked as regression because -22.71798545527993 (T) = (0 -604.3012390064907) / Math.Sqrt((32.965209918129 / (20)) + (144.627238324082 / (32))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (32) - 2, .025) and -0.10297958046312036 = (547.8807130343755 - 604.3012390064907) / 547.8807130343755 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix4x4.Transpose

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 202.76353468152803 > 166.01502529922294.
IsChangePoint: Marked as a change because one of 7/26/2022 8:48:46 PM, 7/28/2022 3:42:13 AM, 10/11/2022 6:23:40 AM, 10/18/2022 8:25:03 PM falls between 10/7/2022 9:25:17 PM and 10/18/2022 8:25:03 PM.
IsRegressionStdDev: Marked as regression because -9.591887988787583 (T) = (0 -204.71918884253105) / Math.Sqrt((321.4809617166684 / (20)) + (14.431697539141858 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.24926945127051084 = (163.87112374702753 - 204.71918884253105) / 163.87112374702753 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix4x4.CreatePerspectiveBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 263.64663300782735 > 167.77215144611932.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 8:25:03 PM falls between 10/7/2022 9:25:17 PM and 10/18/2022 8:25:03 PM.
IsRegressionStdDev: Marked as regression because -79.8248680953714 (T) = (0 -264.9197772589929) / Math.Sqrt((13.884941539633394 / (20)) + (7.436579090419723 / (7))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (7) - 2, .025) and -0.6648847225484668 = (159.12199425644064 - 264.9197772589929) / 159.12199425644064 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix4x4.MultiplyByMatrixBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 620.7601366576256 > 597.5243993570197.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 8:25:03 PM falls between 10/7/2022 9:25:17 PM and 10/18/2022 8:25:03 PM.
IsRegressionStdDev: Marked as regression because -49.855996635294815 (T) = (0 -622.8472534070427) / Math.Sqrt((11.312122864877846 / (16)) + (13.602473578636895 / (29))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (29) - 2, .025) and -0.09505770266984687 = (568.7803043515299 - 622.8472534070427) / 568.7803043515299 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline f745eb9faee1bd67886bf7f7b684821424aa014d
Compare 2b07c8c46523b816b1eac5c8af092e52b20a69d2
Diff Diff

Regressions in System.Memory.ReadOnlySequence

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Slice_Repeat - Duration of single invocation 991.92 ns 1.07 μs 1.08 0.09 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.ReadOnlySequence*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Memory.ReadOnlySequence.Slice_Repeat(Segment: Single)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 1.06810634632899 > 1048.1654207258528.
IsChangePoint: Marked as a change because one of 7/19/2022 10:33:10 AM, 10/11/2022 6:23:40 AM, 10/18/2022 8:25:03 PM falls between 10/7/2022 9:25:17 PM and 10/18/2022 8:25:03 PM.
IsRegressionStdDev: Marked as regression because -6.2066886173990286 (T) = (0 -1080.9052254288672) / Math.Sqrt((761.4957932349139 / (19)) + (2190.9660669853843 / (33))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (33) - 2, .025) and -0.06298181042647728 = (1016.8614503339421 - 1080.9052254288672) / 1016.8614503339421 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@lewing
Copy link
Member

lewing commented Oct 19, 2022

Looks like dotnet/runtime#76669 again

@lewing
Copy link
Member

lewing commented Oct 19, 2022

cc @BrzVlad @SamMonoRT

@BrzVlad
Copy link
Member

BrzVlad commented Oct 19, 2022

@lewing will fix these later today

@BrzVlad BrzVlad mentioned this issue Oct 19, 2022
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
arch-wasm arch-x64 branch-refs/heads/main kind-micro os-linux perf-regression untriaged
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@lewing @BrzVlad