Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Perf] Linux/x64: 31 Regressions on 10/11/2022 10:51:47 AM #9040

Open
performanceautofiler bot opened this issue Oct 18, 2022 · 2 comments
Open

[Perf] Linux/x64: 31 Regressions on 10/11/2022 10:51:47 AM #9040

performanceautofiler bot opened this issue Oct 18, 2022 · 2 comments
Labels
arch-wasm arch-x64 branch-refs/heads/main kind-micro os-linux perf-regression untriaged

Comments

@performanceautofiler
Copy link

performanceautofiler bot commented Oct 18, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline f745eb9faee1bd67886bf7f7b684821424aa014d
Compare 20d4e309ec4f081e4b1d47a8f937abfbb763a4d4
Diff Diff

Regressions in System.Numerics.Tests.Perf_Matrix4x4

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
CreatePerspectiveOffCenterBenchmark - Duration of single invocation 188.28 ns 280.68 ns 1.49 0.12 False
AddOperatorBenchmark - Duration of single invocation 222.40 ns 272.61 ns 1.23 0.07 False
AddBenchmark - Duration of single invocation 245.75 ns 286.69 ns 1.17 0.08 False
CreateFromQuaternionBenchmark - Duration of single invocation 290.33 ns 331.17 ns 1.14 0.10 False
CreateOrthographicOffCenterBenchmark - Duration of single invocation 141.35 ns 182.15 ns 1.29 0.23 False
MultiplyByScalarBenchmark - Duration of single invocation 199.61 ns 244.67 ns 1.23 0.11 False
CreateFromYawPitchRollBenchmarkBenchmark - Duration of single invocation 474.13 ns 582.99 ns 1.23 0.07 False
MultiplyByScalarOperatorBenchmark - Duration of single invocation 188.70 ns 235.62 ns 1.25 0.14 False
TransformBenchmark - Duration of single invocation 508.54 ns 543.46 ns 1.07 0.08 False
CreateRotationXWithCenterBenchmark - Duration of single invocation 156.48 ns 177.82 ns 1.14 0.16 False
MultiplyByMatrixOperatorBenchmark - Duration of single invocation 544.75 ns 597.19 ns 1.10 0.02 False
NegateBenchmark - Duration of single invocation 196.33 ns 243.10 ns 1.24 0.08 False
LerpBenchmark - Duration of single invocation 327.90 ns 378.85 ns 1.16 0.06 False
MultiplyByMatrixBenchmark - Duration of single invocation 568.10 ns 624.90 ns 1.10 0.04 False
Transpose - Duration of single invocation 156.88 ns 202.32 ns 1.29 0.13 False
CreateRotationYWithCenterBenchmark - Duration of single invocation 155.51 ns 166.88 ns 1.07 0.15 False
CreatePerspectiveBenchmark - Duration of single invocation 161.19 ns 270.73 ns 1.68 0.18 False
CreatePerspectiveFieldOfViewBenchmark - Duration of single invocation 176.28 ns 285.28 ns 1.62 0.13 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_Matrix4x4*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_Matrix4x4.CreatePerspectiveOffCenterBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 280.6788356915935 > 196.75194268858522.
IsChangePoint: Marked as a change because one of 9/7/2022 8:46:47 AM, 9/23/2022 4:51:48 AM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 9:25:17 PM and 10/18/2022 1:35:00 AM.
IsRegressionStdDev: Marked as regression because -60.46014594028159 (T) = (0 -283.0411640231793) / Math.Sqrt((13.592836282196336 / (19)) + (5.198517089168111 / (3))) is less than -2.0859634472534285 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (3) - 2, .025) and -0.5020239886220824 = (188.43984261718342 - 283.0411640231793) / 188.43984261718342 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix4x4.AddOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 272.6126400899525 > 236.29542016394498.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 9:25:17 PM and 10/18/2022 1:35:00 AM.
IsRegressionStdDev: Marked as regression because -42.03728217813063 (T) = (0 -269.85189589761416) / Math.Sqrt((12.102943487501989 / (17)) + (13.38983059345168 / (30))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (30) - 2, .025) and -0.2014230588749017 = (224.61021861052237 - 269.85189589761416) / 224.61021861052237 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix4x4.AddBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 286.68828185433904 > 257.18177245069717.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 9:25:17 PM and 10/18/2022 1:35:00 AM.
IsRegressionStdDev: Marked as regression because -26.427644079788667 (T) = (0 -293.14747409768995) / Math.Sqrt((7.690504113654699 / (19)) + (85.42305569731856 / (30))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (30) - 2, .025) and -0.19414055541605113 = (245.48824907429284 - 293.14747409768995) / 245.48824907429284 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix4x4.CreateFromQuaternionBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 331.1661675129441 > 305.266565634454.
IsChangePoint: Marked as a change because one of 9/7/2022 8:46:47 AM, 9/23/2022 4:51:48 AM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 9:25:17 PM and 10/18/2022 1:35:00 AM.
IsRegressionStdDev: Marked as regression because -22.485738679117485 (T) = (0 -333.6528875768923) / Math.Sqrt((20.858671651244272 / (16)) + (46.080339022509904 / (26))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (26) - 2, .025) and -0.13403948320539463 = (294.2162883375215 - 333.6528875768923) / 294.2162883375215 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix4x4.CreateOrthographicOffCenterBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 182.15341585333996 > 146.98615470293345.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 9:25:17 PM and 10/18/2022 1:35:00 AM.
IsRegressionStdDev: Marked as regression because -15.714482704560158 (T) = (0 -185.4945865661309) / Math.Sqrt((11.57837675473383 / (19)) + (28.699852693834483 / (4))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (4) - 2, .025) and -0.30952325296018235 = (141.65047176277295 - 185.4945865661309) / 141.65047176277295 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix4x4.MultiplyByScalarBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 244.67003619601533 > 210.34982954625454.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 9:25:17 PM and 10/18/2022 1:35:00 AM.
IsRegressionStdDev: Marked as regression because -47.329320677109905 (T) = (0 -245.09851785011077) / Math.Sqrt((9.80809725639616 / (19)) + (8.889632196003673 / (29))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (29) - 2, .025) and -0.21235030814598888 = (202.168066608514 - 245.09851785011077) / 202.168066608514 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix4x4.CreateFromYawPitchRollBenchmarkBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 582.9903740918556 > 474.318583323906.
IsChangePoint: Marked as a change because one of 9/7/2022 8:46:47 AM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 9:25:17 PM and 10/18/2022 1:35:00 AM.
IsRegressionStdDev: Marked as regression because -15.229727604799805 (T) = (0 -531.3105485762889) / Math.Sqrt((202.15064443849943 / (19)) + (268.032114260371 / (29))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (29) - 2, .025) and -0.1465425894724274 = (463.4023659084197 - 531.3105485762889) / 463.4023659084197 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix4x4.MultiplyByScalarOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 235.6188319242504 > 198.38207113084277.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 9:25:17 PM and 10/18/2022 1:35:00 AM.
IsRegressionStdDev: Marked as regression because -32.8883171273413 (T) = (0 -232.66869871979324) / Math.Sqrt((13.68103726877884 / (16)) + (21.387748160204506 / (29))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (29) - 2, .025) and -0.21711208146807054 = (191.16456262528442 - 232.66869871979324) / 191.16456262528442 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix4x4.TransformBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 543.4642318715702 > 534.0781301369268.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 9:25:17 PM and 10/18/2022 1:35:00 AM.
IsRegressionStdDev: Marked as regression because -9.596665144436733 (T) = (0 -548.1119566948079) / Math.Sqrt((32.91484631459209 / (19)) + (60.12707536067065 / (4))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (4) - 2, .025) and -0.07722309821406462 = (508.81935005248806 - 548.1119566948079) / 508.81935005248806 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix4x4.CreateRotationXWithCenterBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 177.8216375955916 > 163.34019318613935.
IsChangePoint: Marked as a change because one of 10/10/2022 3:36:43 PM, 10/18/2022 1:35:00 AM falls between 10/7/2022 9:25:17 PM and 10/18/2022 1:35:00 AM.
IsRegressionStdDev: Marked as regression because -14.1888749563434 (T) = (0 -179.0329677639401) / Math.Sqrt((11.844481435756062 / (16)) + (54.22749611162178 / (31))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (31) - 2, .025) and -0.14291946015191218 = (156.64530529574128 - 179.0329677639401) / 156.64530529574128 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix4x4.MultiplyByMatrixOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 597.191944489509 > 574.5809171931131.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 9:25:17 PM and 10/18/2022 1:35:00 AM.
IsRegressionStdDev: Marked as regression because -21.423205075714808 (T) = (0 -604.8228260747137) / Math.Sqrt((32.965209918129 / (20)) + (157.07888905987957 / (29))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (29) - 2, .025) and -0.10393158891279587 = (547.8807130343755 - 604.8228260747137) / 547.8807130343755 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix4x4.NegateBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 243.09956738973722 > 209.23254851746609.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 9:25:17 PM and 10/18/2022 1:35:00 AM.
IsRegressionStdDev: Marked as regression because -31.13662047722784 (T) = (0 -242.9802129704776) / Math.Sqrt((5.222765121943637 / (20)) + (7.012247947723908 / (4))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (4) - 2, .025) and -0.2222938911095908 = (198.790335726788 - 242.9802129704776) / 198.790335726788 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix4x4.LerpBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 378.8507692877004 > 345.3037924881254.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 9:25:17 PM and 10/18/2022 1:35:00 AM.
IsRegressionStdDev: Marked as regression because -44.23931078195552 (T) = (0 -379.7970840431053) / Math.Sqrt((22.415857294533108 / (19)) + (4.729132274108863 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.1564343386455786 = (328.4207942907727 - 379.7970840431053) / 328.4207942907727 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix4x4.MultiplyByMatrixBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 624.9028504518559 > 597.5243993570197.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 9:25:17 PM and 10/18/2022 1:35:00 AM.
IsRegressionStdDev: Marked as regression because -48.87224395453263 (T) = (0 -622.6130097043292) / Math.Sqrt((11.312122864877846 / (16)) + (13.163575207448293 / (26))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (26) - 2, .025) and -0.09464586755368454 = (568.7803043515299 - 622.6130097043292) / 568.7803043515299 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix4x4.Transpose

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 202.32229660498933 > 166.01502529922294.
IsChangePoint: Marked as a change because one of 7/26/2022 8:48:46 PM, 7/27/2022 6:33:05 AM, 7/27/2022 5:17:06 PM, 7/28/2022 3:42:13 AM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 9:25:17 PM and 10/18/2022 1:35:00 AM.
IsRegressionStdDev: Marked as regression because -8.804444129721661 (T) = (0 -204.6284396557853) / Math.Sqrt((321.4809617166684 / (20)) + (21.42086606852416 / (4))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (4) - 2, .025) and -0.24871566739040601 = (163.87112374702753 - 204.6284396557853) / 163.87112374702753 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix4x4.CreateRotationYWithCenterBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 166.8751734459779 > 164.23247920295432.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 9:25:17 PM and 10/18/2022 1:35:00 AM.
IsRegressionStdDev: Marked as regression because -6.737213568766321 (T) = (0 -168.4713844734267) / Math.Sqrt((11.413557955681718 / (19)) + (51.48073327421657 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.06661883318651246 = (157.94900599131577 - 168.4713844734267) / 157.94900599131577 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix4x4.CreatePerspectiveBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 270.7341037748197 > 167.77215144611932.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 9:25:17 PM and 10/18/2022 1:35:00 AM.
IsRegressionStdDev: Marked as regression because -53.4205886267022 (T) = (0 -265.6589222822001) / Math.Sqrt((13.884941539633394 / (20)) + (13.132015757030464 / (4))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (4) - 2, .025) and -0.6695298693533515 = (159.12199425644064 - 265.6589222822001) / 159.12199425644064 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix4x4.CreatePerspectiveFieldOfViewBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 285.28018199116127 > 186.50187410928876.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 9:25:17 PM and 10/18/2022 1:35:00 AM.
IsRegressionStdDev: Marked as regression because -37.885317656185286 (T) = (0 -287.0817360418026) / Math.Sqrt((25.155085795904753 / (18)) + (26.470401041504758 / (4))) is less than -2.0859634472534285 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (4) - 2, .025) and -0.5964527543675936 = (179.82476165135557 - 287.0817360418026) / 179.82476165135557 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
@performanceautofiler
Copy link
Author

performanceautofiler bot commented Oct 18, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline f745eb9faee1bd67886bf7f7b684821424aa014d
Compare 2b07c8c46523b816b1eac5c8af092e52b20a69d2
Diff Diff

Regressions in System.Numerics.Tests.Perf_Quaternion

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
CreateFromRotationMatrixBenchmark - Duration of single invocation 125.39 ns 152.91 ns 1.22 0.14 False
NegationOperatorBenchmark - Duration of single invocation 122.34 ns 142.07 ns 1.16 0.12 False
CreateFromYawPitchRollBenchmark - Duration of single invocation 175.79 ns 205.10 ns 1.17 0.16 False
NormalizeBenchmark - Duration of single invocation 159.10 ns 175.30 ns 1.10 0.12 False
MultiplyByScalarBenchmark - Duration of single invocation 136.74 ns 155.89 ns 1.14 0.10 False
CreateFromAxisAngleBenchmark - Duration of single invocation 139.39 ns 156.23 ns 1.12 0.16 False
InverseBenchmark - Duration of single invocation 156.67 ns 188.35 ns 1.20 0.12 False
ConcatenateBenchmark - Duration of single invocation 233.07 ns 259.41 ns 1.11 0.14 False
MultiplyByScalarOperatorBenchmark - Duration of single invocation 125.46 ns 143.14 ns 1.14 0.13 False
ConjugateBenchmark - Duration of single invocation 118.95 ns 149.62 ns 1.26 0.14 False
MultiplyByQuaternionBenchmark - Duration of single invocation 269.28 ns 300.14 ns 1.11 0.07 False
MultiplyByQuaternionOperatorBenchmark - Duration of single invocation 236.73 ns 264.54 ns 1.12 0.17 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_Quaternion*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_Quaternion.CreateFromRotationMatrixBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 152.90988273012604 > 131.419026912988.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 9:25:17 PM and 10/18/2022 1:35:00 AM.
IsRegressionStdDev: Marked as regression because -12.276061899892563 (T) = (0 -157.00862477633922) / Math.Sqrt((6.156447806743617 / (20)) + (25.203267019023883 / (4))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (4) - 2, .025) and -0.25156120903759877 = (125.45021661151728 - 157.00862477633922) / 125.45021661151728 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Quaternion.NegationOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 142.07089593057177 > 127.90752965027916.
IsChangePoint: Marked as a change because one of 7/19/2022 10:33:10 AM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 9:25:17 PM and 10/18/2022 1:35:00 AM.
IsRegressionStdDev: Marked as regression because -26.976449554195906 (T) = (0 -140.73739450852557) / Math.Sqrt((2.950986089014227 / (16)) + (7.294513769245356 / (26))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (26) - 2, .025) and -0.15036032319375434 = (122.34201029969049 - 140.73739450852557) / 122.34201029969049 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Quaternion.CreateFromYawPitchRollBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 205.09616318884855 > 184.74256709760377.
IsChangePoint: Marked as a change because one of 7/19/2022 5:42:31 PM, 9/23/2022 2:49:56 PM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 9:25:17 PM and 10/18/2022 1:35:00 AM.
IsRegressionStdDev: Marked as regression because -4.985649532311814 (T) = (0 -201.3921984936399) / Math.Sqrt((110.9754047617291 / (18)) + (154.2100504930996 / (4))) is less than -2.0859634472534285 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (4) - 2, .025) and -0.1983887845055949 = (168.0524727012744 - 201.3921984936399) / 168.0524727012744 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Quaternion.NormalizeBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 175.29550660106278 > 165.86239097441103.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 9:25:17 PM and 10/18/2022 1:35:00 AM.
IsRegressionStdDev: Marked as regression because -31.386403714489333 (T) = (0 -173.0635100696496) / Math.Sqrt((3.15966737722625 / (18)) + (2.8691360005616846 / (30))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (30) - 2, .025) and -0.10429034465535005 = (156.71920967819642 - 173.0635100696496) / 156.71920967819642 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Quaternion.MultiplyByScalarBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 155.88969729579483 > 145.04405490025064.
IsChangePoint: Marked as a change because one of 7/19/2022 10:33:10 AM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 9:25:17 PM and 10/18/2022 1:35:00 AM.
IsRegressionStdDev: Marked as regression because -23.92746490206352 (T) = (0 -155.41807331692246) / Math.Sqrt((5.353556616302134 / (19)) + (6.239217175190184 / (30))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (30) - 2, .025) and -0.12074987003420888 = (138.6732914028164 - 155.41807331692246) / 138.6732914028164 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Quaternion.CreateFromAxisAngleBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 156.23044928457549 > 146.76816169940645.
IsChangePoint: Marked as a change because one of 7/20/2022 9:49:36 AM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 9:25:17 PM and 10/18/2022 1:35:00 AM.
IsRegressionStdDev: Marked as regression because -5.222876364577877 (T) = (0 -158.70717012048843) / Math.Sqrt((21.297203572713723 / (19)) + (46.417765581360555 / (4))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (4) - 2, .025) and -0.13300911904741666 = (140.07581002871567 - 158.70717012048843) / 140.07581002871567 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Quaternion.InverseBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 188.35173443012084 > 166.1469699664517.
IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 9:25:17 PM and 10/18/2022 1:35:00 AM.
IsRegressionStdDev: Marked as regression because -23.190524879344885 (T) = (0 -185.02275283409992) / Math.Sqrt((16.971543508364523 / (19)) + (7.048093007413185 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.1548888549702693 = (160.20827635302015 - 185.02275283409992) / 160.20827635302015 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Quaternion.ConcatenateBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 259.4084274629835 > 244.81102414781375.
IsChangePoint: Marked as a change because one of 9/7/2022 8:46:47 AM, 9/23/2022 4:51:48 AM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 9:25:17 PM and 10/18/2022 1:35:00 AM.
IsRegressionStdDev: Marked as regression because -9.933624318590065 (T) = (0 -258.9788682559691) / Math.Sqrt((18.360960870826705 / (20)) + (18.16308418780311 / (4))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (4) - 2, .025) and -0.0984393745504236 = (235.76983332554482 - 258.9788682559691) / 235.76983332554482 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Quaternion.MultiplyByScalarOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 143.13776974071442 > 131.481376803693.
IsChangePoint: Marked as a change because one of 7/19/2022 10:33:10 AM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 9:25:17 PM and 10/18/2022 1:35:00 AM.
IsRegressionStdDev: Marked as regression because -16.275516462902747 (T) = (0 -143.37574924277294) / Math.Sqrt((10.146719320575514 / (20)) + (20.68116581014655 / (30))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (30) - 2, .025) and -0.14178770259194348 = (125.57128520240607 - 143.37574924277294) / 125.57128520240607 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Quaternion.ConjugateBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 149.62201115462622 > 125.46632708663859.
IsChangePoint: Marked as a change because one of 7/19/2022 10:33:10 AM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 9:25:17 PM and 10/18/2022 1:35:00 AM.
IsRegressionStdDev: Marked as regression because -16.13231162453555 (T) = (0 -148.73763837322426) / Math.Sqrt((29.084720603528087 / (18)) + (3.883047557071334 / (4))) is less than -2.0859634472534285 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (4) - 2, .025) and -0.21129462154905287 = (122.79228828987328 - 148.73763837322426) / 122.79228828987328 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Quaternion.MultiplyByQuaternionBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 300.1395098060353 > 283.6012450923734.
IsChangePoint: Marked as a change because one of 7/19/2022 10:33:10 AM, 9/7/2022 8:46:47 AM, 9/23/2022 4:51:48 AM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 9:25:17 PM and 10/18/2022 1:35:00 AM.
IsRegressionStdDev: Marked as regression because -21.416068182615206 (T) = (0 -301.47614541299305) / Math.Sqrt((21.362007116800715 / (18)) + (20.730552419361906 / (30))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (30) - 2, .025) and -0.10784226190011656 = (272.12912503980124 - 301.47614541299305) / 272.12912503980124 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Quaternion.MultiplyByQuaternionOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 264.5442097818664 > 248.80508819415672.
IsChangePoint: Marked as a change because one of 9/7/2022 8:46:47 AM, 9/23/2022 4:51:48 AM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 9:25:17 PM and 10/18/2022 1:35:00 AM.
IsRegressionStdDev: Marked as regression because -16.454777053997688 (T) = (0 -265.30667614064237) / Math.Sqrt((19.064026620809155 / (20)) + (54.365147578728035 / (30))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (30) - 2, .025) and -0.11499932193347724 = (237.94335200185083 - 265.30667614064237) / 237.94335200185083 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline f745eb9faee1bd67886bf7f7b684821424aa014d
Compare 2b07c8c46523b816b1eac5c8af092e52b20a69d2
Diff Diff

Regressions in System.Memory.ReadOnlySequence

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Slice_Repeat - Duration of single invocation 991.92 ns 1.10 μs 1.11 0.09 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.ReadOnlySequence*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Memory.ReadOnlySequence.Slice_Repeat(Segment: Single)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 1.099906840894385 > 1048.1654207258528.
IsChangePoint: Marked as a change because one of 7/19/2022 10:33:10 AM, 10/11/2022 6:23:40 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 9:25:17 PM and 10/18/2022 1:35:00 AM.
IsRegressionStdDev: Marked as regression because -5.919889415931911 (T) = (0 -1081.797584325869) / Math.Sqrt((761.4957932349139 / (19)) + (2407.303438762993 / (30))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (30) - 2, .025) and -0.06385937235657961 = (1016.8614503339421 - 1081.797584325869) / 1016.8614503339421 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@lewing
Copy link
Member

lewing commented Oct 18, 2022

@BrzVlad looks like dotnet/runtime#76669 caused some regressions here?

@BrzVlad BrzVlad mentioned this issue Oct 19, 2022
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
arch-wasm arch-x64 branch-refs/heads/main kind-micro os-linux perf-regression untriaged
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@lewing