Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Perf] Windows/x64: 36 Regressions on 10/3/2022 2:18:04 AM #8295

Closed
performanceautofiler bot opened this issue Oct 11, 2022 · 2 comments
Closed

[Perf] Windows/x64: 36 Regressions on 10/3/2022 2:18:04 AM #8295

performanceautofiler bot opened this issue Oct 11, 2022 · 2 comments
Labels
CoreClr Noise refs/heads/main Regression RunKind=micro untriaged Windows 10.0.19042 x64

Comments

@performanceautofiler
Copy link

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 144a33a01251fce466d7be9b7cafff02f05ae6cc
Compare 039ced57d88703bd88a0490e3209a8e1815bebbc
Diff Diff

Regressions in System.Tests.Perf_Version

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryFormat3 - Duration of single invocation 17.57 ns 19.64 ns 1.12 0.02 False
TryFormat2 - Duration of single invocation 13.85 ns 15.42 ns 1.11 0.04 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Version*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Tests.Perf_Version.TryFormat3

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 19.641482710403558 > 18.805197264324743.
IsChangePoint: Marked as a change because one of 8/17/2022 6:09:21 PM, 8/29/2022 8:26:03 AM, 9/6/2022 1:06:44 PM, 9/23/2022 1:16:35 AM, 9/29/2022 7:42:19 AM, 10/2/2022 11:57:04 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM.
IsRegressionStdDev: Marked as regression because -16.81508469909142 (T) = (0 -19.379391852577232) / Math.Sqrt((0.08462199891162775 / (21)) + (0.09637011455069715 / (35))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (35) - 2, .025) and -0.07696113844411497 = (17.99451360016075 - 19.379391852577232) / 17.99451360016075 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Version.TryFormat2

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 15.418055144525459 > 14.52794625448469.
IsChangePoint: Marked as a change because one of 8/17/2022 6:09:21 PM, 8/29/2022 8:26:03 AM, 9/6/2022 1:06:44 PM, 9/23/2022 1:16:35 AM, 9/29/2022 7:42:19 AM, 10/2/2022 11:57:04 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM.
IsRegressionStdDev: Marked as regression because -45.35556631811137 (T) = (0 -15.3545402860341) / Math.Sqrt((0.013725051362826698 / (21)) + (0.012963237874320166 / (34))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (34) - 2, .025) and -0.10500120984104509 = (13.89549635718757 - 15.3545402860341) / 13.89549635718757 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS Windows 10.0.19042
Baseline 144a33a01251fce466d7be9b7cafff02f05ae6cc
Compare 039ced57d88703bd88a0490e3209a8e1815bebbc
Diff Diff

Regressions in LinqBenchmarks

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Where01ForX - Duration of single invocation 170.61 ms 181.31 ms 1.06 0.03 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'LinqBenchmarks*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

LinqBenchmarks.Where01ForX

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 181.30568666666665 > 179.46412825000002.
IsChangePoint: Marked as a change because one of 8/1/2022 6:03:22 PM, 8/5/2022 10:45:33 AM, 8/16/2022 10:54:40 PM, 8/29/2022 8:26:03 AM, 9/6/2022 1:06:44 PM, 9/23/2022 1:16:35 AM, 9/29/2022 7:42:19 AM, 10/2/2022 11:57:04 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM.
IsRegressionStdDev: Marked as regression because -60.84793994797884 (T) = (0 -180126995.60547298) / Math.Sqrt((250654241221.11523 / (21)) + (422164672244.1919 / (34))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (34) - 2, .025) and -0.05564916806627041 = (170631494.87005058 - 180126995.60547298) / 170631494.87005058 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 144a33a01251fce466d7be9b7cafff02f05ae6cc
Compare 039ced57d88703bd88a0490e3209a8e1815bebbc
Diff Diff

Regressions in System.Text.Encodings.Web.Tests.Perf_Encoders

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EncodeUtf8 - Duration of single invocation 47.41 ns 56.06 ns 1.18 0.03 False
EncodeUtf8 - Duration of single invocation 71.41 ns 79.75 ns 1.12 0.05 False
EncodeUtf16 - Duration of single invocation 55.39 ns 60.13 ns 1.09 0.03 False
EncodeUtf8 - Duration of single invocation 47.43 ns 56.49 ns 1.19 0.03 False

graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Encodings.Web.Tests.Perf_Encoders*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: UnsafeRelaxed,no <escaping /> required,512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 56.06043383493468 > 50.01197430135582.
IsChangePoint: Marked as a change because one of 8/5/2022 10:45:33 AM, 8/29/2022 8:26:03 AM, 9/9/2022 3:12:19 AM, 9/13/2022 12:23:36 PM, 9/29/2022 7:42:19 AM, 10/2/2022 11:57:04 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM.
IsRegressionStdDev: Marked as regression because -61.51746058285239 (T) = (0 -55.49464465680392) / Math.Sqrt((0.16191070990531836 / (21)) + (0.2829726539714127 / (34))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (34) - 2, .025) and -0.16328119208575506 = (47.70527111961847 - 55.49464465680392) / 47.70527111961847 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: Url,©2020,512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 79.75297295499384 > 75.136294904437.
IsChangePoint: Marked as a change because one of 8/5/2022 10:45:33 AM, 8/15/2022 4:29:54 PM, 8/16/2022 10:54:40 PM, 8/24/2022 2:39:26 AM, 8/29/2022 8:26:03 AM, 9/9/2022 3:12:19 AM, 9/13/2022 12:23:36 PM, 9/29/2022 7:42:19 AM, 10/2/2022 11:57:04 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM.
IsRegressionStdDev: Marked as regression because -41.3459196795118 (T) = (0 -79.8295198296453) / Math.Sqrt((0.17091852015943582 / (21)) + (1.1209483088364838 / (34))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (34) - 2, .025) and -0.11733149134366294 = (71.44658541185945 - 79.8295198296453) / 71.44658541185945 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments: JavaScript,no escaping required,512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 60.125849486463515 > 58.12243161930432.
IsChangePoint: Marked as a change because one of 8/5/2022 10:45:33 AM, 8/29/2022 8:26:03 AM, 9/3/2022 11:44:47 AM, 9/6/2022 1:06:44 PM, 9/9/2022 3:12:19 AM, 9/13/2022 12:23:36 PM, 9/29/2022 7:42:19 AM, 10/2/2022 11:57:04 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM.
IsRegressionStdDev: Marked as regression because -14.591857870506622 (T) = (0 -62.32739941690325) / Math.Sqrt((2.9824132187553962 / (21)) + (2.1645669678095225 / (35))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (35) - 2, .025) and -0.11820132175289282 = (55.738978486627786 - 62.32739941690325) / 55.738978486627786 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: JavaScript,no escaping required,512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 56.49487512749784 > 49.87386927660149.
IsChangePoint: Marked as a change because one of 8/5/2022 10:45:33 AM, 8/29/2022 8:26:03 AM, 9/9/2022 3:12:19 AM, 9/13/2022 12:23:36 PM, 9/29/2022 7:42:19 AM, 10/2/2022 11:57:04 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM.
IsRegressionStdDev: Marked as regression because -71.51505169759601 (T) = (0 -55.55950668043871) / Math.Sqrt((0.08594362865565221 / (21)) + (0.2754975539050828 / (34))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (34) - 2, .025) and -0.16570079963154105 = (47.66189291282991 - 55.55950668043871) / 47.66189291282991 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 144a33a01251fce466d7be9b7cafff02f05ae6cc
Compare 039ced57d88703bd88a0490e3209a8e1815bebbc
Diff Diff

Regressions in System.Linq.Tests.Perf_Enumerable

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WhereSelect - Duration of single invocation 1.26 μs 1.39 μs 1.10 0.05 False
WhereAny_LastElementMatches - Duration of single invocation 641.32 ns 693.80 ns 1.08 0.06 False
Where - Duration of single invocation 1.03 μs 1.21 μs 1.18 0.04 False
WhereSelect - Duration of single invocation 838.22 ns 892.46 ns 1.06 0.05 False
WhereSingleOrDefault_LastElementMatches - Duration of single invocation 646.21 ns 699.36 ns 1.08 0.06 False

graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Linq.Tests.Perf_Enumerable*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Linq.Tests.Perf_Enumerable.WhereSelect(input: IEnumerable)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 1.3869659737917246 > 1.3224137782352103.
IsChangePoint: Marked as a change because one of 8/1/2022 6:03:22 PM, 8/5/2022 10:45:33 AM, 8/29/2022 8:26:03 AM, 10/2/2022 11:57:04 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM.
IsRegressionStdDev: Marked as regression because -30.3930071110728 (T) = (0 -1344.9230902395982) / Math.Sqrt((19.416668611202173 / (21)) + (268.51821018506314 / (34))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (34) - 2, .025) and -0.07195148196771066 = (1254.6492195437909 - 1344.9230902395982) / 1254.6492195437909 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.WhereAny_LastElementMatches(input: IEnumerable)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 693.7961281480212 > 674.5933344635762.
IsChangePoint: Marked as a change because one of 8/1/2022 6:03:22 PM, 8/5/2022 10:45:33 AM, 8/29/2022 8:26:03 AM, 9/6/2022 1:06:44 PM, 9/9/2022 3:12:19 AM, 9/29/2022 7:42:19 AM, 10/2/2022 11:57:04 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM.
IsRegressionStdDev: Marked as regression because -14.503167821227432 (T) = (0 -698.8667852014522) / Math.Sqrt((226.5511767872137 / (21)) + (44.97047230400795 / (35))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (35) - 2, .025) and -0.0777102834972359 = (648.4737094032236 - 698.8667852014522) / 648.4737094032236 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.Where(input: IEnumerable)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 1.2143515157595215 > 1.0807193514304996.
IsChangePoint: Marked as a change because one of 8/1/2022 6:03:22 PM, 8/5/2022 10:45:33 AM, 8/29/2022 8:26:03 AM, 9/6/2022 1:06:44 PM, 9/9/2022 3:12:19 AM, 9/29/2022 7:42:19 AM, 10/2/2022 11:57:04 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM.
IsRegressionStdDev: Marked as regression because -42.12369867259615 (T) = (0 -1146.5685709723007) / Math.Sqrt((16.530038579916532 / (21)) + (226.27558581820338 / (34))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (34) - 2, .025) and -0.11139016922043601 = (1031.6526119504367 - 1146.5685709723007) / 1031.6526119504367 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.WhereSelect(input: Array)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 892.4601333433018 > 879.8215841902805.
IsChangePoint: Marked as a change because one of 8/1/2022 6:03:22 PM, 8/5/2022 10:45:33 AM, 8/29/2022 8:26:03 AM, 9/6/2022 1:06:44 PM, 9/9/2022 3:12:19 AM, 9/29/2022 7:42:19 AM, 10/2/2022 11:57:04 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM.
IsRegressionStdDev: Marked as regression because -23.86989115258683 (T) = (0 -900.4321716359568) / Math.Sqrt((1.7687548520732566 / (21)) + (234.03814439037512 / (34))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (34) - 2, .025) and -0.07524015299055056 = (837.4242434414277 - 900.4321716359568) / 837.4242434414277 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.WhereSingleOrDefault_LastElementMatches(input: IEnumerable)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 699.3578264214498 > 678.1024516624277.
IsChangePoint: Marked as a change because one of 8/29/2022 8:26:03 AM, 10/2/2022 11:57:04 PM, 10/6/2022 5:26:59 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM.
IsRegressionStdDev: Marked as regression because -4.220235681974251 (T) = (0 -778.2514538396838) / Math.Sqrt((814.2235098266764 / (47)) + (4411.052633873471 / (8))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (47) + (8) - 2, .025) and -0.14852553588491169 = (677.6091863208418 - 778.2514538396838) / 677.6091863208418 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 144a33a01251fce466d7be9b7cafff02f05ae6cc
Compare 039ced57d88703bd88a0490e3209a8e1815bebbc
Diff Diff

Regressions in System.Tests.Perf_UInt16

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Parse - Duration of single invocation 15.81 ns 17.38 ns 1.10 0.06 False
Parse - Duration of single invocation 15.84 ns 17.30 ns 1.09 0.06 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_UInt16*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Tests.Perf_UInt16.Parse(value: "65535")

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 17.37516774226593 > 16.614001246757937.
IsChangePoint: Marked as a change because one of 8/1/2022 6:03:22 PM, 8/5/2022 6:57:51 AM, 8/29/2022 8:26:03 AM, 9/6/2022 1:06:44 PM, 9/23/2022 1:16:35 AM, 9/29/2022 7:42:19 AM, 10/2/2022 11:57:04 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM.
IsRegressionStdDev: Marked as regression because -22.63197348123363 (T) = (0 -17.20684903057178) / Math.Sqrt((0.006502902139741615 / (21)) + (0.11492093833002531 / (34))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (34) - 2, .025) and -0.0868316971667591 = (15.832119246639557 - 17.20684903057178) / 15.832119246639557 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt16.Parse(value: "12345")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 17.304450664865424 > 16.613814961263316.
IsChangePoint: Marked as a change because one of 8/1/2022 6:03:22 PM, 8/5/2022 10:45:33 AM, 8/29/2022 8:26:03 AM, 9/6/2022 1:06:44 PM, 9/23/2022 1:16:35 AM, 9/29/2022 7:42:19 AM, 10/2/2022 11:57:04 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM.
IsRegressionStdDev: Marked as regression because -29.11850217872461 (T) = (0 -17.167382143415164) / Math.Sqrt((0.009692589136243835 / (21)) + (0.055659942708093886 / (34))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (34) - 2, .025) and -0.08424794799239554 = (15.83344674546303 - 17.167382143415164) / 15.83344674546303 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 144a33a01251fce466d7be9b7cafff02f05ae6cc
Compare 039ced57d88703bd88a0490e3209a8e1815bebbc
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<SByte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetHashCodeBenchmark - Duration of single invocation 89.73 ns 101.82 ns 1.13 0.33 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_VectorOf<SByte>.GetHashCodeBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 101.81753000422552 > 94.33238137929224.
IsChangePoint: Marked as a change because one of 8/16/2022 10:54:40 PM, 8/29/2022 8:26:03 AM, 9/6/2022 5:23:23 PM, 9/13/2022 12:23:36 PM, 9/23/2022 1:16:35 AM, 9/29/2022 7:42:19 AM, 10/2/2022 11:57:04 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM.
IsRegressionStdDev: Marked as regression because -13.417179264415994 (T) = (0 -101.05691415815022) / Math.Sqrt((1.4603047887047276 / (21)) + (20.70411461318209 / (35))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (35) - 2, .025) and -0.12101371335437883 = (90.14779476315269 - 101.05691415815022) / 90.14779476315269 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 144a33a01251fce466d7be9b7cafff02f05ae6cc
Compare 039ced57d88703bd88a0490e3209a8e1815bebbc
Diff Diff

Regressions in System.Collections.IterateForEach<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Dictionary - Duration of single invocation 1.53 μs 1.72 μs 1.13 0.18 False
LinkedList - Duration of single invocation 2.00 μs 2.16 μs 1.08 0.05 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IterateForEach&lt;Int32&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.IterateForEach<Int32>.Dictionary(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 1.7243689213085767 > 1.6028806282245533.
IsChangePoint: Marked as a change because one of 8/5/2022 10:45:33 AM, 9/9/2022 3:12:19 AM, 9/13/2022 12:23:36 PM, 9/29/2022 2:35:34 PM, 10/2/2022 11:57:04 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM.
IsRegressionStdDev: Marked as regression because -10.580238000659914 (T) = (0 -1696.5686490199625) / Math.Sqrt((1876.7036156906656 / (21)) + (4924.765979854304 / (35))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (35) - 2, .025) and -0.10447545039311344 = (1536.0854317007288 - 1696.5686490199625) / 1536.0854317007288 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.IterateForEach&lt;Int32&gt;.LinkedList(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 2.157775841843776 > 2.1033367234688263.
IsChangePoint: Marked as a change because one of 8/1/2022 6:03:22 PM, 9/9/2022 3:12:19 AM, 9/13/2022 12:23:36 PM, 9/23/2022 1:16:35 AM, 9/29/2022 7:42:19 AM, 10/2/2022 11:57:04 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM.
IsRegressionStdDev: Marked as regression because -113.24524011789488 (T) = (0 -2153.5668908807397) / Math.Sqrt((33.14293102088747 / (21)) + (4.441483595292897 / (35))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (35) - 2, .025) and -0.07372849532091591 = (2005.690358657271 - 2153.5668908807397) / 2005.690358657271 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 144a33a01251fce466d7be9b7cafff02f05ae6cc
Compare 039ced57d88703bd88a0490e3209a8e1815bebbc
Diff Diff

Regressions in System.Memory.Span<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
BinarySearch - Duration of single invocation 15.07 ns 18.14 ns 1.20 0.05 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Span&lt;Int32&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Memory.Span<Int32>.BinarySearch(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 18.139930756513003 > 15.849897881448724.
IsChangePoint: Marked as a change because one of 9/1/2022 8:04:48 PM, 9/3/2022 11:44:47 AM, 9/9/2022 3:12:19 AM, 9/13/2022 12:23:36 PM, 9/29/2022 7:42:19 AM, 10/2/2022 11:57:04 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM.
IsRegressionStdDev: Marked as regression because -91.97862971422252 (T) = (0 -17.94074164718675) / Math.Sqrt((0.0018340342285102292 / (21)) + (0.0301997381032416 / (35))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (35) - 2, .025) and -0.1876964200754381 = (15.105494420912056 - 17.94074164718675) / 15.105494420912056 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
@performanceautofiler
Copy link
Author

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 144a33a01251fce466d7be9b7cafff02f05ae6cc
Compare 039ced57d88703bd88a0490e3209a8e1815bebbc
Diff Diff

Regressions in System.Text.Json.Document.Tests.Perf_EnumerateArray

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EnumerateUsingIndexer - Duration of single invocation 1.43 μs 1.51 μs 1.06 0.19 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Document.Tests.Perf_EnumerateArray*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.Json.Document.Tests.Perf_EnumerateArray.EnumerateUsingIndexer(TestCase: ArrayOfStrings)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 1.51382902996182 > 1.5046609303869856.
IsChangePoint: Marked as a change because one of 8/1/2022 6:03:22 PM, 8/28/2022 2:29:57 AM, 9/2/2022 10:11:45 PM, 9/23/2022 1:16:35 AM, 9/29/2022 7:42:19 AM, 10/2/2022 11:57:04 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM.
IsRegressionStdDev: Marked as regression because -20.808339641136502 (T) = (0 -1516.1817546348589) / Math.Sqrt((305.28411572859653 / (21)) + (0.6134032753810549 / (35))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (35) - 2, .025) and -0.05525178898680836 = (1436.7961944803797 - 1516.1817546348589) / 1436.7961944803797 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS Windows 10.0.19042
Baseline 144a33a01251fce466d7be9b7cafff02f05ae6cc
Compare 039ced57d88703bd88a0490e3209a8e1815bebbc
Diff Diff

Regressions in System.Text.Json.Reader.Tests.Perf_Base64

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReadBase64EncodedByteArray_HeavyEscaping - Duration of single invocation 95.36 ns 113.90 ns 1.19 0.09 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Reader.Tests.Perf_Base64*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.Json.Reader.Tests.Perf_Base64.ReadBase64EncodedByteArray_HeavyEscaping(NumberOfBytes: 100)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 113.90336824007501 > 99.96085810645666.
IsChangePoint: Marked as a change because one of 8/15/2022 4:29:54 PM, 9/9/2022 3:12:19 AM, 9/13/2022 12:23:36 PM, 9/29/2022 7:42:19 AM, 10/2/2022 11:57:04 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM.
IsRegressionStdDev: Marked as regression because -69.11276964763628 (T) = (0 -113.24997978936136) / Math.Sqrt((0.6314631660363842 / (21)) + (1.3449984231402763 / (35))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (35) - 2, .025) and -0.1900797292490177 = (95.16167447102565 - 113.24997978936136) / 95.16167447102565 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 144a33a01251fce466d7be9b7cafff02f05ae6cc
Compare 039ced57d88703bd88a0490e3209a8e1815bebbc
Diff Diff

Regressions in System.Buffers.Text.Tests.Utf8FormatterTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
FormatterDecimal - Duration of single invocation 38.66 ns 42.25 ns 1.09 0.04 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Text.Tests.Utf8FormatterTests*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Buffers.Text.Tests.Utf8FormatterTests.FormatterDecimal(value: 123456.789)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 42.24635543598898 > 40.48338243231206.
IsChangePoint: Marked as a change because one of 8/1/2022 6:03:22 PM, 8/5/2022 10:45:33 AM, 8/16/2022 10:54:40 PM, 8/29/2022 11:20:35 AM, 9/6/2022 1:06:44 PM, 9/13/2022 12:23:36 PM, 9/23/2022 1:16:35 AM, 9/29/2022 7:42:19 AM, 10/2/2022 11:57:04 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM.
IsRegressionStdDev: Marked as regression because -67.58319635991398 (T) = (0 -42.73633101397108) / Math.Sqrt((0.03452277839183999 / (21)) + (0.0761342370295623 / (35))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (35) - 2, .025) and -0.10831572638037855 = (38.55970821017097 - 42.73633101397108) / 38.55970821017097 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 144a33a01251fce466d7be9b7cafff02f05ae6cc
Compare 039ced57d88703bd88a0490e3209a8e1815bebbc
Diff Diff

Regressions in System.Collections.ContainsFalse<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ImmutableSortedSet - Duration of single invocation 27.92 μs 31.29 μs 1.12 0.07 False
HashSet - Duration of single invocation 2.52 μs 2.70 μs 1.07 0.03 False
ImmutableList - Duration of single invocation 1.16 ms 1.36 ms 1.17 0.08 False

graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsFalse&lt;Int32&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.ContainsFalse<Int32>.ImmutableSortedSet(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 31.287387515842834 > 29.463662855172593.
IsChangePoint: Marked as a change because one of 8/1/2022 6:03:22 PM, 8/5/2022 10:45:33 AM, 8/27/2022 12:34:40 AM, 9/9/2022 3:12:19 AM, 9/29/2022 7:42:19 AM, 10/2/2022 11:57:04 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM.
IsRegressionStdDev: Marked as regression because -20.585226570753825 (T) = (0 -31282.45672069344) / Math.Sqrt((135764.32985388528 / (21)) + (536714.1833231262 / (34))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (34) - 2, .025) and -0.10884188628927 = (28211.828131222494 - 31282.45672069344) / 28211.828131222494 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;Int32&gt;.HashSet(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 2.6990648981593677 > 2.641557519075349.
IsChangePoint: Marked as a change because one of 8/1/2022 6:03:22 PM, 8/5/2022 10:45:33 AM, 8/11/2022 4:03:37 AM, 8/16/2022 10:54:40 PM, 9/9/2022 3:12:19 AM, 9/29/2022 7:42:19 AM, 10/2/2022 11:57:04 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM.
IsRegressionStdDev: Marked as regression because -34.08007742126385 (T) = (0 -2702.3631352739426) / Math.Sqrt((387.40634373609674 / (21)) + (341.25481771714175 / (35))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (35) - 2, .025) and -0.07177444949522199 = (2521.391638461512 - 2702.3631352739426) / 2521.391638461512 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;Int32&gt;.ImmutableList(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 1.3560554086538463 > 1.2166489866071428.
IsChangePoint: Marked as a change because one of 8/5/2022 10:45:33 AM, 8/29/2022 8:26:03 AM, 10/2/2022 11:57:04 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM.
IsRegressionStdDev: Marked as regression because -10.45928385205396 (T) = (0 -1270800.1793380077) / Math.Sqrt((1044904859.9539095 / (21)) + (1109417968.5437384 / (34))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (34) - 2, .025) and -0.08073745559459147 = (1175863.9184378495 - 1270800.1793380077) / 1175863.9184378495 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 144a33a01251fce466d7be9b7cafff02f05ae6cc
Compare 039ced57d88703bd88a0490e3209a8e1815bebbc
Diff Diff

Regressions in System.Net.Tests.Perf_WebUtility

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Decode_NoDecodingRequired - Duration of single invocation 65.19 ns 77.14 ns 1.18 0.09 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Net.Tests.Perf_WebUtility*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Net.Tests.Perf_WebUtility.Decode_NoDecodingRequired

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 77.14241603259784 > 69.30543583788779.
IsChangePoint: Marked as a change because one of 8/23/2022 11:46:46 PM, 8/29/2022 8:26:03 AM, 9/6/2022 1:06:44 PM, 9/13/2022 12:23:36 PM, 9/23/2022 1:16:35 AM, 9/29/2022 7:42:19 AM, 10/2/2022 11:57:04 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM.
IsRegressionStdDev: Marked as regression because -38.25281611138021 (T) = (0 -77.80634452785289) / Math.Sqrt((1.711865204789127 / (21)) + (0.6703958319935904 / (34))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (34) - 2, .025) and -0.18543479728194776 = (65.63527973554767 - 77.80634452785289) / 65.63527973554767 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 144a33a01251fce466d7be9b7cafff02f05ae6cc
Compare 039ced57d88703bd88a0490e3209a8e1815bebbc
Diff Diff

Regressions in System.Memory.Span<Char>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
BinarySearch - Duration of single invocation 11.32 ns 12.50 ns 1.10 0.05 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Span&lt;Char&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Memory.Span<Char>.BinarySearch(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 12.497387346348425 > 11.887063290155096.
IsChangePoint: Marked as a change because one of 8/1/2022 6:03:22 PM, 8/16/2022 10:54:40 PM, 8/29/2022 8:26:03 AM, 9/6/2022 1:06:44 PM, 9/13/2022 12:23:36 PM, 9/23/2022 1:16:35 AM, 9/29/2022 7:42:19 AM, 10/2/2022 11:57:04 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM.
IsRegressionStdDev: Marked as regression because -13.411480832410863 (T) = (0 -12.69889252054326) / Math.Sqrt((0.00040336672972744075 / (21)) + (0.357299459251205 / (34))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (34) - 2, .025) and -0.12153362245461607 = (11.322792528279404 - 12.69889252054326) / 11.322792528279404 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 144a33a01251fce466d7be9b7cafff02f05ae6cc
Compare 039ced57d88703bd88a0490e3209a8e1815bebbc
Diff Diff

Regressions in System.Collections.IterateFor<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ImmutableList - Duration of single invocation 5.94 μs 6.63 μs 1.12 0.02 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IterateFor&lt;String&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.IterateFor<String>.ImmutableList(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 6.633444116449512 > 6.258621363344347.
IsChangePoint: Marked as a change because one of 8/5/2022 10:45:33 AM, 9/9/2022 3:12:19 AM, 10/2/2022 11:57:04 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM.
IsRegressionStdDev: Marked as regression because -35.70233122071311 (T) = (0 -6557.040079213085) / Math.Sqrt((1537.5374058677664 / (21)) + (7609.014083115113 / (34))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (34) - 2, .025) and -0.1035543737325435 = (5941.746265782318 - 6557.040079213085) / 5941.746265782318 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 144a33a01251fce466d7be9b7cafff02f05ae6cc
Compare 039ced57d88703bd88a0490e3209a8e1815bebbc
Diff Diff

Regressions in System.Tests.Perf_String

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToLower - Duration of single invocation 13.40 ns 14.61 ns 1.09 0.05 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_String*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Tests.Perf_String.ToLower(s: "test")

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 14.61217401727327 > 14.004573194331709.
IsChangePoint: Marked as a change because one of 8/1/2022 6:03:22 PM, 8/20/2022 7:42:33 PM, 8/23/2022 8:42:14 PM, 9/29/2022 7:42:19 AM, 10/2/2022 11:57:04 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM.
IsRegressionStdDev: Marked as regression because -12.206943505597492 (T) = (0 -14.36149305774571) / Math.Sqrt((0.08364555865209725 / (21)) + (0.0891374448318462 / (35))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (35) - 2, .025) and -0.07375035706769002 = (13.375076397614041 - 14.36149305774571) / 13.375076397614041 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 144a33a01251fce466d7be9b7cafff02f05ae6cc
Compare 039ced57d88703bd88a0490e3209a8e1815bebbc
Diff Diff

Regressions in System.Collections.IndexerSet<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IList - Duration of single invocation 2.01 μs 2.15 μs 1.07 0.08 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IndexerSet&lt;Int32&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.IndexerSet<Int32>.IList(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 2.1478599445933493 > 2.1048735295062997.
IsChangePoint: Marked as a change because one of 8/5/2022 10:45:33 AM, 8/16/2022 10:54:40 PM, 8/29/2022 8:26:03 AM, 9/6/2022 1:06:44 PM, 9/13/2022 12:23:36 PM, 9/29/2022 7:42:19 AM, 10/2/2022 11:57:04 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM.
IsRegressionStdDev: Marked as regression because -5.728286352827707 (T) = (0 -2172.4495368599682) / Math.Sqrt((3349.4047591398025 / (21)) + (20104.85036556644 / (35))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (35) - 2, .025) and -0.07692829385888034 = (2017.2647977105187 - 2172.4495368599682) / 2017.2647977105187 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 144a33a01251fce466d7be9b7cafff02f05ae6cc
Compare 039ced57d88703bd88a0490e3209a8e1815bebbc
Diff Diff

Regressions in System.Memory.Span<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SequenceEqual - Duration of single invocation 9.29 ns 11.63 ns 1.25 0.08 False
Fill - Duration of single invocation 4.18 ns 6.08 ns 1.45 0.15 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Span&lt;Byte&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Memory.Span<Byte>.SequenceEqual(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 11.625297074352524 > 9.72884900180449.
IsChangePoint: Marked as a change because one of 8/5/2022 12:58:40 PM, 8/16/2022 10:54:40 PM, 8/29/2022 8:26:03 AM, 9/6/2022 1:06:44 PM, 9/13/2022 12:23:36 PM, 9/22/2022 3:11:11 PM, 9/29/2022 7:42:19 AM, 10/2/2022 11:57:04 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM.
IsRegressionStdDev: Marked as regression because -63.02119762040117 (T) = (0 -11.48678651828297) / Math.Sqrt((0.00034841771982148005 / (21)) + (0.041738169207843144 / (34))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (34) - 2, .025) and -0.239959536375523 = (9.263839811950271 - 11.48678651828297) / 9.263839811950271 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Byte&gt;.Fill(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 6.079492646362768 > 4.349334977089822.
IsChangePoint: Marked as a change because one of 8/3/2022 6:54:00 AM, 8/3/2022 4:57:40 PM, 8/4/2022 6:25:19 PM, 8/17/2022 6:09:21 PM, 8/19/2022 10:17:27 PM, 10/7/2022 9:25:17 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM.
IsRegressionStdDev: Marked as regression because -75.79407318799575 (T) = (0 -6.063078360645481) / Math.Sqrt((0.017916081553116366 / (54)) + (0.0005388575512174846 / (2))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (54) + (2) - 2, .025) and -0.4419962650739596 = (4.204642208511204 - 6.063078360645481) / 4.204642208511204 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 144a33a01251fce466d7be9b7cafff02f05ae6cc
Compare 039ced57d88703bd88a0490e3209a8e1815bebbc
Diff Diff

Regressions in System.Collections.IndexerSetReverse<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IList - Duration of single invocation 1.15 μs 1.29 μs 1.13 0.06 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IndexerSetReverse&lt;Int32&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.IndexerSetReverse<Int32>.IList(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 1.2942392403835494 > 1.2065036850932738.
IsChangePoint: Marked as a change because one of 8/5/2022 10:45:33 AM, 8/16/2022 10:54:40 PM, 8/29/2022 8:26:03 AM, 9/6/2022 1:06:44 PM, 9/13/2022 12:23:36 PM, 9/28/2022 6:40:13 PM, 9/29/2022 7:42:19 AM, 10/2/2022 11:57:04 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM.
IsRegressionStdDev: Marked as regression because -6.240406001585569 (T) = (0 -1327.0959460233444) / Math.Sqrt((0.07158813316757179 / (21)) + (28496.428188007525 / (35))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (35) - 2, .025) and -0.15496822578109434 = (1149.0324291179886 - 1327.0959460233444) / 1149.0324291179886 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 144a33a01251fce466d7be9b7cafff02f05ae6cc
Compare 039ced57d88703bd88a0490e3209a8e1815bebbc
Diff Diff

Regressions in System.Collections.IndexerSetReverse<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IList - Duration of single invocation 1.86 μs 2.01 μs 1.08 0.12 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IndexerSetReverse&lt;String&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.IndexerSetReverse<String>.IList(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 2.0069349182859346 > 1.9552596435453726.
IsChangePoint: Marked as a change because one of 8/1/2022 6:03:22 PM, 8/5/2022 4:49:35 PM, 8/16/2022 10:54:40 PM, 8/29/2022 8:26:03 AM, 9/6/2022 1:06:44 PM, 9/13/2022 12:23:36 PM, 9/23/2022 1:16:35 AM, 9/29/2022 7:42:19 AM, 10/2/2022 11:57:04 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM.
IsRegressionStdDev: Marked as regression because -20.90740644432648 (T) = (0 -2016.1477530545976) / Math.Sqrt((0.2314873883447474 / (21)) + (1840.2104573926367 / (34))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (34) - 2, .025) and -0.08260088356697608 = (1862.3185918819424 - 2016.1477530545976) / 1862.3185918819424 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS Windows 10.0.19042
Baseline 144a33a01251fce466d7be9b7cafff02f05ae6cc
Compare 039ced57d88703bd88a0490e3209a8e1815bebbc
Diff Diff

Regressions in Benchstone.MDBenchF.MDInvMt

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test - Duration of single invocation 2.49 ms 3.03 ms 1.22 0.06 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.MDBenchF.MDInvMt*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

Benchstone.MDBenchF.MDInvMt.Test

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 3.0344604166666667 > 2.624656791666667.
IsChangePoint: Marked as a change because one of 8/1/2022 6:03:22 PM, 8/5/2022 10:45:33 AM, 8/16/2022 10:54:40 PM, 8/29/2022 8:26:03 AM, 9/6/2022 10:17:32 AM, 9/13/2022 12:23:36 PM, 9/23/2022 1:16:35 AM, 9/29/2022 7:42:19 AM, 10/2/2022 11:57:04 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM.
IsRegressionStdDev: Marked as regression because -87.65750959576081 (T) = (0 -3041814.280612246) / Math.Sqrt((803134131.6906115 / (21)) + (84812091.67231621 / (35))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (35) - 2, .025) and -0.2251490297623856 = (2482811.6471693227 - 3041814.280612246) / 2482811.6471693227 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 144a33a01251fce466d7be9b7cafff02f05ae6cc
Compare 039ced57d88703bd88a0490e3209a8e1815bebbc
Diff Diff

Regressions in Benchstone.BenchI.Puzzle

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test - Duration of single invocation 433.41 ms 458.33 ms 1.06 0.02 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.BenchI.Puzzle*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

Benchstone.BenchI.Puzzle.Test

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 458.32670666666667 > 455.5945078461539.
IsChangePoint: Marked as a change because one of 8/1/2022 6:03:22 PM, 8/5/2022 10:45:33 AM, 8/29/2022 8:26:03 AM, 9/6/2022 1:06:44 PM, 9/23/2022 1:16:35 AM, 9/29/2022 7:42:19 AM, 10/2/2022 11:57:04 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM.
IsRegressionStdDev: Marked as regression because -18.52617759137284 (T) = (0 -461720369.1763475) / Math.Sqrt((30078402949145.355 / (21)) + (7843791513292.742 / (35))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (35) - 2, .025) and -0.054452543792653255 = (437876860.2668759 - 461720369.1763475) / 437876860.2668759 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 144a33a01251fce466d7be9b7cafff02f05ae6cc
Compare 039ced57d88703bd88a0490e3209a8e1815bebbc
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetHashCodeBenchmark - Duration of single invocation 89.37 ns 101.59 ns 1.14 0.23 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_VectorOf<Byte>.GetHashCodeBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 101.59027945325748 > 93.93501796476835.
IsChangePoint: Marked as a change because one of 8/17/2022 9:23:46 AM, 8/28/2022 2:29:57 AM, 9/6/2022 5:23:23 PM, 9/13/2022 12:23:36 PM, 9/23/2022 1:16:35 AM, 9/29/2022 7:42:19 AM, 10/2/2022 11:57:04 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM.
IsRegressionStdDev: Marked as regression because -14.968023867964858 (T) = (0 -104.77287283766987) / Math.Sqrt((1.8044255420436397 / (21)) + (28.645907171869236 / (35))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (35) - 2, .025) and -0.15721962812468077 = (90.53845120779567 - 104.77287283766987) / 90.53845120779567 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

This was referenced Oct 11, 2022
Open
Open
Open
Open
Open
Open
Open
Open
Open
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
CoreClr Noise refs/heads/main Regression RunKind=micro untriaged Windows 10.0.19042 x64
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@DrewScoggins