Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Perf] Changes at 6/2/2022 9:03:53 PM #5854

Closed
performanceautofiler bot opened this issue Jun 7, 2022 · 2 comments
Closed

[Perf] Changes at 6/2/2022 9:03:53 PM #5854

performanceautofiler bot opened this issue Jun 7, 2022 · 2 comments
Assignees
@AndyAyersMS
Labels
arch-x64 branch-refs/heads/main kind-micro os-linux perf-improvement runtime-coreclr

Comments

@performanceautofiler
Copy link

performanceautofiler bot commented Jun 7, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 1b4a9c47ba8e4ad53562e003240527b738b0e7d1
Compare fe2c1ed058facc3c29e75c9892888fa521be586b
Diff Diff

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Count - Duration of single invocation 5.33 ms 4.94 ms 0.93 0.01 False
Count - Duration of single invocation 7.02 ms 6.50 ms 0.93 0.01 False
Count - Duration of single invocation 95.51 ms 86.30 ms 0.90 0.03 False

graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "(?i)Twain", Options: Compiled)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.9361183939732145 < 5.070980465731647.
IsChangePoint: Marked as a change because one of 5/5/2022 1:04:22 PM, 5/11/2022 7:02:26 PM, 6/2/2022 5:58:32 PM, 6/7/2022 5:42:36 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 5:42:36 AM.
IsImprovementStdDev: Marked as improvement because 27.802094897507324 (T) = (0 -4920226.432120852) / Math.Sqrt((15669271375.155136 / (299)) + (2869970808.905974 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.06500150234570533 = (5262282.714319453 - 4920226.432120852) / 5262282.714319453 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "Tom.{10,25}river|river.{10,25}Tom", Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.496391366987179 < 6.700317208648312.
IsChangePoint: Marked as a change because one of 4/19/2022 11:38:14 PM, 5/11/2022 7:02:26 PM, 6/2/2022 5:58:32 PM, 6/7/2022 5:42:36 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 5:42:36 AM.
IsImprovementStdDev: Marked as improvement because 34.00160457886795 (T) = (0 -6580635.401818083) / Math.Sqrt((20863319545.278034 / (220)) + (1961513939.5614703 / (29))) is greater than 1.9696147551832122 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (220) + (29) - 2, .975) and 0.06179030489337059 = (7014034.7473921385 - 6580635.401818083) / 7014034.7473921385 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "(?i)Tom|Sawyer|Huckleberry|Finn", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 86.29900943333334 < 90.76328875363095.
IsChangePoint: Marked as a change because one of 4/13/2022 5:30:39 PM, 5/9/2022 5:23:46 PM, 5/11/2022 7:02:26 PM, 5/25/2022 12:28:02 PM, 5/30/2022 12:57:05 PM, 6/7/2022 5:42:36 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 5:42:36 AM.
IsImprovementStdDev: Marked as improvement because 17.987916191316078 (T) = (0 -83483642.5325055) / Math.Sqrt((12184213819913.959 / (299)) + (6250832611051.937 / (50))) is greater than 1.9668240032621265 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (50) - 2, .975) and 0.08065075105423426 = (90807321.18750049 - 83483642.5325055) / 90807321.18750049 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 1b4a9c47ba8e4ad53562e003240527b738b0e7d1
Compare fe2c1ed058facc3c29e75c9892888fa521be586b
Diff Diff

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Count - Duration of single invocation 73.41 μs 62.94 μs 0.86 0.09 False
Count - Duration of single invocation 80.63 μs 68.87 μs 0.85 0.01 False
Count - Duration of single invocation 92.89 μs 82.36 μs 0.89 0.03 False
Count - Duration of single invocation 56.77 μs 44.29 μs 0.78 0.24 False
Count - Duration of single invocation 86.99 μs 77.40 μs 0.89 0.02 False
Count - Duration of single invocation 89.40 μs 80.33 μs 0.90 0.01 False
Count - Duration of single invocation 88.28 μs 75.16 μs 0.85 0.01 False
Count - Duration of single invocation 102.14 μs 90.31 μs 0.88 0.04 False
Count - Duration of single invocation 50.44 μs 38.17 μs 0.76 0.27 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "[a-q][^u-z]{13}x", Options: None)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 62.94274967369477 < 69.49510918218176.
IsChangePoint: Marked as a change because one of 5/11/2022 7:02:26 PM, 6/2/2022 5:58:32 PM, 6/7/2022 5:42:36 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 5:42:36 AM.
IsImprovementStdDev: Marked as improvement because 11.986746312353306 (T) = (0 -60564.776618005155) / Math.Sqrt((35005199.8035854 / (299)) + (1609288.7132313815 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.07597064088658385 = (65544.21244376434 - 60564.776618005155) / 65544.21244376434 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Street", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 68.86863395467837 < 76.51533262207751.
IsChangePoint: Marked as a change because one of 4/19/2022 4:39:14 PM, 5/11/2022 7:02:26 PM, 5/27/2022 11:47:05 PM, 6/2/2022 5:58:32 PM, 6/7/2022 5:42:36 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 5:42:36 AM.
IsImprovementStdDev: Marked as improvement because 34.97812039968424 (T) = (0 -68086.34440568964) / Math.Sqrt((31512758.30524745 / (299)) + (1485577.6740537586 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.16895967624064476 = (81929.05020262944 - 68086.34440568964) / 81929.05020262944 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "[a-q][^u-z]{13}x", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 82.36349774997257 < 88.56456219886192.
IsChangePoint: Marked as a change because one of 4/19/2022 4:39:14 PM, 5/11/2022 7:02:26 PM, 6/2/2022 5:58:32 PM, 6/7/2022 5:42:36 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 5:42:36 AM.
IsImprovementStdDev: Marked as improvement because 10.367257138332999 (T) = (0 -80971.48431574617) / Math.Sqrt((55228250.05682824 / (299)) + (2381842.3824454024 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.062036206470957554 = (86326.8762337776 - 80971.48431574617) / 86326.8762337776 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Street", Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 44.29044169625947 < 53.77985343052034.
IsChangePoint: Marked as a change because one of 5/11/2022 7:02:26 PM, 6/2/2022 5:58:32 PM, 6/7/2022 5:42:36 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 5:42:36 AM.
IsImprovementStdDev: Marked as improvement because 12.111860109493284 (T) = (0 -44381.971514356796) / Math.Sqrt((36372256.88520385 / (299)) + (1873617.3237140712 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.10536638572754797 = (49609.1034433686 - 44381.971514356796) / 49609.1034433686 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Holmes", Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 77.39923653052804 < 82.95013678726963.
IsChangePoint: Marked as a change because one of 4/13/2022 5:30:39 PM, 5/11/2022 7:02:26 PM, 6/2/2022 5:58:32 PM, 6/7/2022 5:42:36 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 5:42:36 AM.
IsImprovementStdDev: Marked as improvement because 24.10278948325327 (T) = (0 -76251.82646646959) / Math.Sqrt((14605199.097869266 / (299)) + (561811.3863345549 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.07626400221502713 = (82547.20683107933 - 76251.82646646959) / 82547.20683107933 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sher[a-z]+|Hol[a-z]+", Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 80.33171192765569 < 85.13092397970547.
IsChangePoint: Marked as a change because one of 5/11/2022 7:02:26 PM, 6/2/2022 5:58:32 PM, 6/7/2022 5:42:36 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 5:42:36 AM.
IsImprovementStdDev: Marked as improvement because 24.559998997514928 (T) = (0 -80031.29375092458) / Math.Sqrt((9347113.52372115 / (220)) + (750309.3162578364 / (29))) is greater than 1.9696147551832122 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (220) + (29) - 2, .975) and 0.07427620993169559 = (86452.67045045854 - 80031.29375092458) / 86452.67045045854 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Holmes.{0,25}Watson|Watson.{0,25}Holmes", Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 75.15698563701925 < 83.82321511879817.
IsChangePoint: Marked as a change because one of 5/11/2022 7:02:26 PM, 6/2/2022 5:58:32 PM, 6/7/2022 5:42:36 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 5:42:36 AM.
IsImprovementStdDev: Marked as improvement because 36.378095371661225 (T) = (0 -72123.2884407862) / Math.Sqrt((16688385.213534953 / (220)) + (649942.0215453061 / (29))) is greater than 1.9696147551832122 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (220) + (29) - 2, .975) and 0.1365272470343493 = (83526.99977280614 - 72123.2884407862) / 83526.99977280614 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Street", Options: None)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 90.3133904868479 < 97.119727529616.
IsChangePoint: Marked as a change because one of 4/6/2022 1:01:14 PM, 5/11/2022 7:02:26 PM, 6/2/2022 5:58:32 PM, 6/7/2022 5:42:36 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 5:42:36 AM.
IsImprovementStdDev: Marked as improvement because 14.894961728484354 (T) = (0 -90283.72176798165) / Math.Sqrt((31313182.504223496 / (299)) + (1877274.3589712593 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.06359558166028768 = (96415.3094536427 - 90283.72176798165) / 96415.3094536427 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "[a-q][^u-z]{13}x", Options: Compiled)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 38.167533035714285 < 47.996937147575885.
IsChangePoint: Marked as a change because one of 5/11/2022 7:02:26 PM, 6/2/2022 5:58:32 PM, 6/7/2022 5:42:36 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 5:42:36 AM.
IsImprovementStdDev: Marked as improvement because 10.579181037018559 (T) = (0 -39046.06698139009) / Math.Sqrt((35927660.50655235 / (299)) + (2146489.048888431 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.10665737271079666 = (43707.829212038305 - 39046.06698139009) / 43707.829212038305 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 1b4a9c47ba8e4ad53562e003240527b738b0e7d1
Compare fe2c1ed058facc3c29e75c9892888fa521be586b
Diff Diff

Improvements in MicroBenchmarks.Serializers.Xml_ToStream<LoginViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DataContractSerializer_ - Duration of single invocation 963.27 ns 843.15 ns 0.88 0.02 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'MicroBenchmarks.Serializers.Xml_ToStream&lt;LoginViewModel&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Xml_ToStream<LoginViewModel>.DataContractSerializer_

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 843.1461364796004 < 922.0672538802505.
IsChangePoint: Marked as a change because one of 5/11/2022 7:02:26 PM, 6/2/2022 5:58:32 PM, 6/7/2022 5:42:36 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 5:42:36 AM.
IsImprovementStdDev: Marked as improvement because 10.894807740402639 (T) = (0 -834.5772702981759) / Math.Sqrt((4495.795015003779 / (299)) + (301.91249805678655 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.06178361268227143 = (889.5360191737356 - 834.5772702981759) / 889.5360191737356 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 1b4a9c47ba8e4ad53562e003240527b738b0e7d1
Compare fe2c1ed058facc3c29e75c9892888fa521be586b
Diff Diff

Improvements in System.Text.Encodings.Web.Tests.Perf_Encoders

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EncodeUtf8 - Duration of single invocation 51.09 ns 43.58 ns 0.85 0.05 False
EncodeUtf8 - Duration of single invocation 51.48 ns 42.15 ns 0.82 0.05 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Encodings.Web.Tests.Perf_Encoders*'

Payloads

Baseline
Compare

Histogram

System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: UnsafeRelaxed,no <escaping /> required,512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 43.57633969832132 < 47.98945437425616.
IsChangePoint: Marked as a change because one of 4/20/2022 2:49:46 PM, 4/25/2022 11:05:49 AM, 5/9/2022 11:56:12 AM, 5/31/2022 6:06:22 PM, 6/2/2022 5:58:32 PM, 6/7/2022 5:42:36 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 5:42:36 AM.
IsImprovementStdDev: Marked as improvement because 11.116956345053174 (T) = (0 -43.728780729656464) / Math.Sqrt((25.396709938977686 / (299)) + (0.00853347970276048 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.06909224282460157 = (46.97434347559877 - 43.728780729656464) / 46.97434347559877 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: JavaScript,no escaping required,512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 42.14522667802958 < 48.390695152692885.
IsChangePoint: Marked as a change because one of 4/20/2022 2:49:46 PM, 4/25/2022 11:05:49 AM, 5/9/2022 11:56:12 AM, 5/31/2022 6:06:22 PM, 6/2/2022 5:58:32 PM, 6/7/2022 5:42:36 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 5:42:36 AM.
IsImprovementStdDev: Marked as improvement because 12.296250106683864 (T) = (0 -43.205642377845166) / Math.Sqrt((26.955609289281778 / (299)) + (0.0979497475710694 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.08006879812813386 = (46.96616691545062 - 43.205642377845166) / 46.96616691545062 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 1b4a9c47ba8e4ad53562e003240527b738b0e7d1
Compare fe2c1ed058facc3c29e75c9892888fa521be586b
Diff Diff

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Count - Duration of single invocation 6.49 ms 5.34 ms 0.82 0.01 False
Count - Duration of single invocation 591.94 μs 537.99 μs 0.91 0.13 False
Count - Duration of single invocation 580.27 μs 514.51 μs 0.89 0.10 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Count(Pattern: "(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]).){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9])", Options: Compiled)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.336329031250001 < 6.16879928202505.
IsChangePoint: Marked as a change because one of 5/11/2022 9:09:01 PM, 6/2/2022 5:58:32 PM, 6/7/2022 5:42:36 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 5:42:36 AM.
IsImprovementStdDev: Marked as improvement because 16.305260557087127 (T) = (0 -5339301.218188313) / Math.Sqrt((294270398642.6992 / (299)) + (1060055231.9193994 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.0888933289971381 = (5860237.212741845 - 5339301.218188313) / 5860237.212741845 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Count(Pattern: "[\w\.+-]+@[\w\.-]+\.[\w\.-]+", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 537.9913172222222 < 561.1001019469245.
IsChangePoint: Marked as a change because one of 5/11/2022 11:37:12 AM, 6/2/2022 5:58:32 PM, 6/7/2022 5:42:36 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 5:42:36 AM.
IsImprovementStdDev: Marked as improvement because 10.421644038452982 (T) = (0 -534795.497117075) / Math.Sqrt((564582217.4023261 / (299)) + (199314927.75390196 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.05453494325655725 = (565642.7948369908 - 534795.497117075) / 565642.7948369908 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Count(Pattern: "[\w\.+-]+@[\w\.-]+\.[\w\.-]+", Options: None)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 514.5094084101383 < 556.7105487907847.
IsChangePoint: Marked as a change because one of 5/11/2022 7:02:26 PM, 6/2/2022 5:58:32 PM, 6/7/2022 5:42:36 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 5:42:36 AM.
IsImprovementStdDev: Marked as improvement because 11.90298445330233 (T) = (0 -524714.8129071216) / Math.Sqrt((527301227.7027579 / (299)) + (217839032.69157392 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.06462226120856789 = (560965.6838584663 - 524714.8129071216) / 560965.6838584663 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 1b4a9c47ba8e4ad53562e003240527b738b0e7d1
Compare fe2c1ed058facc3c29e75c9892888fa521be586b
Diff Diff

Improvements in System.Memory.Span<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IndexOfAnyFourValues - Duration of single invocation 789.61 ns 504.10 ns 0.64 0.02 False
BinarySearch - Duration of single invocation 17.16 ns 15.59 ns 0.91 0.07 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Span&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Int32>.IndexOfAnyFourValues(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 504.0950592766879 < 750.0000836270827.
IsChangePoint: Marked as a change because one of 5/11/2022 7:02:26 PM, 6/2/2022 5:58:32 PM, 6/7/2022 5:42:36 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 5:42:36 AM.
IsImprovementStdDev: Marked as improvement because 6.648108003070228 (T) = (0 -613.8155553511665) / Math.Sqrt((12278.392235609736 / (299)) + (965.0909882435719 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.08541008020520167 = (671.137459604721 - 613.8155553511665) / 671.137459604721 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Int32&gt;.BinarySearch(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 15.585748482853642 < 16.077918610844026.
IsChangePoint: Marked as a change because one of 5/30/2022 12:57:05 PM, 6/7/2022 5:42:36 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 5:42:36 AM.
IsImprovementStdDev: Marked as improvement because 19.140878067260914 (T) = (0 -15.618153104313958) / Math.Sqrt((0.1319640915624287 / (299)) + (0.3560547784640908 / (50))) is greater than 1.9668240032621265 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (50) - 2, .975) and 0.09631229420629328 = (17.28268848207531 - 15.618153104313958) / 17.28268848207531 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 1b4a9c47ba8e4ad53562e003240527b738b0e7d1
Compare fe2c1ed058facc3c29e75c9892888fa521be586b
Diff Diff

Improvements in BenchmarksGame.FannkuchRedux_2

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
RunBench - Duration of single invocation 144.00 ms 123.69 ms 0.86 0.01 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'BenchmarksGame.FannkuchRedux_2*'

Payloads

Baseline
Compare

Histogram

BenchmarksGame.FannkuchRedux_2.RunBench(n: 10, expectedSum: 73196)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 123.69279159999999 < 136.82005989916667.
IsChangePoint: Marked as a change because one of 5/11/2022 7:02:26 PM, 6/2/2022 5:58:32 PM, 6/7/2022 5:42:36 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 5:42:36 AM.
IsImprovementStdDev: Marked as improvement because 15.40291678549438 (T) = (0 -121440130.48789723) / Math.Sqrt((124939815420247.67 / (299)) + (1560822292373.527 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.08012931416743276 = (132018698.23472284 - 121440130.48789723) / 132018698.23472284 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
@performanceautofiler
Copy link
Author

performanceautofiler bot commented Jun 7, 2022 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 1b4a9c47ba8e4ad53562e003240527b738b0e7d1
Compare fe2c1ed058facc3c29e75c9892888fa521be586b
Diff Diff

Improvements in System.Memory.ReadOnlySpan

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IndexOfString - Duration of single invocation 38.00 ns 23.08 ns 0.61 0.02 False
IndexOfString - Duration of single invocation 37.22 ns 23.07 ns 0.62 0.02 False
IndexOfString - Duration of single invocation 21.56 ns 14.71 ns 0.68 0.03 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.ReadOnlySpan*'

Payloads

Baseline
Compare

Histogram

System.Memory.ReadOnlySpan.IndexOfString(input: "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAXAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA", value: "X", comparisonType: Ordinal)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 23.082598151962298 < 35.97428361143132.
IsChangePoint: Marked as a change because one of 5/11/2022 7:02:26 PM, 6/2/2022 5:58:32 PM, 6/7/2022 5:42:36 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 5:42:36 AM.
IsImprovementStdDev: Marked as improvement because 13.267888285538927 (T) = (0 -23.116204672945827) / Math.Sqrt((49.28088954417491 / (299)) + (0.05794395409513715 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.18990680855743325 = (28.53524127487337 - 23.116204672945827) / 28.53524127487337 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.ReadOnlySpan.IndexOfString(input: "だだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだxだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだ", value: "x", comparisonType: Ordinal)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 23.065788175751788 < 35.530160896701766.
IsChangePoint: Marked as a change because one of 5/11/2022 7:02:26 PM, 6/2/2022 5:58:32 PM, 6/7/2022 5:42:36 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 5:42:36 AM.
IsImprovementStdDev: Marked as improvement because 13.196151182787045 (T) = (0 -23.11045060601579) / Math.Sqrt((49.504978207166744 / (299)) + (0.011666009072086859 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.1887226471956486 = (28.486497898813074 - 23.11045060601579) / 28.486497898813074 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.ReadOnlySpan.IndexOfString(input: "Hello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello Worldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylong!xyz", value: "~", comparisonType: Ordinal)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 14.706710123621297 < 20.478819925300897.
IsChangePoint: Marked as a change because one of 5/11/2022 7:02:26 PM, 6/2/2022 10:27:31 PM, 6/7/2022 5:42:36 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 5:42:36 AM.
IsImprovementStdDev: Marked as improvement because 13.95478444382205 (T) = (0 -14.767048454138491) / Math.Sqrt((10.074193379122105 / (299)) + (0.0023655314137862138 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.14798280577017361 = (17.331866720702788 - 14.767048454138491) / 17.331866720702788 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 1b4a9c47ba8e4ad53562e003240527b738b0e7d1
Compare fe2c1ed058facc3c29e75c9892888fa521be586b
Diff Diff

Improvements in Benchstone.BenchF.LLoops

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test - Duration of single invocation 539.01 ms 483.99 ms 0.90 0.01 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.BenchF.LLoops*'

Payloads

Baseline
Compare

Histogram

Benchstone.BenchF.LLoops.Test

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 483.98768884615384 < 513.8116249633333.
IsChangePoint: Marked as a change because one of 5/23/2022 7:50:08 PM, 6/2/2022 5:58:32 PM, 6/7/2022 5:42:36 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 5:42:36 AM.
IsImprovementStdDev: Marked as improvement because 40.767691192107506 (T) = (0 -463001680.59115833) / Math.Sqrt((185787249522978.94 / (299)) + (40846445645689.05 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.11146559950441817 = (521084698.95247525 - 463001680.59115833) / 521084698.95247525 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 1b4a9c47ba8e4ad53562e003240527b738b0e7d1
Compare fe2c1ed058facc3c29e75c9892888fa521be586b
Diff Diff

Improvements in MicroBenchmarks.Serializers.Xml_ToStream<IndexViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DataContractSerializer_ - Duration of single invocation 60.99 μs 51.74 μs 0.85 0.03 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'MicroBenchmarks.Serializers.Xml_ToStream&lt;IndexViewModel&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Xml_ToStream<IndexViewModel>.DataContractSerializer_

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 51.740600054847114 < 57.37620955926484.
IsChangePoint: Marked as a change because one of 5/11/2022 7:02:26 PM, 6/2/2022 5:58:32 PM, 6/7/2022 5:42:36 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 5:42:36 AM.
IsImprovementStdDev: Marked as improvement because 13.535831302748875 (T) = (0 -50871.09727249371) / Math.Sqrt((18864002.36447249 / (299)) + (582956.9554598513 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.07127579761856385 = (54775.246668548054 - 50871.09727249371) / 54775.246668548054 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 1b4a9c47ba8e4ad53562e003240527b738b0e7d1
Compare fe2c1ed058facc3c29e75c9892888fa521be586b
Diff Diff

Improvements in Benchstone.BenchI.CSieve

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test - Duration of single invocation 4.38 ms 3.51 ms 0.80 0.01 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.BenchI.CSieve*'

Payloads

Baseline
Compare

Histogram

Benchstone.BenchI.CSieve.Test

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.5116784679166666 < 4.151560255234375.
IsChangePoint: Marked as a change because one of 5/11/2022 7:02:26 PM, 6/2/2022 5:58:32 PM, 6/7/2022 5:42:36 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 5:42:36 AM.
IsImprovementStdDev: Marked as improvement because 13.209654326722156 (T) = (0 -3538995.3362710774) / Math.Sqrt((179985496776.31744 / (299)) + (616298691.1916943 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.08523862582783472 = (3868763.413271329 - 3538995.3362710774) / 3868763.413271329 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 1b4a9c47ba8e4ad53562e003240527b738b0e7d1
Compare fe2c1ed058facc3c29e75c9892888fa521be586b
Diff Diff

Improvements in Benchstone.BenchI.IniArray

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test - Duration of single invocation 103.20 ms 46.09 ms 0.45 0.03 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.BenchI.IniArray*'

Payloads

Baseline
Compare

Histogram

Benchstone.BenchI.IniArray.Test

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 46.08617995 < 97.939310058125.
IsChangePoint: Marked as a change because one of 5/11/2022 7:02:26 PM, 6/2/2022 5:58:32 PM, 6/7/2022 5:42:36 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 5:42:36 AM.
IsImprovementStdDev: Marked as improvement because 12.924262647965065 (T) = (0 -51627198.98318966) / Math.Sqrt((729013966687812.5 / (299)) + (2538587780785.1963 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.284615521100103 = (72167066.11049329 - 51627198.98318966) / 72167066.11049329 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 1b4a9c47ba8e4ad53562e003240527b738b0e7d1
Compare fe2c1ed058facc3c29e75c9892888fa521be586b
Diff Diff

Improvements in MicroBenchmarks.Serializers.Xml_ToStream<MyEventsListerViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DataContractSerializer_ - Duration of single invocation 607.93 μs 550.69 μs 0.91 0.02 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'MicroBenchmarks.Serializers.Xml_ToStream&lt;MyEventsListerViewModel&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Xml_ToStream<MyEventsListerViewModel>.DataContractSerializer_

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 550.6895631424376 < 575.6849668064501.
IsChangePoint: Marked as a change because one of 5/11/2022 7:02:26 PM, 6/2/2022 5:58:32 PM, 6/7/2022 5:42:36 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 5:42:36 AM.
IsImprovementStdDev: Marked as improvement because 12.148302162030093 (T) = (0 -540270.7938509464) / Math.Sqrt((1218103091.8573647 / (299)) + (59758554.31988647 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.052754379902122506 = (570359.7698294144 - 540270.7938509464) / 570359.7698294144 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 1b4a9c47ba8e4ad53562e003240527b738b0e7d1
Compare fe2c1ed058facc3c29e75c9892888fa521be586b
Diff Diff

Improvements in System.Net.Primitives.Tests.IPAddressPerformanceTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryWriteBytes - Duration of single invocation 15.95 ns 13.75 ns 0.86 0.01 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Net.Primitives.Tests.IPAddressPerformanceTests*'

Payloads

Baseline
Compare

Histogram

System.Net.Primitives.Tests.IPAddressPerformanceTests.TryWriteBytes(address: 1020:3040:5060:7080:9010:1112:1314:1516)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.752308812807968 < 15.175850651095699.
IsChangePoint: Marked as a change because one of 5/11/2022 7:02:26 PM, 6/2/2022 5:58:32 PM, 6/7/2022 5:42:36 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 5:42:36 AM.
IsImprovementStdDev: Marked as improvement because 15.212444249708938 (T) = (0 -13.258262117059484) / Math.Sqrt((1.7549422354421642 / (299)) + (0.022518412507564107 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.08553707405332074 = (14.498414031748894 - 13.258262117059484) / 14.498414031748894 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 1b4a9c47ba8e4ad53562e003240527b738b0e7d1
Compare fe2c1ed058facc3c29e75c9892888fa521be586b
Diff Diff

Improvements in Benchstone.BenchF.MatInv4

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test - Duration of single invocation 1.68 ms 1.49 ms 0.89 0.02 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.BenchF.MatInv4*'

Payloads

Baseline
Compare

Histogram

Benchstone.BenchF.MatInv4.Test

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.4910623789772723 < 1.5854401082932692.
IsChangePoint: Marked as a change because one of 5/11/2022 7:02:26 PM, 5/31/2022 6:06:22 PM, 6/2/2022 5:58:32 PM, 6/7/2022 5:42:36 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 5:42:36 AM.
IsImprovementStdDev: Marked as improvement because 59.63910965844001 (T) = (0 -1484953.8103342883) / Math.Sqrt((1358838518.9740174 / (299)) + (36716713.694760494 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.08826735042355476 = (1628716.2810546916 - 1484953.8103342883) / 1628716.2810546916 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 1b4a9c47ba8e4ad53562e003240527b738b0e7d1
Compare fe2c1ed058facc3c29e75c9892888fa521be586b
Diff Diff

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_SliceSlice

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Count - Duration of single invocation 1.69 secs 1.56 secs 0.93 0.01 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_SliceSlice*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_SliceSlice.Count(Options: IgnoreCase, Compiled)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.56388443825 < 1.6159319103612502.
IsChangePoint: Marked as a change because one of 5/5/2022 1:04:22 PM, 5/11/2022 7:02:26 PM, 6/2/2022 5:58:32 PM, 6/7/2022 5:42:36 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 5:42:36 AM.
IsImprovementStdDev: Marked as improvement because 30.763177831298368 (T) = (0 -1576612843.4241378) / Math.Sqrt((1192481450643898.2 / (299)) + (137387865723785.72 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.054496727982171214 = (1667485338.320869 - 1576612843.4241378) / 1667485338.320869 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 1b4a9c47ba8e4ad53562e003240527b738b0e7d1
Compare fe2c1ed058facc3c29e75c9892888fa521be586b
Diff Diff

Improvements in MicroBenchmarks.Serializers.Xml_ToStream<Location>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DataContractSerializer_ - Duration of single invocation 1.69 μs 1.47 μs 0.87 0.03 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'MicroBenchmarks.Serializers.Xml_ToStream&lt;Location&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Xml_ToStream<Location>.DataContractSerializer_

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.4731838212246628 < 1.6358268186366374.
IsChangePoint: Marked as a change because one of 5/11/2022 7:02:26 PM, 6/2/2022 5:58:32 PM, 6/7/2022 5:42:36 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 5:42:36 AM.
IsImprovementStdDev: Marked as improvement because 12.799415954278594 (T) = (0 -1461.0351856929465) / Math.Sqrt((17308.770667504337 / (299)) + (431.05237342958713 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.06952766141669496 = (1570.208081540019 - 1461.0351856929465) / 1570.208081540019 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 1b4a9c47ba8e4ad53562e003240527b738b0e7d1
Compare fe2c1ed058facc3c29e75c9892888fa521be586b
Diff Diff

Improvements in PerfLabTests.LowLevelPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ForeachOverList100Elements - Duration of single invocation 11.75 ms 10.52 ms 0.90 0.13 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'PerfLabTests.LowLevelPerf*'

Payloads

Baseline
Compare

Histogram

PerfLabTests.LowLevelPerf.ForeachOverList100Elements

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.520100266666667 < 11.125787494285714.
IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 6/2/2022 5:58:32 PM, 6/7/2022 5:42:36 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 5:42:36 AM.
IsImprovementStdDev: Marked as improvement because 19.9847000502773 (T) = (0 -9547818.273450734) / Math.Sqrt((1033240317654.6504 / (299)) + (113967521326.22206 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.15245694207883786 = (11265289.92741613 - 9547818.273450734) / 11265289.92741613 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 1b4a9c47ba8e4ad53562e003240527b738b0e7d1
Compare fe2c1ed058facc3c29e75c9892888fa521be586b
Diff Diff

Improvements in Microsoft.Extensions.Primitives.StringSegmentBenchmark

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Indexer - Duration of single invocation 12.99 ns 6.65 ns 0.51 0.02 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Microsoft.Extensions.Primitives.StringSegmentBenchmark*'

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.Primitives.StringSegmentBenchmark.Indexer

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.648674383566641 < 12.22722479098961.
IsChangePoint: Marked as a change because one of 5/11/2022 7:02:26 PM, 6/2/2022 5:58:32 PM, 6/7/2022 5:42:36 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 5:42:36 AM.
IsImprovementStdDev: Marked as improvement because 13.242705881329384 (T) = (0 -6.921942772785003) / Math.Sqrt((8.750792299410906 / (299)) + (0.04961850337969621 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.2519025296544589 = (9.252728484147667 - 6.921942772785003) / 9.252728484147667 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@AndyAyersMS
Copy link
Member

Likely from dotnet/runtime#70144

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@AndyAyersMS AndyAyersMS

Labels
arch-x64 branch-refs/heads/main kind-micro os-linux perf-improvement runtime-coreclr
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@AndyAyersMS