Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Perf] Windows/x64: 27 Improvements on 10/3/2022 2:18:04 AM #8328

Open
performanceautofiler bot opened this issue Oct 11, 2022 · 1 comment
Open

[Perf] Windows/x64: 27 Improvements on 10/3/2022 2:18:04 AM #8328

performanceautofiler bot opened this issue Oct 11, 2022 · 1 comment
Labels
CoreClr Improvement refs/heads/main RunKind=micro untriaged Windows 10.0.18362 x64

Comments

@performanceautofiler
Copy link

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 144a33a01251fce466d7be9b7cafff02f05ae6cc
Compare 039ced57d88703bd88a0490e3209a8e1815bebbc
Diff Diff

Improvements in PerfLabTests.CastingPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
CheckIsInstAnyIsInterfaceNo - Duration of single invocation 280.65 μs 249.47 μs 0.89 0.03 False
CheckObjIsInterfaceNo - Duration of single invocation 280.67 μs 249.47 μs 0.89 0.05 False
CheckObjIsInterfaceYes - Duration of single invocation 280.65 μs 218.29 μs 0.78 0.03 False
CheckIsInstAnyIsInterfaceYes - Duration of single invocation 281.82 μs 218.29 μs 0.77 0.03 False

graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'PerfLabTests.CastingPerf*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

PerfLabTests.CastingPerf.CheckIsInstAnyIsInterfaceNo

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 249.46830357142858 < 266.61935255592624.
IsChangePoint: Marked as a change because one of 8/1/2022 6:03:22 PM, 8/5/2022 10:45:33 AM, 8/16/2022 10:54:40 PM, 8/29/2022 8:26:03 AM, 9/6/2022 1:06:44 PM, 9/13/2022 12:23:36 PM, 9/23/2022 1:16:35 AM, 9/29/2022 7:42:19 AM, 10/2/2022 11:57:04 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM.
IsImprovementStdDev: Marked as improvement because 16.901235050416627 (T) = (0 -249751.21835629412) / Math.Sqrt((246860578.86374336 / (299)) + (220132.9031427259 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.05814109072986637 = (265168.3982581124 - 249751.21835629412) / 265168.3982581124 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.CastingPerf.CheckObjIsInterfaceNo

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 249.4707738095238 < 266.6372752577593.
IsChangePoint: Marked as a change because one of 8/1/2022 6:03:22 PM, 8/5/2022 10:45:33 AM, 8/16/2022 10:54:40 PM, 8/29/2022 8:26:03 AM, 9/6/2022 1:06:44 PM, 9/13/2022 12:23:36 PM, 9/23/2022 1:16:35 AM, 9/29/2022 7:42:19 AM, 10/2/2022 11:57:04 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM.
IsImprovementStdDev: Marked as improvement because 17.132364865886903 (T) = (0 -249719.16110837474) / Math.Sqrt((244478160.14241832 / (299)) + (212716.37923810593 / (35))) is greater than 1.9671350567190904 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (35) - 2, .975) and 0.058617093686547134 = (265268.4252429219 - 249719.16110837474) / 265268.4252429219 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.CastingPerf.CheckObjIsInterfaceYes

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 218.29124503968254 < 266.61740811011896.
IsChangePoint: Marked as a change because one of 8/1/2022 6:03:22 PM, 8/5/2022 10:45:33 AM, 8/16/2022 10:54:40 PM, 8/29/2022 8:26:03 AM, 9/6/2022 1:06:44 PM, 9/13/2022 12:23:36 PM, 9/23/2022 1:16:35 AM, 9/29/2022 7:42:19 AM, 10/2/2022 11:57:04 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM.
IsImprovementStdDev: Marked as improvement because 17.112235173284667 (T) = (0 -218481.51126115513) / Math.Sqrt((975226674.9607886 / (299)) + (138067.36949680833 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.12399045765290691 = (249405.40108247843 - 218481.51126115513) / 249405.40108247843 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.CastingPerf.CheckIsInstAnyIsInterfaceYes

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 218.2899906517094 < 266.61509808550824.
IsChangePoint: Marked as a change because one of 8/1/2022 6:03:22 PM, 8/5/2022 10:45:33 AM, 8/16/2022 10:54:40 PM, 8/29/2022 8:26:03 AM, 9/6/2022 1:06:44 PM, 9/13/2022 12:23:36 PM, 9/23/2022 1:16:35 AM, 9/29/2022 7:42:19 AM, 10/2/2022 11:57:04 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM.
IsImprovementStdDev: Marked as improvement because 16.98217090805701 (T) = (0 -218463.06396402043) / Math.Sqrt((974745865.0913061 / (299)) + (145807.19990104166 / (35))) is greater than 1.9671350567190904 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (35) - 2, .975) and 0.12314838260278974 = (249144.8491735604 - 218463.06396402043) / 249144.8491735604 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS Windows 10.0.18362
Baseline 144a33a01251fce466d7be9b7cafff02f05ae6cc
Compare 039ced57d88703bd88a0490e3209a8e1815bebbc
Diff Diff

Improvements in PerfLabTests.LowLevelPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GenericClassGenericStaticMethod - Duration of single invocation 218.28 μs 156.02 μs 0.71 0.04 False
GenericGenericMethod - Duration of single invocation 218.28 μs 155.98 μs 0.71 0.04 False
IntegerFormatting - Duration of single invocation 2.06 ms 1.85 ms 0.90 0.05 False

graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'PerfLabTests.LowLevelPerf*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

PerfLabTests.LowLevelPerf.GenericClassGenericStaticMethod

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 156.02377665651179 < 207.3695505229001.
IsChangePoint: Marked as a change because one of 8/1/2022 6:03:22 PM, 8/5/2022 10:45:33 AM, 8/16/2022 10:54:40 PM, 8/29/2022 8:26:03 AM, 9/6/2022 1:06:44 PM, 9/13/2022 12:23:36 PM, 9/23/2022 1:16:35 AM, 9/29/2022 7:42:19 AM, 10/2/2022 11:57:04 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM.
IsImprovementStdDev: Marked as improvement because 16.846810227519295 (T) = (0 -156400.43760803778) / Math.Sqrt((971756531.0621381 / (299)) + (513667.3586842295 / (35))) is greater than 1.9671350567190904 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (35) - 2, .975) and 0.16291805126966494 = (186840.0553198669 - 156400.43760803778) / 186840.0553198669 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.LowLevelPerf.GenericGenericMethod

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 155.9809501142422 < 207.3700009403935.
IsChangePoint: Marked as a change because one of 8/1/2022 6:03:22 PM, 8/5/2022 10:45:33 AM, 8/16/2022 10:54:40 PM, 8/29/2022 8:26:03 AM, 9/6/2022 1:06:44 PM, 9/13/2022 12:23:36 PM, 9/23/2022 1:16:35 AM, 9/29/2022 7:42:19 AM, 10/2/2022 11:57:04 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM.
IsImprovementStdDev: Marked as improvement because 17.073195091615133 (T) = (0 -156187.99028197874) / Math.Sqrt((969180591.1718987 / (299)) + (151343.03696516232 / (35))) is greater than 1.9671350567190904 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (35) - 2, .975) and 0.16453290344147764 = (186946.90781402684 - 156187.99028197874) / 186946.90781402684 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.LowLevelPerf.IntegerFormatting

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.8544347685185187 < 1.9974431222098212.
IsChangePoint: Marked as a change because one of 8/1/2022 6:03:22 PM, 8/5/2022 10:45:33 AM, 8/16/2022 10:54:40 PM, 9/13/2022 12:23:36 PM, 9/23/2022 1:16:35 AM, 9/29/2022 7:42:19 AM, 10/2/2022 11:57:04 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM.
IsImprovementStdDev: Marked as improvement because 18.977371048972778 (T) = (0 -1874323.483152182) / Math.Sqrt((5781288630.836337 / (299)) + (1429864834.118542 / (35))) is greater than 1.9671350567190904 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (35) - 2, .975) and 0.07282969295376433 = (2021552.5334535053 - 1874323.483152182) / 2021552.5334535053 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 144a33a01251fce466d7be9b7cafff02f05ae6cc
Compare 039ced57d88703bd88a0490e3209a8e1815bebbc
Diff Diff

Improvements in System.Linq.Tests.Perf_Enumerable

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Contains_ElementNotFound - Duration of single invocation 494.93 ns 467.00 ns 0.94 0.02 False
FirstWithPredicate_LastElementMatches - Duration of single invocation 4.09 μs 2.98 μs 0.73 0.01 False
ElementAt - Duration of single invocation 8.72 ns 7.44 ns 0.85 0.03 False
Count - Duration of single invocation 344.97 ns 315.09 ns 0.91 0.01 False
AnyWithPredicate_LastElementMatches - Duration of single invocation 4.11 μs 2.89 μs 0.70 0.02 False
OrderByDescending - Duration of single invocation 5.86 μs 4.49 μs 0.77 0.02 False
OrderBy - Duration of single invocation 4.50 μs 3.40 μs 0.76 0.02 False

graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Linq.Tests.Perf_Enumerable*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Linq.Tests.Perf_Enumerable.Contains_ElementNotFound(input: IEnumerable)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 466.99786579345044 < 470.231135451296.
IsChangePoint: Marked as a change because one of 9/9/2022 3:12:19 AM, 10/2/2022 11:57:04 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM.
IsImprovementStdDev: Marked as improvement because 50.964507845879204 (T) = (0 -467.96603210821485) / Math.Sqrt((194.07207526910696 / (299)) + (2.5725466185695525 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.08484698667680092 = (511.3527741212235 - 467.96603210821485) / 511.3527741212235 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.FirstWithPredicate_LastElementMatches(input: IOrderedEnumerable)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.9832497465769525 < 3.8886975790499037.
IsChangePoint: Marked as a change because one of 10/2/2022 11:57:04 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM.
IsImprovementStdDev: Marked as improvement because 118.87702447795282 (T) = (0 -2931.627477611201) / Math.Sqrt((6958.58767912735 / (299)) + (3155.6177753654024 / (35))) is greater than 1.9671350567190904 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (35) - 2, .975) and 0.30161574634712174 = (4197.728488690015 - 2931.627477611201) / 4197.728488690015 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.ElementAt(input: IList)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.438552713347215 < 8.220219894370075.
IsChangePoint: Marked as a change because one of 8/1/2022 6:03:22 PM, 8/5/2022 10:45:33 AM, 8/29/2022 8:26:03 AM, 9/6/2022 1:06:44 PM, 9/9/2022 3:12:19 AM, 9/29/2022 7:42:19 AM, 10/2/2022 11:57:04 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM.
IsImprovementStdDev: Marked as improvement because 19.180018885717182 (T) = (0 -7.453788754156468) / Math.Sqrt((0.3036998923261603 / (299)) + (0.003804197070127721 / (35))) is greater than 1.9671350567190904 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (35) - 2, .975) and 0.07943110522583859 = (8.09693744430184 - 7.453788754156468) / 8.09693744430184 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.Count(input: IEnumerable)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 315.09485016002327 < 327.96379977717874.
IsChangePoint: Marked as a change because one of 9/23/2022 1:16:35 AM, 9/29/2022 7:42:19 AM, 10/2/2022 11:57:04 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM.
IsImprovementStdDev: Marked as improvement because 41.53156889602791 (T) = (0 -315.8764076648796) / Math.Sqrt((103.03914395788831 / (299)) + (0.7916135988236538 / (35))) is greater than 1.9671350567190904 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (35) - 2, .975) and 0.07379664286647102 = (341.0443346291394 - 315.8764076648796) / 341.0443346291394 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.AnyWithPredicate_LastElementMatches(input: IOrderedEnumerable)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.8904372263842646 < 3.9036521800049675.
IsChangePoint: Marked as a change because one of 10/2/2022 11:57:04 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM.
IsImprovementStdDev: Marked as improvement because 103.90207744854854 (T) = (0 -2936.283364807647) / Math.Sqrt((6776.5233492581665 / (299)) + (4510.311695917193 / (35))) is greater than 1.9671350567190904 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (35) - 2, .975) and 0.3034211600914213 = (4215.29222046569 - 2936.283364807647) / 4215.29222046569 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.OrderByDescending(input: IEnumerable)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.486256906735258 < 5.581465816539507.
IsChangePoint: Marked as a change because one of 10/2/2022 11:57:04 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM.
IsImprovementStdDev: Marked as improvement because 293.92844157031874 (T) = (0 -4487.398098665333) / Math.Sqrt((2383.5669741223123 / (299)) + (506.34569911903054 / (35))) is greater than 1.9671350567190904 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (35) - 2, .975) and 0.23680151619843262 = (5879.726170724499 - 4487.398098665333) / 5879.726170724499 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.OrderBy(input: IEnumerable)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.3977113742236025 < 4.2711790705957755.
IsChangePoint: Marked as a change because one of 10/2/2022 11:57:04 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM.
IsImprovementStdDev: Marked as improvement because 199.25999430169597 (T) = (0 -3413.43195175151) / Math.Sqrt((3984.625063772097 / (299)) + (615.983690733368 / (35))) is greater than 1.9671350567190904 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (35) - 2, .975) and 0.24507321581737707 = (4521.540397387428 - 3413.43195175151) / 4521.540397387428 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 144a33a01251fce466d7be9b7cafff02f05ae6cc
Compare 039ced57d88703bd88a0490e3209a8e1815bebbc
Diff Diff

Improvements in Benchstone.BenchF.NewtR

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test - Duration of single invocation 118.13 ms 93.04 ms 0.79 0.03 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.BenchF.NewtR*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

Benchstone.BenchF.NewtR.Test

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 93.0408725 < 113.91451543749999.
IsChangePoint: Marked as a change because one of 8/1/2022 6:03:22 PM, 8/5/2022 10:45:33 AM, 8/16/2022 10:54:40 PM, 8/29/2022 8:26:03 AM, 9/6/2022 1:06:44 PM, 9/13/2022 12:23:36 PM, 9/23/2022 1:16:35 AM, 9/29/2022 7:42:19 AM, 10/2/2022 11:57:04 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM.
IsImprovementStdDev: Marked as improvement because 16.840279016217732 (T) = (0 -92338753.07142857) / Math.Sqrt((189671540458917.28 / (299)) + (298094719143.04614 / (35))) is greater than 1.9671350567190904 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (35) - 2, .975) and 0.12757227630786605 = (105841149.43143813 - 92338753.07142857) / 105841149.43143813 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 144a33a01251fce466d7be9b7cafff02f05ae6cc
Compare 039ced57d88703bd88a0490e3209a8e1815bebbc
Diff Diff

Improvements in System.Tests.Perf_String

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IndexOfAny - Duration of single invocation 9.07 ns 7.73 ns 0.85 0.03 False
IndexerCheckPathLength - Duration of single invocation 137.90 ns 116.63 ns 0.85 0.04 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_String*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Tests.Perf_String.IndexOfAny

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.732195341420679 < 8.613670409977892.
IsChangePoint: Marked as a change because one of 8/16/2022 10:54:40 PM, 8/29/2022 8:26:03 AM, 9/6/2022 1:06:44 PM, 9/13/2022 12:23:36 PM, 9/29/2022 7:42:19 AM, 10/2/2022 11:57:04 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM.
IsImprovementStdDev: Marked as improvement because 14.687283619500715 (T) = (0 -7.79427275692708) / Math.Sqrt((0.2426060958259417 / (299)) + (0.008392394935951314 / (35))) is greater than 1.9671350567190904 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (35) - 2, .975) and 0.05757701257584778 = (8.270461205780357 - 7.79427275692708) / 8.270461205780357 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.IndexerCheckPathLength

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 116.62850982395561 < 131.19796968996593.
IsChangePoint: Marked as a change because one of 8/16/2022 10:54:40 PM, 8/29/2022 8:26:03 AM, 9/6/2022 1:06:44 PM, 9/23/2022 1:16:35 AM, 9/29/2022 7:42:19 AM, 10/2/2022 11:57:04 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM.
IsImprovementStdDev: Marked as improvement because 22.093560076858832 (T) = (0 -116.90531869038468) / Math.Sqrt((105.90545733552722 / (299)) + (0.03479035933872328 / (35))) is greater than 1.9671350567190904 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (35) - 2, .975) and 0.10123060313851034 = (130.0726516708502 - 116.90531869038468) / 130.0726516708502 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 144a33a01251fce466d7be9b7cafff02f05ae6cc
Compare 039ced57d88703bd88a0490e3209a8e1815bebbc
Diff Diff

Improvements in Benchstone.BenchI.XposMatrix

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test - Duration of single invocation 18.06 μs 16.98 μs 0.94 0.03 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.BenchI.XposMatrix*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

Benchstone.BenchI.XposMatrix.Test

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.978273000776397 < 17.186845488583867.
IsChangePoint: Marked as a change because one of 8/1/2022 6:03:22 PM, 8/5/2022 10:45:33 AM, 8/29/2022 8:26:03 AM, 9/6/2022 1:06:44 PM, 9/23/2022 1:16:35 AM, 9/29/2022 7:42:19 AM, 10/2/2022 11:57:04 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM.
IsImprovementStdDev: Marked as improvement because 24.37206872303324 (T) = (0 -16821.90075927648) / Math.Sqrt((312912.9179469003 / (299)) + (12442.235596081693 / (35))) is greater than 1.9671350567190904 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (35) - 2, .975) and 0.05145785453808018 = (17734.478999965333 - 16821.90075927648) / 17734.478999965333 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
@performanceautofiler
Copy link
Author

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 144a33a01251fce466d7be9b7cafff02f05ae6cc
Compare 039ced57d88703bd88a0490e3209a8e1815bebbc
Diff Diff

Improvements in System.Collections.Sort<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
LinqQuery - Duration of single invocation 50.67 μs 38.01 μs 0.75 0.00 False
LinqOrderByExtension - Duration of single invocation 51.92 μs 37.81 μs 0.73 0.00 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Sort&lt;Int32&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.Sort<Int32>.LinqQuery(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 38.01333857142857 < 49.383438159340656.
IsChangePoint: Marked as a change because one of 10/2/2022 11:57:04 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM.
IsImprovementStdDev: Marked as improvement because 207.88758052037142 (T) = (0 -37557.85474620617) / Math.Sqrt((293654.63877403014 / (299)) + (129990.41189315598 / (35))) is greater than 1.9671350567190904 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (35) - 2, .975) and 0.27500153056541754 = (51804.04694577781 - 37557.85474620617) / 51804.04694577781 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Sort&lt;Int32&gt;.LinqOrderByExtension(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 37.807022857142854 < 48.78608677197801.
IsChangePoint: Marked as a change because one of 10/2/2022 11:57:04 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM.
IsImprovementStdDev: Marked as improvement because 208.86404214199393 (T) = (0 -37653.13010779696) / Math.Sqrt((240055.42434146686 / (299)) + (133181.23850938765 / (35))) is greater than 1.9671350567190904 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (35) - 2, .975) and 0.27354528011073576 = (51831.35173729279 - 37653.13010779696) / 51831.35173729279 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS Windows 10.0.18362
Baseline 144a33a01251fce466d7be9b7cafff02f05ae6cc
Compare 039ced57d88703bd88a0490e3209a8e1815bebbc
Diff Diff

Improvements in System.Linq.Tests.Perf_OrderBy

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
OrderByValueType - Duration of single invocation 54.18 μs 42.30 μs 0.78 0.11 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Linq.Tests.Perf_OrderBy*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Linq.Tests.Perf_OrderBy.OrderByValueType(NumberOfPeople: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 42.2978704954955 < 51.51879229518093.
IsChangePoint: Marked as a change because one of 10/2/2022 11:57:04 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM.
IsImprovementStdDev: Marked as improvement because 114.93943866941986 (T) = (0 -41751.11171367355) / Math.Sqrt((933271.0776876921 / (299)) + (393203.1021016726 / (35))) is greater than 1.9671350567190904 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (35) - 2, .975) and 0.24803397185234194 = (55522.60361617718 - 41751.11171367355) / 55522.60361617718 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 144a33a01251fce466d7be9b7cafff02f05ae6cc
Compare 039ced57d88703bd88a0490e3209a8e1815bebbc
Diff Diff

Improvements in System.Globalization.Tests.StringEquality

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Compare_Same_Upper - Duration of single invocation 2.27 μs 1.58 μs 0.69 0.04 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Globalization.Tests.StringEquality*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Globalization.Tests.StringEquality.Compare_Same_Upper(Count: 1024, Options: (en-US, OrdinalIgnoreCase))

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.5780418853351819 < 2.16297146932513.
IsChangePoint: Marked as a change because one of 8/17/2022 2:11:12 PM, 9/29/2022 7:42:19 AM, 10/2/2022 11:57:04 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM.
IsImprovementStdDev: Marked as improvement because 17.8976705003882 (T) = (0 -1580.79127757459) / Math.Sqrt((33948.94846680425 / (299)) + (837.7599191626286 / (35))) is greater than 1.9671350567190904 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (35) - 2, .975) and 0.11719349889760246 = (1790.642995492885 - 1580.79127757459) / 1790.642995492885 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 144a33a01251fce466d7be9b7cafff02f05ae6cc
Compare 039ced57d88703bd88a0490e3209a8e1815bebbc
Diff Diff

Improvements in System.Collections.Sort<BigStruct>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
LinqOrderByExtension - Duration of single invocation 66.16 μs 47.55 μs 0.72 0.00 False
LinqQuery - Duration of single invocation 66.21 μs 47.89 μs 0.72 0.00 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Sort&lt;BigStruct&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.Sort<BigStruct>.LinqOrderByExtension(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 47.55399142857143 < 63.120704966666665.
IsChangePoint: Marked as a change because one of 10/2/2022 11:57:04 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM.
IsImprovementStdDev: Marked as improvement because 302.3493573444947 (T) = (0 -47610.44115416012) / Math.Sqrt((171567.66484906542 / (299)) + (109433.28966132936 / (35))) is greater than 1.9671350567190904 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (35) - 2, .975) and 0.27866031910606115 = (66002.80341594081 - 47610.44115416012) / 66002.80341594081 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Sort&lt;BigStruct&gt;.LinqQuery(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 47.88879200000001 < 63.04911075000001.
IsChangePoint: Marked as a change because one of 10/2/2022 11:57:04 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM.
IsImprovementStdDev: Marked as improvement because 258.8539548691379 (T) = (0 -47752.86547315541) / Math.Sqrt((175767.59322455412 / (299)) + (153295.3101386404 / (35))) is greater than 1.9671350567190904 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (35) - 2, .975) and 0.2764434441924386 = (65997.41939986771 - 47752.86547315541) / 65997.41939986771 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 144a33a01251fce466d7be9b7cafff02f05ae6cc
Compare 039ced57d88703bd88a0490e3209a8e1815bebbc
Diff Diff

Improvements in System.Collections.Sort<IntStruct>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
LinqQuery - Duration of single invocation 51.77 μs 36.70 μs 0.71 0.00 False
LinqOrderByExtension - Duration of single invocation 51.78 μs 36.84 μs 0.71 0.00 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Sort&lt;IntStruct&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.Sort<IntStruct>.LinqQuery(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 36.703402857142855 < 49.17015428571428.
IsChangePoint: Marked as a change because one of 10/2/2022 11:57:04 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM.
IsImprovementStdDev: Marked as improvement because 293.0962013203297 (T) = (0 -37414.1115456829) / Math.Sqrt((287668.89689681394 / (299)) + (56622.59609175107 / (35))) is greater than 1.9671350567190904 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (35) - 2, .975) and 0.28464197876186503 = (52301.23998739387 - 37414.1115456829) / 52301.23998739387 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Sort&lt;IntStruct&gt;.LinqOrderByExtension(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 36.835925714285715 < 49.131293230769224.
IsChangePoint: Marked as a change because one of 10/2/2022 11:57:04 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM.
IsImprovementStdDev: Marked as improvement because 320.9735507801943 (T) = (0 -37347.5758266876) / Math.Sqrt((287194.75201055955 / (299)) + (42495.9108451489 / (35))) is greater than 1.9671350567190904 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (35) - 2, .975) and 0.2861116056395068 = (52315.706659083386 - 37347.5758266876) / 52315.706659083386 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 144a33a01251fce466d7be9b7cafff02f05ae6cc
Compare 039ced57d88703bd88a0490e3209a8e1815bebbc
Diff Diff

Improvements in System.Collections.IterateForEach<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ConcurrentStack - Duration of single invocation 4.68 μs 4.02 μs 0.86 0.01 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IterateForEach&lt;String&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.IterateForEach<String>.ConcurrentStack(Size: 512)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.0245324050197695 < 4.4333905629890085.
IsChangePoint: Marked as a change because one of 9/23/2022 1:16:35 AM, 9/29/2022 7:42:19 AM, 10/2/2022 11:57:04 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM.
IsImprovementStdDev: Marked as improvement because 36.44705788543948 (T) = (0 -4030.532296818431) / Math.Sqrt((19092.114635060243 / (299)) + (76.90030817176506 / (35))) is greater than 1.9671350567190904 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (35) - 2, .975) and 0.06846034325343227 = (4326.742578942044 - 4030.532296818431) / 4326.742578942044 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

This was referenced Oct 11, 2022
Open
Open
Open
Open
Open
Open
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
CoreClr Improvement refs/heads/main RunKind=micro untriaged Windows 10.0.18362 x64
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
0 participants