Regressions in System.Collections.Tests.Perf_PriorityQueue<Int32, Int32>

[DrewScoggins]

Run Information

Architecture	x64
OS	Windows 10.0.18362
Baseline	5d6c8aa343bcd62cfb2f2a4b2364e39623ebf34d
Compare	04ff9bd1c1cdd0046af652c74821e73668207cac

Regressions in System.Collections.Tests.Perf_PriorityQueue<Int32, Int32>

Benchmark	Baseline	Test	Test/Base	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
HeapSort	3.07 μs	3.22 μs	1.05

Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f netcoreapp5.0 --filter 'System.Collections.Tests.Perf_PriorityQueue<Int32, Int32>*'

.

Payloads

Baseline
Compare

Histogram

System.Collections.Tests.Perf_PriorityQueue<Int32, Int32>.HeapSort(Size: 100)

[3039.342 ; 3151.722) | @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
[3151.722 ; 3249.705) | @@@@@
[3249.705 ; 3380.575) | 
[3380.575 ; 3558.514) | @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
[3558.514 ; 3638.772) | @@@@@

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[ghost]

Tagging subscribers to this area: @eiriktsarpalis
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Run Information

Architecture	x64
OS	Windows 10.0.18362
Baseline	5d6c8aa343bcd62cfb2f2a4b2364e39623ebf34d
Compare	04ff9bd1c1cdd0046af652c74821e73668207cac

Regressions in System.Collections.Tests.Perf_PriorityQueue<Int32, Int32>

Benchmark	Baseline	Test	Test/Base	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
HeapSort	3.07 μs	3.22 μs	1.05

Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f netcoreapp5.0 --filter 'System.Collections.Tests.Perf_PriorityQueue<Int32, Int32>*'

.

Payloads

Baseline
Compare

Histogram

System.Collections.Tests.Perf_PriorityQueue<Int32, Int32>.HeapSort(Size: 100)

[3039.342 ; 3151.722) | @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
[3151.722 ; 3249.705) | @@@@@
[3249.705 ; 3380.575) | 
[3380.575 ; 3558.514) | @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
[3558.514 ; 3638.772) | @@@@@

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Author:	DrewScoggins
Assignees:	-
Labels:	area-System.Collections, untriaged
Milestone:	-

[DrewScoggins]

Seems related to #48539

[eiriktsarpalis]

Interesting. I could not reproduce the regression locally, if anything the compare commit performed better on my machine:

baseline: 5d6c8aa

Method	Size	Mean	Error	StdDev	Median	Min	Max	Gen 0	Gen 1	Gen 2	Allocated
HeapSort	10	203.3 ns	11.25 ns	12.96 ns	204.0 ns	167.8 ns	227.5 ns	0.0096	-	-	104 B
HeapSort	100	3,074.7 ns	279.93 ns	322.36 ns	3,128.8 ns	2,631.7 ns	3,611.9 ns	0.0805	-	-	824 B
HeapSort	1000	87,159.9 ns	4,426.74 ns	5,097.84 ns	86,941.3 ns	80,013.2 ns	98,113.3 ns	0.5656	-	-	8,024 B

compare: 04ff9bd

Method	Size	Mean	Error	StdDev	Median	Min	Max	Gen 0	Gen 1	Gen 2	Allocated
HeapSort	10	188.2 ns	6.49 ns	7.22 ns	185.6 ns	179.0 ns	202.8 ns	0.0101	-	-	104 B
HeapSort	100	2,859.8 ns	73.87 ns	85.07 ns	2,894.7 ns	2,684.3 ns	3,000.9 ns	0.0780	-	-	824 B
HeapSort	1000	78,737.3 ns	1,775.20 ns	2,044.32 ns	79,492.3 ns	74,640.1 ns	81,288.0 ns	0.5841	-	-	8,024 B

That being said, when re-auditing #48539 I noticed a performance bug in the EnqueueRange method that predates that particular change. I'll follow up with a PR that addresses it.

[eiriktsarpalis]

Regression should be addressed by #50065.