Running application code Inline reduces performance of JSON benchmark

[tmds]

See #25 (comment)

I'm surprised inlining the application code doesn't get us at the same level of the other benchmarks with t=ProcessorCount.

What is the root cause?

[antonfirsov]

@tmds @adamsitnik I collected traces + speedscope json for this issue into the InliningTraces folder on my OneDrive.

EDIT: updated the link

[tmds]

Thanks for running these benchmarks. I hope they tell us what this weirdness is with i=true.

[tmds]

@adamsitnik I had a look at the trace file, this is what I noticed:

A lot of time is spent in CLRLifoSemaphore::Wait:

This is called by ThreadPool threads. Zooming in on a thread:

What do you make of this?

[tmds]

Based on dotnet/runtime#33669 (comment), it may be interesting to see what happens if we set COMPlus_ThreadPool_UnfairSemaphoreSpinLimit=0.

[tmds]

I believe these traces are from this benchmark:

--arg "-e=epoll" --arg "-t=7" --arg "-s=false" --arg "-r=false" --arg "-c=false" --arg "-a=true" --arg "-w=false" --arg "-i=true" --connections 256

@antonfirsov @adamsitnik i controls the input scheduler. I'd like to see what happens if we also change the output pipe scheduler.
Can you run these benchmarks also (without setting COMPlus_ThreadPool_UnfairSemaphoreSpinLimit=0)?

--arg "-e=epoll" --arg "-t=7" --arg "-s=false" --arg "-r=false" --arg "-c=false" --arg "-a=true" --arg "-w=false" --arg "-i=true" --arg "-o=ioqueue" --connections 256
--arg "-e=epoll" --arg "-t=7" --arg "-s=false" --arg "-r=false" --arg "-c=false" --arg "-a=true" --arg "-w=false" --arg "-i=true" --arg "-o=iothread" --connections 256
--arg "-e=epoll" --arg "-t=7" --arg "-s=false" --arg "-r=false" --arg "-c=false" --arg "-a=true" --arg "-w=false" --arg "-i=true" --arg "-o=inline" --connections 256

Note: the first is the default value, and matches the benchmark we ran already.

[antonfirsov]

@tmds looks like it makes big difference:

	RPS	CPU (%)	Avg. Latency (ms)
-o=ioqueue	362,086	85	0.97
-o=iothread	494,267	58	0.89
-o=inline	495,721	59	0.59

[adamsitnik]

A lot of time is spent in CLRLifoSemaphore::Wait:

Most probably these are the thread pool threads that are waiting for some work to execute.

[antonfirsov]

Even without COMPlus_ThreadPool_UnfairSemaphoreSpinLimit=0 there is still significant gain from -o=iothread and -o=inline :

	RPS	CPU (%)	Avg. Latency (ms)
-o=ioqueue	332,801	100	1.93
-o=iothread	487,377	58	0.96
-o=inline	475,064	59	0.88

[tmds]

-o=ioqueue | 332,801

When we run application code inline, the IoQueue is challenging the ThreadPool.

-o=iothread | 487,377

This is the first benchmark where we see batching sends improving performance!

[antonfirsov]

Based on dotnet/runtime#33669 (comment), I also tried COMPlus_HillClimbing_Disable=1 in all configurations, but it did not seem to have any effect.

[tmds]

@antonfirsov can you run these accross a range of t without COMPlus_ThreadPool_UnfairSemaphoreSpinLimit=0?

--arg "-e=epoll" --arg "-t=x" --arg "-s=false" --arg "-r=false" --arg "-c=false" --arg "-a=true" --arg "-w=false" --arg "-i=true" --arg "-o=ioqueue" --connections 256
--arg "-e=epoll" --arg "-t=x" --arg "-s=false" --arg "-r=false" --arg "-c=false" --arg "-a=true" --arg "-w=false" --arg "-i=true" --arg "-o=iothread" --connections 256
--arg "-e=epoll" --arg "-t=x" --arg "-s=false" --arg "-r=false" --arg "-c=false" --arg "-a=true" --arg "-w=false" --arg "-i=true" --arg "-o=inline" --connections 256

[antonfirsov]

Done:

t	ioqueue	iothread	inline
4	261,520	354,702	368,774
5	313,798	445,904	443,246
6	333,697	497,961	493,417
7	338,994	480,170	481,411
8	332,807	479,579	479,094
9	342,327	473,435	467,595
10	351,746	470,477	467,275

Looks like we peak at ht/2 again

[tmds]

@antonfirsov can you perform this benchmark also with ThreadPool as output scheduler (added in #71)?

--arg "-e=epoll" --arg "-t=x" --arg "-s=false" --arg "-r=false" --arg "-c=false" --arg "-a=true" --arg "-w=false" --arg "-i=true" --arg "-o=threadpool" --connections 256

[antonfirsov]

It's much worse unfortunately:

t	RPS	CPU	latency
3	222580	83	1.16
4	270670	97	1.01
5	291501	99	1.22
6	323535	100	1.57
7	320456	99	2.04
8	333158	100	2.3

[tmds]

It's much worse unfortunately:

That's expected. ioqueue gets used because it improves performance compared to threadpool.

Though it varies based on t:

t	threadpool	ioqueue	ioqueue vs threadpool (%)
4	270670	261520	-3.4
5	291501	313798	7.6
6	323535	333697	3.1
7	320456	338994	5.8
8	333158	332807	-0.1

[tmds]

We've observed in #76 that when using iothread and inline have Max CPU 50%.

Looking at trace from:

--arg "-e=epoll" --arg "-t=6" --arg "-s=false" --arg "-r=false" --arg "-c=false" --arg "-a=true" --arg "-w=false" --arg "-i=true" --arg "-o=inline" --connections 256

When we look at the whole application: we see system is loaded 50%, and load is spread across the epoll threads:

And if we look at a single epoll thread, it loads a full core:

@antonfirsov can you create a trace for t=10?

[antonfirsov]

RequestsPerSecond:           477,527
Max CPU (%):                 83

See: inline-t10.03-24-15-10-12.RPS-478K.trace.zip in OneDrive root.

[tmds]

See: inline-t10.03-24-15-10-12.RPS-478K.trace.zip in OneDrive root.

These traces look very similar to t=6.

What happen when we increase parallelism for requests? How does that affect Max CPU and RPS?

--arg "-e=epoll" --arg "-t=x" --arg "-s=false" --arg "-r=false" --arg "-c=false" --arg "-a=true" --arg "-w=false" --arg "-i=true" --arg "-o=inline" --connections 512

@antonfirsov this sets connections to 512. Can you run it for a range of t? No need for traces. RPS and Max CPU is enough.

[antonfirsov]

--connections 512 e=epoll t=4..12 s=false r=false c=false a=true w=false i=true o=inline

t	RPS	CPU	latency
4	376621	35	1.36
5	446699	43	1.14
6	517477	50	1.07
7	511184	59	1.07
8	508134	67	1.15
9	513596	77	1.26
10	512927	84	1.27
11	508421	93	1.12
12	513530	100	1.16

[tmds]

@antonfirsov as we've discussed, we'll run the same benchmark on Citrine and see at what nr of t, RPS flatlines.

[tmds]

@antonfirsov when you run these benchmarks. Probably we'll get a similar graph as the one above. It would be interesting to have traces for lowest t and highest t that have the max performance. In the previous graph these are t=6, and t=12 which have about 500k RPS.

[tmds]

Some observations from the previous benchmark worth calling out:

• At t=12, we max CPU 100%. I guess also with 256 connections this is probably the case.
• At t=6, we have higher +4.9% RPS than 256 connections. So these threads have some room to deal with additional requests.

[antonfirsov]

Although there is still a slowdown at ht/2, it seems to scale much better on Citrine:
e=epoll s=false r=false c=false a=true w=false i=true o=inline t=1,7,14,21,28 (with --connections 512):

t	RPS	CPU	latency
1	84513	4	6.03
7	494744	25	1.04
14	832746	50	0.63
21	947215	75	0.55
28	1022530	100	0.51

There are trace files for t=14 and t=28 in the folder CitrineQuick.

Update
No trace for t=28 for now, acquisition fails for some reason.

[adamsitnik]

--path "/json" --arg "-e=epoll" --arg "-t=4" --arg "-s=false" --arg "-r=false" --arg "-c=false" --arg "-a=true" --arg "-w=false" --arg "-i=true" --arg "-o=inline" --connections 512

RequestsPerSecond:           283,945
Max CPU (%):                 15
WorkingSet (MB):             483
Avg. Latency (ms):           1.8
Startup (ms):                1
First Request (ms):          84.02
Latency (ms):                0.23
Total Requests:              4,287,496
Duration: (ms)               15,100
Socket Errors:               0
Bad Responses:               0
Build Time (ms):             4,501
Published Size (KB):         101,009
SDK:                         5.0.100-preview.3.20174.1
Runtime:                     5.0.0-preview.3.20173.5
ASP.NET Core:                5.0.0-preview.3.20173.4

--path "/json" --arg "-e=epoll" --arg "-t=5" --arg "-s=false" --arg "-r=false" --arg "-c=false" --arg "-a=true" --arg "-w=false" --arg "-i=true" --arg "-o=inline" --connections 512

RequestsPerSecond:           348,337
Max CPU (%):                 18
WorkingSet (MB):             512
Avg. Latency (ms):           1.47
Startup (ms):                1
First Request (ms):          85.25
Latency (ms):                0.1
Total Requests:              5,259,569
Duration: (ms)               15,100
Socket Errors:               0
Bad Responses:               0
Build Time (ms):             4,001
Published Size (KB):         101,009
SDK:                         5.0.100-preview.3.20174.1
Runtime:                     5.0.0-preview.3.20173.5
ASP.NET Core:                5.0.0-preview.3.20173.4

--path "/json" --arg "-e=epoll" --arg "-t=6" --arg "-s=false" --arg "-r=false" --arg "-c=false" --arg "-a=true" --arg "-w=false" --arg "-i=true" --arg "-o=inline" --connections 512

RequestsPerSecond:           395,479
Max CPU (%):                 22
WorkingSet (MB):             460
Avg. Latency (ms):           1.29
Startup (ms):                1
First Request (ms):          85.84
Latency (ms):                0.1
Total Requests:              5,971,437
Duration: (ms)               15,100
Socket Errors:               0
Bad Responses:               0
Build Time (ms):             4,001
Published Size (KB):         101,009
SDK:                         5.0.100-preview.3.20174.1
Runtime:                     5.0.0-preview.3.20173.5
ASP.NET Core:                5.0.0-preview.3.20173.4

--path "/json" --arg "-e=epoll" --arg "-t=7" --arg "-s=false" --arg "-r=false" --arg "-c=false" --arg "-a=true" --arg "-w=false" --arg "-i=true" --arg "-o=inline" --connections 512

RequestsPerSecond:           475,568
Max CPU (%):                 25
WorkingSet (MB):             477
Avg. Latency (ms):           1.08
Startup (ms):                1
First Request (ms):          82.89
Latency (ms):                0.1
Total Requests:              7,180,654
Duration: (ms)               15,100
Socket Errors:               0
Bad Responses:               0
Build Time (ms):             4,001
Published Size (KB):         101,009
SDK:                         5.0.100-preview.3.20174.1
Runtime:                     5.0.0-preview.3.20173.5
ASP.NET Core:                5.0.0-preview.3.20173.4

--path "/json" --arg "-e=epoll" --arg "-t=8" --arg "-s=false" --arg "-r=false" --arg "-c=false" --arg "-a=true" --arg "-w=false" --arg "-i=true" --arg "-o=inline" --connections 512

RequestsPerSecond:           521,995
Max CPU (%):                 29
WorkingSet (MB):             484
Avg. Latency (ms):           0.98
Startup (ms):                1
First Request (ms):          85.05
Latency (ms):                0.09
Total Requests:              7,881,961
Duration: (ms)               15,100
Socket Errors:               0
Bad Responses:               0
Build Time (ms):             6,502
Published Size (KB):         101,009
SDK:                         5.0.100-preview.3.20174.1
Runtime:                     5.0.0-preview.3.20173.5
ASP.NET Core:                5.0.0-preview.3.20173.4

--path "/json" --arg "-e=epoll" --arg "-t=9" --arg "-s=false" --arg "-r=false" --arg "-c=false" --arg "-a=true" --arg "-w=false" --arg "-i=true" --arg "-o=inline" --connections 512

RequestsPerSecond:           582,889
Max CPU (%):                 33
WorkingSet (MB):             515
Avg. Latency (ms):           0.88
Startup (ms):                1
First Request (ms):          82.26
Latency (ms):                0.1
Total Requests:              8,801,339
Duration: (ms)               15,100
Socket Errors:               0
Bad Responses:               0
Build Time (ms):             4,001
Published Size (KB):         101,009
SDK:                         5.0.100-preview.3.20174.1
Runtime:                     5.0.0-preview.3.20173.5
ASP.NET Core:                5.0.0-preview.3.20173.4

--path "/json" --arg "-e=epoll" --arg "-t=10" --arg "-s=false" --arg "-r=false" --arg "-c=false" --arg "-a=true" --arg "-w=false" --arg "-i=true" --arg "-o=inline" --connections 512

RequestsPerSecond:           623,679
Max CPU (%):                 36
WorkingSet (MB):             478
Avg. Latency (ms):           0.83
Startup (ms):                1
First Request (ms):          83.77
Latency (ms):                0.11
Total Requests:              9,417,329
Duration: (ms)               15,100
Socket Errors:               0
Bad Responses:               0
Build Time (ms):             4,501
Published Size (KB):         101,009
SDK:                         5.0.100-preview.3.20174.1
Runtime:                     5.0.0-preview.3.20173.5
ASP.NET Core:                5.0.0-preview.3.20173.4

--path "/json" --arg "-e=epoll" --arg "-t=11" --arg "-s=false" --arg "-r=false" --arg "-c=false" --arg "-a=true" --arg "-w=false" --arg "-i=true" --arg "-o=inline" --connections 512

RequestsPerSecond:           684,301
Max CPU (%):                 40
WorkingSet (MB):             425
Avg. Latency (ms):           0.75
Startup (ms):                1
First Request (ms):          85.77
Latency (ms):                0.09
Total Requests:              10,332,687
Duration: (ms)               15,100
Socket Errors:               0
Bad Responses:               0
Build Time (ms):             4,501
Published Size (KB):         101,009
SDK:                         5.0.100-preview.3.20174.1
Runtime:                     5.0.0-preview.3.20173.5
ASP.NET Core:                5.0.0-preview.3.20173.4

--path "/json" --arg "-e=epoll" --arg "-t=12" --arg "-s=false" --arg "-r=false" --arg "-c=false" --arg "-a=true" --arg "-w=false" --arg "-i=true" --arg "-o=inline" --connections 512

RequestsPerSecond:           723,484
Max CPU (%):                 43
WorkingSet (MB):             460
Avg. Latency (ms):           0.71
Startup (ms):                1
First Request (ms):          85.1
Latency (ms):                0.11
Total Requests:              10,924,307
Duration: (ms)               15,100
Socket Errors:               0
Bad Responses:               0
Build Time (ms):             4,501
Published Size (KB):         101,009
SDK:                         5.0.100-preview.3.20174.1
Runtime:                     5.0.0-preview.3.20173.5
ASP.NET Core:                5.0.0-preview.3.20173.4

[adamsitnik]

--path "/json" --arg "-e=epoll" --arg "-t=13" --arg "-s=false" --arg "-r=false" --arg "-c=false" --arg "-a=true" --arg "-w=false" --arg "-i=true" --arg "-o=inline" --connections 512

RequestsPerSecond:           779,569
Max CPU (%):                 47
WorkingSet (MB):             451
Avg. Latency (ms):           0.65
Startup (ms):                1
First Request (ms):          84.31
Latency (ms):                0.1
Total Requests:              11,771,247
Duration: (ms)               15,100
Socket Errors:               0
Bad Responses:               0
Build Time (ms):             4,502
Published Size (KB):         101,009
SDK:                         5.0.100-preview.3.20174.1
Runtime:                     5.0.0-preview.3.20173.5
ASP.NET Core:                5.0.0-preview.3.20173.4

--path "/json" --arg "-e=epoll" --arg "-t=14" --arg "-s=false" --arg "-r=false" --arg "-c=false" --arg "-a=true" --arg "-w=false" --arg "-i=true" --arg "-o=inline" --connections 512

RequestsPerSecond:           785,280
Max CPU (%):                 50
WorkingSet (MB):             513
Avg. Latency (ms):           0.65
Startup (ms):                1
First Request (ms):          85.58
Latency (ms):                0.13
Total Requests:              11,856,862
Duration: (ms)               15,100
Socket Errors:               0
Bad Responses:               0
Build Time (ms):             6,002
Published Size (KB):         101,009
SDK:                         5.0.100-preview.3.20174.1
Runtime:                     5.0.0-preview.3.20173.5
ASP.NET Core:                5.0.0-preview.3.20173.4

--path "/json" --arg "-e=epoll" --arg "-t=15" --arg "-s=false" --arg "-r=false" --arg "-c=false" --arg "-a=true" --arg "-w=false" --arg "-i=true" --arg "-o=inline" --connections 512

RequestsPerSecond:           833,490
Max CPU (%):                 55
WorkingSet (MB):             467
Avg. Latency (ms):           0.62
Startup (ms):                7
First Request (ms):          83.42
Latency (ms):                0.09
Total Requests:              12,585,110
Duration: (ms)               15,100
Socket Errors:               0
Bad Responses:               0
Build Time (ms):             6,002
Published Size (KB):         101,009
SDK:                         5.0.100-preview.3.20174.1
Runtime:                     5.0.0-preview.3.20173.5
ASP.NET Core:                5.0.0-preview.3.20173.4

--path "/json" --arg "-e=epoll" --arg "-t=16" --arg "-s=false" --arg "-r=false" --arg "-c=false" --arg "-a=true" --arg "-w=false" --arg "-i=true" --arg "-o=inline" --connections 512

RequestsPerSecond:           874,434
Max CPU (%):                 57
WorkingSet (MB):             458
Avg. Latency (ms):           0.59
Startup (ms):                1
First Request (ms):          82.53
Latency (ms):                0.09
Total Requests:              13,203,234
Duration: (ms)               15,100
Socket Errors:               0
Bad Responses:               0
Build Time (ms):             4,501
Published Size (KB):         101,009
SDK:                         5.0.100-preview.3.20174.1
Runtime:                     5.0.0-preview.3.20173.5
ASP.NET Core:                5.0.0-preview.3.20173.4

--path "/json" --arg "-e=epoll" --arg "-t=17" --arg "-s=false" --arg "-r=false" --arg "-c=false" --arg "-a=true" --arg "-w=false" --arg "-i=true" --arg "-o=inline" --connections 512

RequestsPerSecond:           879,827
Max CPU (%):                 62
WorkingSet (MB):             475
Avg. Latency (ms):           0.58
Startup (ms):                7
First Request (ms):          83.33
Latency (ms):                0.09
Total Requests:              13,284,901
Duration: (ms)               15,100
Socket Errors:               0
Bad Responses:               0
Build Time (ms):             6,002
Published Size (KB):         101,009
SDK:                         5.0.100-preview.3.20174.1
Runtime:                     5.0.0-preview.3.20173.5
ASP.NET Core:                5.0.0-preview.3.20173.4

--path "/json" --arg "-e=epoll" --arg "-t=18" --arg "-s=false" --arg "-r=false" --arg "-c=false" --arg "-a=true" --arg "-w=false" --arg "-i=true" --arg "-o=inline" --connections 512

RequestsPerSecond:           902,174
Max CPU (%):                 64
WorkingSet (MB):             471
Avg. Latency (ms):           0.57
Startup (ms):                7
First Request (ms):          84.78
Latency (ms):                0.09
Total Requests:              13,622,130
Duration: (ms)               15,100
Socket Errors:               0
Bad Responses:               0
Build Time (ms):             6,003
Published Size (KB):         101,009
SDK:                         5.0.100-preview.3.20174.1
Runtime:                     5.0.0-preview.3.20173.5
ASP.NET Core:                5.0.0-preview.3.20173.4

--path "/json" --arg "-e=epoll" --arg "-t=19" --arg "-s=false" --arg "-r=false" --arg "-c=false" --arg "-a=true" --arg "-w=false" --arg "-i=true" --arg "-o=inline" --connections 512

RequestsPerSecond:           905,851
Max CPU (%):                 68
WorkingSet (MB):             465
Avg. Latency (ms):           0.56
Startup (ms):                7
First Request (ms):          84.34
Latency (ms):                0.11
Total Requests:              13,677,376
Duration: (ms)               15,100
Socket Errors:               0
Bad Responses:               0
Build Time (ms):             6,002
Published Size (KB):         101,009
SDK:                         5.0.100-preview.3.20174.1
Runtime:                     5.0.0-preview.3.20173.5
ASP.NET Core:                5.0.0-preview.3.20173.4

--path "/json" --arg "-e=epoll" --arg "-t=20" --arg "-s=false" --arg "-r=false" --arg "-c=false" --arg "-a=true" --arg "-w=false" --arg "-i=true" --arg "-o=inline" --connections 512

RequestsPerSecond:           913,109
Max CPU (%):                 72
WorkingSet (MB):             464
Avg. Latency (ms):           0.56
Startup (ms):                7
First Request (ms):          85.11
Latency (ms):                0.08
Total Requests:              13,787,286
Duration: (ms)               15,100
Socket Errors:               0
Bad Responses:               0
Build Time (ms):             6,502
Published Size (KB):         101,009
SDK:                         5.0.100-preview.3.20174.1
Runtime:                     5.0.0-preview.3.20173.5
ASP.NET Core:                5.0.0-preview.3.20173.4

--path "/json" --arg "-e=epoll" --arg "-t=21" --arg "-s=false" --arg "-r=false" --arg "-c=false" --arg "-a=true" --arg "-w=false" --arg "-i=true" --arg "-o=inline" --connections 512

RequestsPerSecond:           962,565
Max CPU (%):                 76
WorkingSet (MB):             433
Avg. Latency (ms):           0.53
Startup (ms):                14
First Request (ms):          89.09
Latency (ms):                0.13
Total Requests:              14,534,261
Duration: (ms)               15,100
Socket Errors:               0
Bad Responses:               0
Build Time (ms):             7,503
Published Size (KB):         101,009
SDK:                         5.0.100-preview.3.20174.1
Runtime:                     5.0.0-preview.3.20173.5
ASP.NET Core:                5.0.0-preview.3.20173.4

--path "/json" --arg "-e=epoll" --arg "-t=22" --arg "-s=false" --arg "-r=false" --arg "-c=false" --arg "-a=true" --arg "-w=false" --arg "-i=true" --arg "-o=inline" --connections 512

RequestsPerSecond:           948,799
Max CPU (%):                 79
WorkingSet (MB):             419
Avg. Latency (ms):           0.54
Startup (ms):                14
First Request (ms):          80.2
Latency (ms):                0.11
Total Requests:              14,326,540
Duration: (ms)               15,100
Socket Errors:               0
Bad Responses:               0
Build Time (ms):             6,002
Published Size (KB):         101,009
SDK:                         5.0.100-preview.3.20174.1
Runtime:                     5.0.0-preview.3.20173.5
ASP.NET Core:                5.0.0-preview.3.20173.4

--path "/json" --arg "-e=epoll" --arg "-t=23" --arg "-s=false" --arg "-r=false" --arg "-c=false" --arg "-a=true" --arg "-w=false" --arg "-i=true" --arg "-o=inline" --connections 512

RequestsPerSecond:           957,004
Max CPU (%):                 82
WorkingSet (MB):             454
Avg. Latency (ms):           0.53
Startup (ms):                14
First Request (ms):          84.58
Latency (ms):                0.09
Total Requests:              14,450,254
Duration: (ms)               15,100
Socket Errors:               0
Bad Responses:               0
Build Time (ms):             6,002
Published Size (KB):         101,009
SDK:                         5.0.100-preview.3.20174.1
Runtime:                     5.0.0-preview.3.20173.5
ASP.NET Core:                5.0.0-preview.3.20173.4

--path "/json" --arg "-e=epoll" --arg "-t=24" --arg "-s=false" --arg "-r=false" --arg "-c=false" --arg "-a=true" --arg "-w=false" --arg "-i=true" --arg "-o=inline" --connections 512

RequestsPerSecond:           976,965
Max CPU (%):                 87
WorkingSet (MB):             435
Avg. Latency (ms):           0.52
Startup (ms):                16
First Request (ms):          82.72
Latency (ms):                0.1
Total Requests:              14,751,503
Duration: (ms)               15,100
Socket Errors:               0
Bad Responses:               0
Build Time (ms):             6,002
Published Size (KB):         101,009
SDK:                         5.0.100-preview.3.20174.1
Runtime:                     5.0.0-preview.3.20173.5
ASP.NET Core:                5.0.0-preview.3.20173.4

--path "/json" --arg "-e=epoll" --arg "-t=25" --arg "-s=false" --arg "-r=false" --arg "-c=false" --arg "-a=true" --arg "-w=false" --arg "-i=true" --arg "-o=inline" --connections 512

RequestsPerSecond:           1,001,570
Max CPU (%):                 89
WorkingSet (MB):             418
Avg. Latency (ms):           0.52
Startup (ms):                14
First Request (ms):          84.13
Latency (ms):                0.09
Total Requests:              15,122,776
Duration: (ms)               15,100
Socket Errors:               0
Bad Responses:               0
Build Time (ms):             6,002
Published Size (KB):         101,009
SDK:                         5.0.100-preview.3.20174.1
Runtime:                     5.0.0-preview.3.20173.5
ASP.NET Core:                5.0.0-preview.3.20173.4

[adamsitnik]

--path "/json" --arg "-e=epoll" --arg "-t=26" --arg "-s=false" --arg "-r=false" --arg "-c=false" --arg "-a=true" --arg "-w=false" --arg "-i=true" --arg "-o=inline" --connections 512

RequestsPerSecond:           1,027,891
Max CPU (%):                 93
WorkingSet (MB):             484
Avg. Latency (ms):           0.5
Startup (ms):                12
First Request (ms):          58.26
Latency (ms):                0.1
Total Requests:              15,520,569
Duration: (ms)               15,100
Socket Errors:               0
Bad Responses:               0
Build Time (ms):             6,002
Published Size (KB):         101,009
SDK:                         5.0.100-preview.3.20174.1
Runtime:                     5.0.0-preview.3.20173.5
ASP.NET Core:                5.0.0-preview.3.20173.4

--path "/json" --arg "-e=epoll" --arg "-t=27" --arg "-s=false" --arg "-r=false" --arg "-c=false" --arg "-a=true" --arg "-w=false" --arg "-i=true" --arg "-o=inline" --connections 512

RequestsPerSecond:           1,042,081
Max CPU (%):                 97
WorkingSet (MB):             432
Avg. Latency (ms):           0.49
Startup (ms):                1
First Request (ms):          56.26
Latency (ms):                0.09
Total Requests:              15,733,839
Duration: (ms)               15,100
Socket Errors:               0
Bad Responses:               0
Build Time (ms):             5,502
Published Size (KB):         101,009
SDK:                         5.0.100-preview.3.20174.1
Runtime:                     5.0.0-preview.3.20173.5
ASP.NET Core:                5.0.0-preview.3.20173.4

--path "/json" --arg "-e=epoll" --arg "-t=28" --arg "-s=false" --arg "-r=false" --arg "-c=false" --arg "-a=true" --arg "-w=false" --arg "-i=true" --arg "-o=inline" --connections 512

RequestsPerSecond:           1,049,023
Max CPU (%):                 100
WorkingSet (MB):             473
Avg. Latency (ms):           0.49
Startup (ms):                1
First Request (ms):          55.6
Latency (ms):                0.09
Total Requests:              15,839,414
Duration: (ms)               15,100
Socket Errors:               0
Bad Responses:               0
Build Time (ms):             4,001
Published Size (KB):         101,009
SDK:                         5.0.100-preview.3.20174.1
Runtime:                     5.0.0-preview.3.20173.5
ASP.NET Core:                5.0.0-preview.3.20173.4

--path "/json" --arg "-e=epoll" --arg "-t=28" --arg "-s=false" --arg "-r=false" --arg "-c=false" --arg "-a=false" --arg "-w=false" --arg "-i=true" --arg "-o=inline" --connections 512

RequestsPerSecond:           893,776
Max CPU (%):                 100
WorkingSet (MB):             461
Avg. Latency (ms):           0.57
Startup (ms):                1
First Request (ms):          55.64
Latency (ms):                0.13
Total Requests:              13,495,451
Duration: (ms)               15,100
Socket Errors:               0
Bad Responses:               0
Build Time (ms):             4,001
Published Size (KB):         101,009
SDK:                         5.0.100-preview.3.20174.1
Runtime:                     5.0.0-preview.3.20173.5
ASP.NET Core:                5.0.0-preview.3.20173.4

[tmds]

"-t=28", with AIO

RequestsPerSecond: 1,049,023

"-t=28", without AIO

RequestsPerSecond: 893,776

+17%, that's a nice difference!

[tmds]

Closing. Benchmarks (see #78) will give us an overview of how this behaves with different schedulers, and COMPlus_ThreadPool_UnfairSemaphoreSpinLimit=0.