Performance Test: Multithreaded vs EventLoop

Server nature

Server takes requests hits downstream service, looks up cache, and hits another downstream microservice.

Sync

Results with multithreaded server handling:

server.tomcat.threads.max=200
server.tomcat.accept-count=100

ab -l -H "x-request-id:prayag-benchmark-sync-test" -T 'application/json' -n 5000 -c 50 -p post-data.json http://127.0.0.1:8080/v1/sync
This is ApacheBench, Version 2.3 <$Revision: 1901567 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 127.0.0.1 (be patient)
Completed 500 requests
Completed 1000 requests

Completed 1500 requests
Completed 2000 requests
Completed 2500 requests
Completed 3000 requests
Completed 3500 requests

Completed 4000 requests


Completed 4500 requests


Completed 5000 requests
Finished 5000 requests


Server Software:        
Server Hostname:        127.0.0.1
Server Port:            8080

Document Path:          v1/sync
Document Length:        Variable

Concurrency Level:      50
Time taken for tests:   337.435 seconds
Complete requests:      5000
Failed requests:        0
Total transferred:      3499560 bytes
Total body sent:        4610000
HTML transferred:       2764560 bytes
Requests per second:    14.82 [#/sec] (mean)
Time per request:       3374.352 [ms] (mean)
Time per request:       67.487 [ms] (mean, across all concurrent requests)
Transfer rate:          10.13 [Kbytes/sec] received
                        13.34 kb/s sent
                        23.47 kb/s total

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.4      0      12
Processing:   409 3337 986.9   3303    7719
Waiting:      409 3337 986.9   3303    7719
Total:        409 3338 987.0   3304    7720

Percentage of the requests served within a certain time (ms)
  50%   3304
  66%   3690
  75%   3946
  80%   4120
  90%   4601
  95%   5024
  98%   5584
  99%   5879
 100%   7720 (longest request)

Async

Results with single threaded server handling:

server.tomcat.threads.max=2
server.tomcat.accept-count=15000

Results:

ab -l -H "x-request-id:prayag-benchmark-sync-test" -T 'application/json' -n 5000 -c 50 -p post-data.json http://127.0.0.1:8080/v1/async
This is ApacheBench, Version 2.3 <$Revision: 1901567 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 127.0.0.1 (be patient)
Completed 500 requests
Completed 1000 requests
Completed 1500 requests
Completed 2000 requests
Completed 2500 requests
Completed 3000 requests
Completed 3500 requests
Completed 4000 requests
Completed 4500 requests
Completed 5000 requests
Finished 5000 requests


Server Software:        
Server Hostname:        127.0.0.1
Server Port:            8080

Document Path:          /v1/async
Document Length:        Variable

Concurrency Level:      50
Time taken for tests:   279.900 seconds
Complete requests:      5000
Failed requests:        0
Total transferred:      3499736 bytes
Total body sent:        4640000
HTML transferred:       2764736 bytes
Requests per second:    17.86 [#/sec] (mean)
Time per request:       2799.004 [ms] (mean)
Time per request:       55.980 [ms] (mean, across all concurrent requests)
Transfer rate:          12.21 [Kbytes/sec] received
                        16.19 kb/s sent
                        28.40 kb/s total

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.2      0       3
Processing:   659 2767 706.7   2675    5316
Waiting:      659 2767 706.7   2675    5316
Total:        662 2767 706.7   2676    5317

Percentage of the requests served within a certain time (ms)
  50%   2502
  66%   3005
  75%   3349
  80%   3568
  90%   3968
  95%   4272
  98%   4697
  99%   4932
 100%   5592 (longest request)