Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix(perf) pin openresty version in perf test #7411

Merged
merged 3 commits into from
Jun 2, 2021
Merged

fix(perf) pin openresty version in perf test #7411

merged 3 commits into from
Jun 2, 2021

Conversation

fffonion
Copy link
Contributor

@fffonion fffonion commented Jun 2, 2021

No description provided.

gszr
gszr reviewed Jun 2, 2021
View reviewed changes
.github/workflows/perf.yml Outdated
@@ -9,6 +9,7 @@ on:

env:
terraform_version: '0.15.3'
openresty_version: '1.19.3.1'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a way for us to pick it from .requirements instead?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added and verified!

@fffonion fffonion changed the title fix(perf) pin openresty version in perf test perf(...) fix(perf) pin openresty version in perf test Jun 2, 2021
@fffonion fffonion force-pushed the fix/perf-pin-openresty branch from 6d9ae91 to a9126d4 Compare June 2, 2021 15:24
@fffonion fffonion changed the title perf(...) fix(perf) pin openresty version in perf test fix(perf) pin openresty version in perf test Jun 2, 2021
@fffonion fffonion changed the title fix(perf) pin openresty version in perf test perf(...) fix(perf) pin openresty version in perf test Jun 2, 2021
@github-actions
Copy link
Contributor

github-actions bot commented Jun 2, 2021

🚀 Performance test result

Test Suite: 01-rps 02-flamegraph (baseline,single_route,simple)

Click to expand 
Please see Github Actions artifacts for flamegraphs.

  ### Result for upstream directly (run 1):
Running 10s test @ http://10.88.145.3:8088/test
  5 threads and 1000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     3.32ms    4.16ms  40.90ms   87.16%
    Req/Sec    78.74k    18.44k  113.13k    59.76%
  3878484 requests in 10.10s, 628.62MB read
Requests/sec: 384032.16
Transfer/sec:     62.24MB
### Result for upstream directly (run 2):
Running 10s test @ http://10.88.145.3:8088/test
  5 threads and 1000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     3.23ms    3.98ms  40.50ms   86.95%
    Req/Sec    79.50k    19.81k  112.71k    57.52%
  3917858 requests in 10.10s, 635.00MB read
Requests/sec: 388065.84
Transfer/sec:     62.90MB
### Result for upstream directly (run 3):
Running 10s test @ http://10.88.145.3:8088/test
  5 threads and 1000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     3.27ms    4.11ms  40.78ms   87.00%
    Req/Sec    79.39k    19.67k  111.69k    57.23%
  3911626 requests in 10.09s, 633.99MB read
Requests/sec: 387588.30
Transfer/sec:     62.82MB
### Combined result for upstream directly:
RPS     Avg: 386562.10
Latency Avg: 3.27ms    Max: 40.90ms
  
### Result for Kong git:881e856819d80d58eb7fdfc329ea333e61dd007c (run 1):
Running 10s test @ http://10.88.145.1:8000/s1-r1
  5 threads and 1000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    12.85ms    8.57ms  75.22ms   64.07%
    Req/Sec    15.95k     1.94k   24.19k    74.20%
  793845 requests in 10.07s, 181.83MB read
Requests/sec:  78834.40
Transfer/sec:     18.06MB
### Result for Kong git:881e856819d80d58eb7fdfc329ea333e61dd007c (run 2):
Running 10s test @ http://10.88.145.1:8000/s1-r1
  5 threads and 1000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    12.72ms    8.02ms  62.69ms   66.16%
    Req/Sec    16.16k     1.47k   21.98k    74.40%
  803963 requests in 10.07s, 184.15MB read
Requests/sec:  79873.51
Transfer/sec:     18.30MB
### Result for Kong git:881e856819d80d58eb7fdfc329ea333e61dd007c (run 3):
Running 10s test @ http://10.88.145.1:8000/s1-r1
  5 threads and 1000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    12.92ms    9.02ms  73.82ms   62.37%
    Req/Sec    16.19k     1.70k   24.99k    77.80%
  805581 requests in 10.07s, 184.54MB read
Requests/sec:  80033.24
Transfer/sec:     18.33MB
### Combined result for Kong git:881e856819d80d58eb7fdfc329ea333e61dd007c:
RPS     Avg: 79580.38
Latency Avg: 12.83ms    Max: 75.22ms
  
### Result for Kong git:881e856819d80d58eb7fdfc329ea333e61dd007c (run 1):
Running 10s test @ http://10.88.145.1:8000/
  5 threads and 1000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    70.19ms   97.74ms 781.44ms   86.14%
    Req/Sec     6.17k     3.66k   15.55k    61.80%
  307062 requests in 10.04s, 70.36MB read
Requests/sec:  30582.43
Transfer/sec:      7.01MB
### Result for Kong git:881e856819d80d58eb7fdfc329ea333e61dd007c (run 2):
Running 10s test @ http://10.88.145.1:8000/
  5 threads and 1000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   114.84ms  254.09ms   2.00s    91.76%
    Req/Sec     4.29k     3.99k   14.23k    47.99%
  213082 requests in 10.05s, 48.88MB read
  Socket errors: connect 0, read 0, write 0, timeout 410
  Non-2xx or 3xx responses: 903
Requests/sec:  21194.98
Transfer/sec:      4.86MB
### Result for Kong git:881e856819d80d58eb7fdfc329ea333e61dd007c (run 3):
Running 10s test @ http://10.88.145.1:8000/
  5 threads and 1000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    75.24ms  235.07ms   1.99s    93.09%
    Req/Sec    10.83k     6.73k   21.18k    71.31%
  533564 requests in 10.07s, 122.25MB read
  Socket errors: connect 0, read 0, write 0, timeout 672
  Non-2xx or 3xx responses: 531
Requests/sec:  52982.42
Transfer/sec:     12.14MB
### Combined result for Kong git:881e856819d80d58eb7fdfc329ea333e61dd007c:
RPS     Avg: 34919.94
Latency Avg: 81.78ms    Max: 2000.00ms
  
### Result for Kong git:881e856819d80d58eb7fdfc329ea333e61dd007c (run 1):
Running 10s test @ http://10.88.145.1:8000/
  5 threads and 1000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    18.42ms   16.22ms 318.91ms   96.23%
    Req/Sec    11.87k     1.71k   14.07k    92.80%
  590415 requests in 10.03s, 135.30MB read
Requests/sec:  58842.88
Transfer/sec:     13.48MB
### Result for Kong git:881e856819d80d58eb7fdfc329ea333e61dd007c (run 2):
Running 10s test @ http://10.88.145.1:8000/
  5 threads and 1000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    16.84ms    8.32ms  89.08ms   73.09%
    Req/Sec    12.12k   823.93    15.08k    71.20%
  603148 requests in 10.04s, 138.19MB read
Requests/sec:  60064.74
Transfer/sec:     13.76MB
### Result for Kong git:881e856819d80d58eb7fdfc329ea333e61dd007c (run 3):
Running 10s test @ http://10.88.145.1:8000/
  5 threads and 1000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    16.88ms    8.91ms  78.39ms   69.64%
    Req/Sec    12.11k   827.88    15.49k    69.40%
  602227 requests in 10.07s, 138.01MB read
Requests/sec:  59805.20
Transfer/sec:     13.71MB
### Combined result for Kong git:881e856819d80d58eb7fdfc329ea333e61dd007c:
RPS     Avg: 59570.94
Latency Avg: 17.37ms    Max: 318.91ms
  
### Result for Kong git:master (run 1):
Running 10s test @ http://10.88.145.1:8000/s1-r1
  5 threads and 1000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    12.15ms    5.54ms  52.26ms   75.52%
    Req/Sec    16.71k     1.60k   20.47k    71.80%
  831435 requests in 10.07s, 190.31MB read
Requests/sec:  82525.43
Transfer/sec:     18.89MB
### Result for Kong git:master (run 2):
Running 10s test @ http://10.88.145.1:8000/s1-r1
  5 threads and 1000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    12.11ms    6.09ms  57.85ms   71.75%
    Req/Sec    16.83k     1.26k   21.61k    70.00%
  837495 requests in 10.07s, 191.73MB read
Requests/sec:  83195.96
Transfer/sec:     19.05MB
### Result for Kong git:master (run 3):
Running 10s test @ http://10.88.145.1:8000/s1-r1
  5 threads and 1000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    12.09ms    5.26ms  55.56ms   72.67%
    Req/Sec    16.82k     0.92k   20.67k    71.00%
  837020 requests in 10.07s, 191.60MB read
Requests/sec:  83153.68
Transfer/sec:     19.03MB
### Combined result for Kong git:master:
RPS     Avg: 82958.36
Latency Avg: 12.12ms    Max: 57.85ms
  
### Result for Kong git:master (run 1):
Running 10s test @ http://10.88.145.1:8000/
  5 threads and 1000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    74.56ms  157.21ms   1.57s    89.59%
    Req/Sec    11.07k     2.33k   18.48k    72.00%
  550687 requests in 10.06s, 126.11MB read
Requests/sec:  54731.02
Transfer/sec:     12.53MB
### Result for Kong git:master (run 2):
Running 10s test @ http://10.88.145.1:8000/
  5 threads and 1000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    74.64ms  133.22ms   1.03s    86.99%
    Req/Sec    10.21k     2.50k   16.40k    69.80%
  507778 requests in 10.05s, 116.29MB read
Requests/sec:  50542.88
Transfer/sec:     11.58MB
### Result for Kong git:master (run 3):
Running 10s test @ http://10.88.145.1:8000/
  5 threads and 1000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    39.54ms   59.56ms 516.25ms   86.63%
    Req/Sec    11.40k     2.39k   19.54k    74.90%
  565395 requests in 10.04s, 129.50MB read
Requests/sec:  56300.31
Transfer/sec:     12.90MB
### Combined result for Kong git:master:
RPS     Avg: 53858.07
Latency Avg: 62.39ms    Max: 1570.00ms
  
### Result for Kong git:master (run 1):
Running 10s test @ http://10.88.145.1:8000/
  5 threads and 1000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    20.77ms   28.30ms 562.12ms   96.25%
    Req/Sec    11.93k     2.32k   33.27k    92.59%
  592407 requests in 10.04s, 135.72MB read
Requests/sec:  58993.50
Transfer/sec:     13.52MB
### Result for Kong git:master (run 2):
Running 10s test @ http://10.88.145.1:8000/
  5 threads and 1000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    16.99ms    9.69ms 105.10ms   77.32%
    Req/Sec    12.33k     0.87k   16.62k    70.80%
  613332 requests in 10.03s, 140.48MB read
Requests/sec:  61127.65
Transfer/sec:     14.00MB
### Result for Kong git:master (run 3):
Running 10s test @ http://10.88.145.1:8000/
  5 threads and 1000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    50.42ms   68.98ms 601.53ms   86.97%
    Req/Sec     7.56k     2.14k   14.46k    69.40%
  376283 requests in 10.05s, 86.23MB read
Requests/sec:  37424.19
Transfer/sec:      8.58MB
### Combined result for Kong git:master:
RPS     Avg: 52515.11
Latency Avg: 26.36ms    Max: 601.53ms
  

Kong error logs are also available in Github Actions artifacts.

Download Artifacts

@fffonion fffonion changed the title perf(...) fix(perf) pin openresty version in perf test fix(perf) pin openresty version in perf test Jun 2, 2021
@fffonion fffonion merged commit 17bfffe into master Jun 2, 2021
@fffonion fffonion deleted the fix/perf-pin-openresty branch June 2, 2021 16:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@gszr gszr gszr approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@fffonion @gszr