Skip to content

Latest commit

 

History

History
281 lines (236 loc) · 19.7 KB

README.md

File metadata and controls

281 lines (236 loc) · 19.7 KB

Which is the fastest?

Build Status Join the chat at https://gitter.im/which_is_the_fastest/Lobby

This project aims to be a load benchmarking suite, no more, no less

Measuring response times (routing times) for each framework (middleware).

⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️
Results are not production-ready yet
⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️

Additional purposes :

  • Helping decide between languages, depending on use case
  • Learning languages, best practices, devops culture ...
  • Having fun ❤️

Requirements

  • Crystal as built-in tools are made in this language
  • Docker as frameworks are isolated into containers
  • wrk as benchmarking tool, >= 4.1.0
  • postgresql to store data, >= 10

ℹ️ you need wrk stable

git clone --branch 4.1.0 https://github.com/wg/wrk

⚠️ docker is used for development purpose, production results will be computed on DigitalOcean ⚠️

Usage

  • Install all dependencies
shards install
  • Build internal tools
shards build
  • Create and initialize the database
createdb -U postgres benchmark
psql -U postgres -d benchmark < .ci/dump.sql

Docker can be used to set up the database:

docker run -it --rm -d \
  -p 5432:5432 \
  -e POSTGRES_DB=benchmark \
  -e POSTGRES_HOST_AUTH_METHOD=trust \
  -v /tmp/pg-data:/var/lib/postgresql/data \
  --name pg postgres:12-alpine

Wait several seconds for the container to start, then inject the dump:

docker exec pg sh -c "echo \"$(cat .ci/dump.sql)\" | psql -U postgres -d benchmark"

After creating the database, export its URL:

export DATABASE_URL="postgresql://postgres@localhost/benchmark"
  • Make configuration
bin/make config
  • Build containers

jobs are either languages (example : crystal) or frameworks (example : router.cr)

bin/neph [job1] [job2] [job3] ...
  • Export all results readme
bin/db to_readme

Results

ℹ️ Updated on 2020-04-14 ℹ️

Benchmarking with wrk

  • Threads : 8
  • Timeout : 8
  • Duration : 15s (seconds)

ℹ️ Sorted by max req/s on concurrency 64 ℹ️

Language Framework Speed (64) Speed (256) Speed (512) Speed (1024) Speed (2048)
1 nim (1.0) httpbeast (2.2) 181 909 194 450 195 118 190 290 190 628
2 php (7.4) simps (1.0) 169 068 185 705 186 351 183 263 183 014
3 go (1.14) fasthttp (1.9) 159 078 168 009 170 886 165 526 164 820
4 javascript (13.7) nanoexpress-pro (1.11) 156 498 169 215 167 921 166 555 163 500
5 crystal (0.34) toro (0.4) 153 545 160 329 156 443 148 714 147 993
6 crystal (0.34) router.cr (0.2) 153 431 160 062 156 898 148 615 147 800
7 go (1.14) fasthttprouter (0.1) 153 323 161 677 164 610 159 914 159 052
8 javascript (13.7) sifrr (0.0) 153 264 170 170 173 839 165 638 165 156
9 go (1.14) fiber (1.9) 152 550 156 035 153 168 144 271 144 560
10 go (1.14) router (1.0) 152 497 161 444 164 327 159 767 159 649
11 go (1.14) gorouter-fasthttp (4.4) 152 018 160 222 163 448 158 927 158 757
12 go (1.14) atreugo (11.0) 151 975 160 271 163 175 158 813 158 753
13 crystal (0.34) ricr (0.1) 151 477 158 383 154 467 145 531 145 066
14 crystal (0.34) spider-gazelle (2.3) 149 680 153 649 147 380 138 615 144 339
15 java (8) rapidoid (5.5) 149 492 157 210 157 317 154 414 154 873
16 javascript (13.7) nanoexpress (2.0) 145 269 157 805 163 244 155 358 159 742
17 crystal (0.34) grip (0.28) 142 394 148 293 143 787 135 714 133 886
18 crystal (0.34) kemal (0.26) 142 283 147 398 143 844 135 292 133 498
19 php (7.4) workerman (4.0) 142 272 152 808 153 883 151 583 151 380
20 nim (1.0) jester (0.4) 141 684 151 574 151 745 147 761 147 979
21 rust (1.41) actix (2.0) 139 649 141 045 135 937 131 369 133 946
22 crystal (0.34) amber (0.34) 134 817 140 148 136 442 129 428 128 422
23 crystal (0.34) orion (2.3) 130 930 135 151 129 230 119 301 117 367
24 crystal (0.34) lucky (0.2) 129 659 133 824 128 073 118 351 118 207
25 crystal (0.34) athena (0.8) 121 841 123 582 117 422 105 059 103 505
26 java (8) act (1.8) 108 779 118 254 122 878 121 993 121 010
27 go (1.14) rte (0.0) 107 921 108 153 111 326 110 523 109 996
28 go (1.14) httprouter (1.3) 106 438 106 312 109 406 109 073 108 635
29 go (1.14) chi (4.1) 101 726 100 921 103 727 103 158 103 267
30 go (1.14) gorouter (4.4) 101 579 105 878 108 348 107 125 106 775
31 go (1.14) aero (1.3) 100 046 101 191 103 515 103 225 102 946
32 go (1.14) violetear (7.0) 99 554 99 610 102 459 102 284 102 074
33 ruby (2.7) agoo (2.12) 99 366 118 355 122 010 121 976 121 657
34 go (1.14) goroute (0.0) 98 260 97 070 100 210 100 125 99 980
35 go (1.14) kami (2.2) 97 875 101 941 103 484 101 437 101 287
36 go (1.14) echo (4.1) 97 553 96 672 99 660 99 449 99 382
37 c (99) kore (3.3) 96 528 117 750 147 560 155 560 139 864
38 go (1.14) gorilla-mux (1.7) 95 431 93 829 96 593 96 745 96 410
39 go (1.14) beego (1.12) 94 167 97 416 100 106 99 604 99 524
40 go (1.14) gin (1.6) 93 648 97 512 99 572 99 223 99 099
41 go (1.14) webgo (3.0) 93 023 92 466 95 580 95 784 95 128
42 csharp (8.0) aspnetcore (3.1) 92 622 98 971 99 697 97 847 98 258
43 cpp (14/17) drogon (1.0) 86 928 89 406 90 977 91 191 90 387
44 go (1.14) air (0.15) 81 146 85 338 86 990 86 861 86 708
45 javascript (13.7) polkadot (1.0) 78 457 91 197 91 046 87 287 86 754
46 javascript (13.7) restana (4.3) 75 875 86 859 86 698 82 485 80 412
47 javascript (13.7) 0http (2.2) 75 768 87 596 90 919 87 779 85 131
48 go (1.14) gf (1.12) 75 029 80 296 81 561 81 361 81 129
49 c (11) agoo-c (0.7) 72 586 160 159 172 258 180 260 178 804
50 scala (2.12) akkahttp (10.1) 71 151 75 299 74 169 72 303 73 003
51 javascript (13.7) rayo (1.3) 70 038 78 564 78 617 76 211 76 549
52 javascript (13.7) polka (0.5) 67 280 77 878 76 103 71 457 71 426
53 python (3.8) falcon (2.0) 66 567 74 880 74 711 75 226 74 462
54 java (8) javalin (3.5) 65 857 74 748 73 078 71 511 70 796
55 php (7.4) one (2.0) 63 660 67 528 67 467 67 004 66 913
56 swift (5.1) perfect (3.1) 63 184 74 817 77 206 76 621 75 772
57 elixir (1.1) cowboy_stream (2.7) 63 161 63 415 62 777 61 235 61 062
58 javascript (13.7) muneem (2.4) 62 381 63 553 58 890 60 297 62 473
59 java (8) spring-boot (2.1) 62 040 68 667 68 420 65 760 66 901
60 go (1.14) mars (1.0) 61 011 63 448 66 039 65 798 65 791
61 kotlin (1.3) ktor (1.2) 60 224 71 643 72 561 72 260 69 229
62 javascript (13.7) foxify (0.1) 58 576 62 082 61 425 58 826 56 721
63 java (8) micronaut (1.2) 57 060 63 734 62 669 60 710 61 317
64 python (3.8) apidaora (0.24) 56 634 62 625 63 741 59 976 59 744
65 javascript (13.7) fastify (2.13) 56 294 62 338 59 009 56 769 55 991
66 rust (1.41) nickel (0.11) 56 055 55 687 55 470 56 611 55 895
67 python (3.8) bottle (0.12) 55 839 59 855 59 306 57 831 58 117
68 javascript (13.7) koa (2.11) 53 925 60 611 59 197 55 537 53 762
69 scala (2.12) http4s (0.18) 52 976 60 426 61 858 62 919 63 309
70 clojure (1.1) coast (1.0) 52 839 54 239 54 315 53 972 54 048
71 python (3.8) asgineer (0.7) 51 875 57 334 57 271 55 142 55 027
72 javascript (13.7) iotjs-express (0.0) 50 906 53 982 56 415 54 254 52 538
73 java (8) spring-framework (5.2) 50 196 56 926 55 246 57 926 59 285
74 python (3.8) blacksheep (0.2) 49 771 54 361 52 554 51 274 51 388
75 javascript (13.7) feathersjs (4.5) 49 637 51 133 51 280 50 215 49 445
76 swift (5.1) kitura (2.8) 48 720 47 958 48 268 47 186 47 100
77 swift (5.1) kitura-nio (2.8) 48 534 48 940 48 669 48 044 47 579
78 javascript (13.7) express (4.17) 48 225 49 720 49 332 46 876 46 196
79 php (7.4) hyperf (1.0) 47 584 49 286 48 842 49 029 48 909
80 javascript (13.7) moleculer (0.14) 47 545 50 034 48 760 47 976 47 860
81 python (3.8) pyramid (1.1) 46 389 44 880 41 048 43 111 45 260
82 cpp (11) evhtp (1.2) 46 289 46 313 46 357 46 452 45 246
83 swift (5.1) vapor (3.3) 46 248 48 547 48 062 47 119 46 519
84 rust (1.41) gotham (0.4) 42 685 48 705 51 457 52 502 52 496
85 javascript (13.7) hapi (19.1) 42 247 44 096 43 481 42 943 42 887
86 python (3.8) hug (2.6) 42 090 43 373 46 136 44 642 47 413
87 php (7.4) imi (1.0) 40 745 41 698 42 114 41 813 41 873
88 elixir (1.1) cowboy (2.7) 40 511 41 238 40 931 39 605 39 672
89 python (3.8) starlette (0.13) 39 410 47 222 46 781 44 649 42 451
90 ruby (2.7) hanami-api (0.1) 39 220 39 530 37 676 36 995 37 953
91 ruby (2.7) syro (3.2) 39 026 40 518 38 476 38 128 38 267
92 php (7.4) sw-fw-less (preview) 38 588 41 686 41 744 41 253 41 187
93 javascript (13.7) restify (8.5) 37 078 38 706 37 568 37 338 37 896
94 python (3.8) emmett (2.0.0b2) 36 951 40 872 39 754 38 537 38 148
95 elixir (1.1) plug (1.8) 36 150 36 393 36 191 34 833 34 634
96 ruby (2.7) roda (3.3) 35 638 37 314 35 531 35 694 35 534
97 php (7.4) swoft (2.0) 35 534 36 602 36 404 35 924 35 895
98 ruby (2.7) cuba (3.9) 33 391 33 364 32 404 32 333 32 345
99 dart (2.7) aqueduct (3.3) 31 373 30 595 31 499 30 812 30 992
100 elixir (1.1) phoenix (1.4) 31 086 31 250 30 968 29 878 29 970
101 ruby (2.7) rack-routing (0.0) 28 190 27 682 27 072 27 177 27 054
102 python (3.8) responder (2.0) 27 816 30 392 30 247 29 195 29 193
103 ruby (2.7) rack_app (7.6) 27 531 27 504 26 908 26 819 26 889
104 python (3.8) fastapi (0.54) 26 541 29 793 28 820 27 196 26 811
105 python (3.8) aiohttp (3.6) 26 457 28 506 28 550 27 473 27 487
106 python (3.8) molten (1.0) 25 584 27 164 26 925 26 529 26 595
107 ruby (2.7) camping (2.1) 25 112 24 398 23 994 23 892 23 914
108 python (3.8) flask (1.1) 23 611 23 935 23 636 23 591 23 726
109 python (3.8) clastic (19.9) 22 571 23 046 23 983 23 731 23 426
110 javascript (13.7) turbo_polka (0.3) 22 372 21 335 19 724 20 046 19 895
111 python (3.8) masonite (2.3) 21 403 22 387 22 968 20 409 22 429
112 rust (1.41) iron (0.6) 18 667 18 809 18 781 18 648 18 595
113 php (7.4) spiral (2.4) 18 180 18 342 18 435 18 359 18 469
114 fsharp (4.7) suave (2.5) 17 358 23 842 32 745 33 859 32 846
115 ruby (2.7) sinatra (2.0) 15 665 15 436 15 395 15 451 15 394
116 python (3.8) sanic (19.12) 15 477 15 525 13 798 14 438 14 336
117 ruby (2.7) grape (1.3) 14 193 14 397 14 317 14 290 14 346
118 javascript (13.7) sails (1.2) 11 975 12 328 11 923 12 933 12 712
119 python (3.8) quart (0.11) 11 940 12 129 11 313 10 745 10 764
120 ruby (2.7) flame (4.18) 11 270 11 131 11 097 10 994 11 001
121 swift (5.1) swifter (1.4) 10 733 10 952 10 929 10 827 11 002
122 ruby (2.7) hanami (1.3) 10 617 10 649 10 560 10 538 10 516
123 php (7.4) ubiquity (2.3) 9 082 8 962 8 938 50 843 47 658
124 python (3.8) tornado (6.0) 8 826 9 881 9 093 9 190 9 181
125 python (3.8) cherrypy (18.5) 8 375 9 099 9 033 8 970 9 075
126 php (7.4) one-fpm (2.0) 7 824 7 763 7 669 43 495 40 967
127 php (7.4) hamlet (3.2) 7 450 7 404 7 311 43 665 42 628
128 php (7.4) phalcon (4.0) 7 441 7 433 7 368 49 522 47 587
129 go (1.14) gramework (1.7) 7 258 9 817 7 922 4 920 4 903
130 python (3.8) django (3.0) 7 161 9 377 9 490 9 510 9 189
131 php (7.4) chubbyphp (2.8) 5 580 5 585 5 686 42 418 39 443
132 php (7.4) slim (4.4) 4 630 4 647 4 740 42 796 39 763
133 php (7.4) lumen (7.0) 4 282 4 284 4 350 42 332 40 222
134 php (7.4) yii (2.0) 4 095 4 068 4 177 41 476 40 107
135 ruby (2.7) rails (6.0) 3 805 3 598 3 600 3 582 3 562
136 crystal (0.34) onyx (0.5) 3 228 3 325 3 314 3 319 2 470
137 php (7.4) symfony (4.3) 3 155 3 191 3 248 40 577 38 062
138 php (7.4) mezzio (3.2) 2 844 2 876 2 903 40 690 38 978
139 python (3.8) cyclone (1.3) 2 318 2 373 2 381 2 316 2 333
140 julia (1.3) merly (0.2) 1 922 5 626 4 431 3 470 2 212
141 python (3.8) klein (19.6) 1 572 1 604 1 582 1 564 1 566
142 python (3.8) nameko (2.12) 1 530 1 489 1 477 1 446 1 407
143 php (7.4) laminas (3.1) 1 424 1 454 1 516 38 185 36 482
144 perl (5.3) dancer2 (2.0) 885 825 1 572 1 142 587
145 php (7.4) basicphp (0.9) 565 459 1 461 33 165 32 868
146 php (7.4) laravel (7.5) 184 163 3 866 23 579 21 997

How to contribute ?

In any way you want ...

  • Request a framework addition
  • Report a bug (on any implementation)
  • Suggest an idea
  • ...

Any kind of idea is ❤️

Contributors