Performance
Since all the heavy lifting is done in compiled Java, it should be no surprise that Manticore is very fast.
ruby_http_benchmark was used to compare throughput for various frameworks, including:
- Manticore
- Typhoeus
- HTTPClient
- HTTParty
- rest-client
The benchmark was performs a warmup pass of 5000 requests per framework, then performs several hundred passes in various scenarios, including:
- Synchronous download of 1k, 10k, 100k, 1000k, and 10,000k files, both gzipped and plaintext
- Asynchronous download of 1k, 10k, 100k, 1000k, and 10,000k files, both gzipped and plaintext, at a concurrency rate of 8.
Results were collected as gnuplot graphs and the 90th percentile of requests charted as boxplots to illustrate their standard maxima, minima, average, and outliers.
These results were performed against a local server. A proper benchmark would test against remote servers, but it's somewhat rude to go download several GB of data every time one wants to test a code change. :)
The local server was nginx with uncompressed and pre-compressed files served via the gzip_static module. ulimits were relaxed to ensure that there would be no concurrency issues. The tests were run on an AWS EC2 c3.xlarge.
| Framework | Test | Speed (seconds) |
|---|---|---|
| Manticore | async_gzip (large) | 1.558 |
| Typhoeus | async_gzip (large) | 4.928 |
| Framework | Test | Speed (seconds) |
|---|---|---|
| Manticore | async_gzip (medium) | 0.902 |
| Typhoeus | async_gzip (medium) | 3.525 |
| Framework | Test | Speed (seconds) |
|---|---|---|
| Manticore | async_gzip (small) | 0.943 |
| Typhoeus | async_gzip (small) | 3.463 |
| Framework | Test | Speed (seconds) |
|---|---|---|
| Manticore | async_gzip (very_large) | 1.38 |
| Typhoeus | async_gzip (very_large) | 4.316 |
| Framework | Test | Speed (seconds) |
|---|---|---|
| Manticore | async_gzip (x_large) | 23.181 |
| Typhoeus | async_gzip (x_large) | 47.685 |
| Framework | Test | Speed (seconds) |
|---|---|---|
| Manticore | async_plain (large) | 2.072 |
| Typhoeus | async_plain (large) | 8.371 |
| Framework | Test | Speed (seconds) |
|---|---|---|
| Manticore | async_plain (medium) | 1.548 |
| Typhoeus | async_plain (medium) | 4.086 |
| Framework | Test | Speed (seconds) |
|---|---|---|
| Manticore | async_plain (small) | 1.595 |
| Typhoeus | async_plain (small) | 3.903 |
| Framework | Test | Speed (seconds) |
|---|---|---|
| Manticore | async_plain (very_large) | 0.771 |
| Typhoeus | async_plain (very_large) | 10.025 |
| Framework | Test | Speed (seconds) |
|---|---|---|
| Manticore | async_plain (x_large | 17.95 |
| Typhoeus | async_plain (x_large | 35.971 |
| Framework | Test | Speed (seconds) |
|---|---|---|
| Manticore | gzip (large) | 0.5 |
| Typhoeus | gzip (large) | 0.581 |
| HTTPClient | gzip (large) | 0.907 |
| RestClient | gzip (large) | 0.932 |
| HTTParty | gzip (large) | 0.939 |
| Excon | gzip (large) | 1.762 |
| Framework | Test | Speed (seconds) |
|---|---|---|
| Manticore | gzip (medium) | 0.293 |
| Typhoeus | gzip (medium) | 0.354 |
| HTTPClient | gzip (medium) | 0.584 |
| HTTParty | gzip (medium) | 0.586 |
| RestClient | gzip (medium) | 0.586 |
| Excon | gzip (medium) | 1.407 |
| Framework | Test | Speed (seconds) |
|---|---|---|
| Manticore | gzip (small) | 0.287 |
| Typhoeus | gzip (small) | 0.344 |
| HTTParty | gzip (small) | 0.54 |
| RestClient | gzip (small) | 0.573 |
| HTTPClient | gzip (small) | 0.573 |
| Excon | gzip (small) | 1.362 |
| Framework | Test | Speed (seconds) |
|---|---|---|
| Manticore | gzip (very_large) | 0.434 |
| Typhoeus | gzip (very_large) | 0.525 |
| HTTParty | gzip (very_large) | 0.832 |
| RestClient | gzip (very_large) | 0.839 |
| HTTPClient | gzip (very_large) | 0.851 |
| Excon | gzip (very_large) | 0.995 |
| Framework | Test | Speed (seconds) |
|---|---|---|
| Manticore | gzip (x_large) | 4.327 |
| Typhoeus | gzip (x_large) | 4.465 |
| RestClient | gzip (x_large) | 7.779 |
| HTTParty | gzip (x_large) | 7.787 |
| Excon | gzip (x_large) | 7.954 |
| HTTPClient | gzip (x_large) | 8.281 |
| Framework | Test | Speed (seconds) |
|---|---|---|
| Typhoeus | sync (large) | 0.488 |
| Manticore | sync (large) | 0.489 |
| HTTParty | sync (large) | 0.687 |
| HTTPClient | sync (large) | 0.906 |
| RestClient | sync (large) | 1.071 |
| Excon | sync (large) | 1.424 |
| Framework | Test | Speed (seconds) |
|---|---|---|
| Manticore | sync (medium) | 0.351 |
| Typhoeus | sync (medium) | 0.422 |
| HTTParty | sync (medium) | 0.552 |
| HTTPClient | sync (medium) | 0.608 |
| RestClient | sync (medium) | 0.695 |
| Excon | sync (medium) | 1.324 |
| Framework | Test | Speed (seconds) |
|---|---|---|
| Manticore | sync (small) | 0.428 |
| Typhoeus | sync (small) | 0.49 |
| HTTParty | sync (small) | 0.648 |
| HTTPClient | sync (small) | 0.668 |
| RestClient | sync (small) | 0.7 |
| Excon | sync (small) | 1.417 |
| Framework | Test | Speed (seconds) |
|---|---|---|
| Manticore | sync (very_large) | 0.223 |
| Typhoeus | sync (very_large) | 0.359 |
| HTTParty | sync (very_large) | 0.452 |
| Excon | sync (very_large) | 0.621 |
| HTTPClient | sync (very_large) | 0.647 |
| RestClient | sync (very_large) | 0.836 |
| Framework | Test | Speed (seconds) |
|---|---|---|
| Manticore | sync (x_large) | 1.913 |
| Excon | sync (x_large) | 2.538 |
| Typhoeus | sync (x_large) | 2.644 |
| HTTParty | sync (x_large) | 3.582 |
| HTTPClient | sync (x_large) | 4.455 |
| RestClient | sync (x_large) | 7.939 |
