Demo of a distributed REST facade fetching data from Ethereum via JSON-RPC over HTTP.
- horizontally scalable
- request grouping
- redis cache
- redis locks
- load balancer
- docker-compose
- separate test container (delve)
Terminal 1:
$ go install github.com/go-task/task/v3/cmd/task@latest
$ task docker-start
Terminal 2:
$ http :8080/v1/block/0x5BAD55
$ task benchmark-blocks
GET /v1/block/:number
- docker
- httpie
- siege
- nodejs
$ task --list
task: Available tasks for this project:
* benchmark-blocks: Benchmark 100 random blocks
* build: Build a binary
* clean: Stop and remove all the data
* debug: Run in debug mode
* docker-rebuild-start: Rebuild and start docker compose
* docker-start: Start docker compose
* docker-stop: Stop docker compose
* flush-cache: Flush redis cache
* kill: Kill the service process
* rebuild-test-instance: Rebuild the test instance
* restart-load-balancer: Restart the load balancer with a new config
* start: Start from source
* test: Run go test
* test-load-blocks: Load test 10k random blocks for 1 minute
* test-locks: Test if redis locking works
Semi-automated test verifying lock synchronization, executed in a separate container. After running the test, check the logs for "locked, waiting for".
$ task test-locks
Output:
eth-rest-test-1 | 2022/11/27 11:51:14 [service.getBlock] get 0x5BAD55
eth-rest-test-1 | 2022/11/27 11:51:14 [service.getBlock] check cache for 0x5BAD55
eth-rest-test-1 | 2022/11/27 11:51:14 [service.requestBlock] request 0x5BAD55
eth-rest-test-1 | 2022/11/27 11:51:14 [service.getBlock] get 0x5BAD55
eth-rest-test-1 | 2022/11/27 11:51:14 [service.getBlock] check cache for 0x5BAD55
eth-rest-test-1 | 2022/11/27 11:51:14 [service.getBlock] locked, waiting for 0x5BAD55
eth-rest-test-1 | 2022/11/27 11:51:15 [eth.GetBlock] sleeping for 0.500000
eth-rest-test-1 | 2022/11/27 11:51:15 [service.dispatchCompleted] sending block-completed for 0x5BAD55
eth-rest-test-1 | 2022/11/27 11:51:15 [service.GetBlock] block-completed received for 0x5BAD55
Benchmarks can be found in ./BENCHMARK.md.
One example is full cache, 3 workers, redis 1g, http-rpc with the following results:
Transactions: 2500 hits
Availability: 100.00 %
Elapsed time: 0.63 secs
Data transferred: 10.91 MB
Response time: 0.01 secs
Transaction rate: 3968.25 trans/sec
Throughput: 17.31 MB/sec
Concurrency: 23.22
Successful transactions: 2500
Failed transactions: 0
Longest transaction: 0.05
Shortest transaction: 0.00
$ task test-load-blocks
- 130k hits / minute (10k random blocks)
- 0.04s avg response time
Transactions: 130736 hits
Availability: 100.00 %
Elapsed time: 60.04 secs
Data transferred: 942.20 MB
Response time: 0.04 secs
Transaction rate: 2177.48 trans/sec
Throughput: 15.69 MB/sec
Concurrency: 79.49
Successful transactions: 130833
Failed transactions: 0
Longest transaction: 1.24
Shortest transaction: 0.00
- retry logic for ETH calls
- lock support
- throttling for ETH calls
- transactions support the way blocks are implemented
- transactions extracted directly from blocks data
- kept separately in cache
- benchmarks for transactions
- JSON-RPC via websocket
- unit tests
- fully automated load tests
- predefined docker-compose files
- more documentation
- OpenAPI & swagger