English | 中文
This is rotom, a high performance, low latency tiny Redis Server written in Go. It replicates the core event loop mechanism AeLoop in Redis based on I/O multiplexing.
- Implements the AeLoop single-threaded event loop from Redis using the epoll network model.
- Compatible with the Redis RESP protocol, allowing any Redis client to connect to rotom.
- Implements data structures such as dict, list, map, zipmap, set, zipset, and zset.
- Supports AOF.
- Supports 20+ commonly used commands.
- Supports execute lua scripts.
AeLoop (Async Event Loop) is the core asynchronous event-driven mechanism in Redis, which mainly includes:
- FileEvent: Uses I/O multiplexing to handle read and write events on network sockets, categorized into
READABLE
andWRITABLE
. - TimeEvent: Handles tasks that need to be executed after a delay or periodically, such as expiring items every
100ms
. - When events are ready, they are processed by callback functions bound to those events.
In rotom, the AeLoop event loop mechanism in Redis is replicated, specifically:
- When a new TCP connection arrives, the
AcceptHandler
obtains the socket fd and adds it to the event loop, registering a read event. - When the read event is ready,
ReadQueryFromClient
reads buffered data intoqueryBuf
. ProcessQueryBuf
parses and executes the corresponding commands fromqueryBuf
.- The command execution result is saved, and the socket fd's write event is registered.
- When the write event is ready,
SendReplyToClient
writes all results back to the client. A write event may return multiple read event results at once. - Resources are released, and the process continues until the service is shut down.
Rotom has made several optimizations in data structures:
- dict: Rotom uses
swissmap
as the db hash table, which has better performance and memory efficiency thanstdmap
. - hash: Uses
zipmap
when the hash is small andhashmap
when it is large. - set: Uses
zipset
when the set is small andmapset
when it is large. - list: Uses a
quicklist
based onlistpack
for a doubly linked list. - zset: Based on
hashmap
+rbtree
.
Notably, zipmap
and zipset
are space-efficient data structures based on listpack
, which is a new compressed list proposed by Redis to replace ziplist
, supporting both forward and reverse traversal and solving the cascading update issue in ziplist
.
The test will run rotom on the same machine with appendonly
disabled, and use redis-benchmark
tool to test the latency of different commands.
goos: linux
goarch: amd64
pkg: github.com/xgzlucario/rotom
cpu: 13th Gen Intel(R) Core(TM) i5-13600KF
redis rotom redis_P10 rotom_P10 redis_P50 rotom_P50
SET 268817 268817 2222222 2173913 3448276 5263158
GET 265957 259740 2702702 1818181 4347826 4545454
INCR 271739 261780 2500000 2439024 4347826 7692307
LPUSH 289017 282485 2083333 2272727 2941176 4347826
RPUSH 283286 271739 2272727 2439024 3333333 7692307
SADD 273972 269541 2439024 2631579 4000000 7142857
HSET 282485 277777 2000000 2127659 3030303 3703703
ZADD 273224 272479 1960784 2702702 2941176 6249999
- Support for LRU cache and memory eviction.
- Support for gradual rehashing in dict.
- Support for RDB and AOF Rewrite.
- Compatibility with more commonly used commands.
Running Locally
First, clone the project to your local machine:
git clone https://github.com/xgzlucario/rotom
Ensure your local Go environment is >= 1.22
. In the project directory, run go run .
to start the service, which listens on port 6379
by default:
$ go run .
2024-07-18 23:37:13 DBG
________ _____
___ __ \_______ /_____________ ___ Rotom 64 bit (amd64/linux)
__ /_/ / __ \ __/ __ \_ __ '__ \ Port: 6379, Pid: 15817
_ _, _// /_/ / /_ / /_/ / / / / / / Build:
/_/ |_| \____/\__/ \____//_/ /_/ /_/
2024-07-18 23:37:13 INF read config file config=config.json
2024-07-18 23:37:13 INF rotom server is ready to accept.
Running in a Container
Alternatively, you can run it in a container. First, build the Docker image by running make build-docker
:
REPOSITORY TAG IMAGE ID CREATED SIZE
rotom latest 22f42ce9ae0e 8 seconds ago 20.5MB
Then, start the container:
docker run --rm -p 6379:6379 --name rotom rotom:latest