RedisClient

redis-client is a simple, low-level, client for Redis 6+.

Contrary to the redis gem, redis-client doesn't try to map all Redis commands to Ruby constructs, it merely is a thin wrapper on top of the RESP3 protocol.

Installation

Add this line to your application's Gemfile:

gem 'redis-client'

And then execute:

$ bundle install

Or install it yourself as:

$ gem install redis-client

Usage

To use RedisClient you first define a connection configuration, from which you can create a connection pool:

redis_config = RedisClient.config(host: "10.0.1.1", port: 6380, db: 15)
redis = redis_config.new_pool(timeout: 0.5, size: Integer(ENV.fetch("RAILS_MAX_THREADS", 5)))

redis.call("PING") # => "PONG"

If you are issuing multiple commands in a raw, but can't pipeline them, it's best to use #with to avoid going through the connection checkout several times:

redis.with do |r|
  r.call("SET", "mykey", "hello world") # => "OK"
  r.call("GET", "mykey") # => "hello world"
end

If you are working in a single-threaded environment, or wish to use your own connection pooling mechanism, you can obtain a raw client with #new_client

redis_config = RedisClient.config(host: "10.0.1.1", port: 6380, db: 15)
redis = redis_config.new_client
redis.call("PING") # => "PONG"

NOTE: Raw RedisClient instances must not be shared between threads. Make sure to read the section on thread safety.

For simple use cases where only a single connection is needed, you can use the RedisClient.new shortcut:

redis = RedisClient.new
redis.call("GET", "mykey")

Configuration

• url: A Redis connection URL, e.g. redis://example.com:6379/5 - a rediss:// scheme enables SSL, and the path is interpreted as a database number. To connect to UNIX domain sockets, the url can also just be a path, and the database specified as query parameter: /run/redis/foo.sock?db=5, or optionally have a unix:// scheme: unix:///run/redis/foo.sock?db=5 Note that all other configurations take precedence, e.g. RedisClient.config(url: "redis://localhost:3000", port: 6380) will connect on port 6380.
• host: The server hostname or IP address. Defaults to "localhost".
• port: The server port. Defaults to 6379.
• path: The path to a UNIX socket, if set url, host and port are ignored.
• ssl: Whether to connect using SSL or not.
• ssl_params: A configuration Hash passed to OpenSSL::SSL::SSLContext#set_params, notable options include:
  • cert: The path to the client certificate (e.g. client.crt).
  • key: The path to the client key (e.g. client.key).
  • ca_file: The certificate authority to use, useful for self-signed certificates (e.g. ca.crt),
• db: The database to select after connecting, defaults to 0.
• id ID for the client connection, assigns name to current connection by sending CLIENT SETNAME.
• username Username to authenticate against server, defaults to "default".
• password Password to authenticate against server.
• timeout: The general timeout in seconds, default to 1.0.
• connect_timeout: The connection timeout, takes precedence over the general timeout when connecting to the server.
• read_timeout: The read timeout, takes precedence over the general timeout when reading responses from the server.
• write_timeout: The write timeout, takes precedence over the general timeout when sending commands to the server.
• reconnect_attempts: Specify how many times the client should retry to send queries. Defaults to 0. Makes sure to read the reconnection section before enabling it.
• circuit_breaker: A Hash with circuit breaker configuration. Defaults to nil. See the circuit breaker section for details.
• protocol: The version of the RESP protocol to use. Default to 3.
• custom: A user-owned value ignored by redis-client but available as Config#custom. This can be used to hold middleware configurations and other user-specific metadata.

Sentinel support

The client is able to perform automatic failover by using Redis Sentinel.

To connect using Sentinel, use:

redis_config = RedisClient.sentinel(
  name: "mymaster",
  sentinels: [
    { host: "127.0.0.1", port: 26380 },
    { host: "127.0.0.1", port: 26381 },
  ],
  role: :master,
)

or:

redis_config = RedisClient.sentinel(
  name: "mymaster",
  sentinels: [
    "redis://127.0.0.1:26380",
    "redis://127.0.0.1:26381",
  ],
  role: :master,
)

• The name identifies a group of Redis instances composed of a master and one or more replicas (mymaster in the example).

• It is possible to optionally provide a role. The allowed roles are :master and :replica. When the role is :replica, the client will try to connect to a random replica of the specified master. If a role is not specified, the client will connect to the master.

• When using the Sentinel support you need to specify a list of sentinels to connect to. The list does not need to enumerate all your Sentinel instances, but a few so that if one is down the client will try the next one. The client is able to remember the last Sentinel that was able to reply correctly and will use it for the next requests.

To authenticate Sentinel itself, you can specify the sentinel_username and sentinel_password options per instance. Exclude the sentinel_username option if you're using password-only authentication.

SENTINELS = [{ host: '127.0.0.1', port: 26380},
             { host: '127.0.0.1', port: 26381}]

redis_config = RedisClient.sentinel(name: 'mymaster', sentinel_username: 'appuser', sentinel_password: 'mysecret', sentinels: SENTINELS, role: :master)

If you specify a username and/or password at the top level for your main Redis instance, Sentinel will not using thouse credentials

# Use 'mysecret' to authenticate against the mymaster instance, but skip authentication for the sentinels:
SENTINELS = [{ host: '127.0.0.1', port: 26380 },
             { host: '127.0.0.1', port: 26381 }]

redis_config = RedisClient.sentinel(name: 'mymaster', sentinels: SENTINELS, role: :master, password: 'mysecret')

So you have to provide Sentinel credential and Redis explictly even they are the same

# Use 'mysecret' to authenticate against the mymaster instance and sentinel
SENTINELS = [{ host: '127.0.0.1', port: 26380 },
             { host: '127.0.0.1', port: 26381 }]

redis_config = RedisClient.sentinel(name: 'mymaster', sentinels: SENTINELS, role: :master, password: 'mysecret', sentinel_password: 'mysecret')

Also the name, password, username and db for Redis instance can be passed as an url:

redis_config = RedisClient.sentinel(url: "redis://appuser:mysecret@mymaster/10", sentinels: SENTINELS, role: :master)

Type support

Only a select few Ruby types are supported as arguments beside strings.

Integer and Float are supported:

redis.call("SET", "mykey", 42)
redis.call("SET", "mykey", 1.23)

is equivalent to:

redis.call("SET", "mykey", 42.to_s)
redis.call("SET", "mykey", 1.23.to_s)

Arrays are flattened as arguments:

redis.call("LPUSH", "list", [1, 2, 3], 4)

is equivalent to:

redis.call("LPUSH", "list", "1", "2", "3", "4")

Hashes are flattened as well:

redis.call("HMSET", "hash", { "foo" => "1", "bar" => "2" })

is equivalent to:

redis.call("HMSET", "hash", "foo", "1", "bar", "2")

Any other type requires the caller to explicitly cast the argument as a string.

Keywords arguments are treated as Redis command flags:

redis.call("SET", "mykey", "value", nx: true, ex: 60)
redis.call("SET", "mykey", "value", nx: false, ex: nil)

is equivalent to:

redis.call("SET", "mykey", "value", "nx", "ex", "60")
redis.call("SET", "mykey", "value")

If flags are built dynamically, you'll have to explicitly pass them as keyword arguments with **:

flags = {}
flags[:nx] = true if something?
redis.call("SET", "mykey", "value", **flags)

Important Note: because of the keyword argument semantic change between Ruby 2 and Ruby 3, unclosed hash literals with string keys may be interpreted differently:

redis.call("HMSET", "hash", "foo" => "bar")

On Ruby 2 "foo" => "bar" will be passed as a positional argument, but on Ruby 3 it will be interpreted as keyword arguments. To avoid such problem, make sure to enclose hash literals:

redis.call("HMSET", "hash", { "foo" => "bar" })

Commands return values

Contrary to the redis gem, redis-client doesn't do any type casting on the return value of commands.

If you wish to cast the return value, you can pass a block to the #call family of methods:

redis.call("INCR", "counter") # => 1
redis.call("GET", "counter") # => "1"
redis.call("GET", "counter", &:to_i) # => 1

redis.call("EXISTS", "counter") # => 1
redis.call("EXISTS", "counter") { |c| c > 0 } # => true

*_v methods

In some it's more convenient to pass commands as arrays, for that _v versions of call methods are available.

redis.call_v(["MGET"] + keys)
redis.blocking_call_v(1, ["MGET"] + keys)
redis.call_once_v(1, ["MGET"] + keys)

Blocking commands

For blocking commands such as BRPOP, a custom timeout duration can be passed as first argument of the #blocking_call method:

redis.blocking_call(timeout, "BRPOP", "key", 0)

If timeout is reached, #blocking_call raises RedisClient::ReadTimeoutError and doesn't retry regardless of the reconnect_attempts configuration.

timeout is expressed in seconds, you can pass false or 0 to mean no timeout.

Scan commands

For easier use of the SCAN family of commands, #scan, #sscan, #hscan and #zscan methods are provided

redis.scan("MATCH", "pattern:*") do |key|
  ...
end

redis.sscan("myset", "MATCH", "pattern:*") do |key|
  ...
end

For HSCAN and ZSCAN, pairs are yielded

redis.hscan("myhash", "MATCH", "pattern:*") do |key, value|
  ...
end

redis.zscan("myzset") do |element, score|
  ...
end

In all cases the cursor parameter must be omitted and starts at 0.

Pipelining

When multiple commands are executed sequentially, but are not dependent, the calls can be pipelined. This means that the client doesn't wait for reply of the first command before sending the next command. The advantage is that multiple commands are sent at once, resulting in faster overall execution.

The client can be instructed to pipeline commands by using the #pipelined method. After the block is executed, the client sends all commands to Redis and gathers their replies. These replies are returned by the #pipelined method.

redis.pipelined do |pipeline|
  pipeline.call("SET", "foo", "bar") # => nil
  pipeline.call("INCR", "baz") # => nil
end
# => ["OK", 1]

Exception management

The exception flag in the #pipelined method of RedisClient is a feature that modifies the pipeline execution behavior. When set to false, it doesn't raise an exception when a command error occurs. Instead, it allows the pipeline to execute all commands, and any failed command will be available in the returned array. (Defaults to true)

results = redis.pipelined(exception: false) do |pipeline|
  pipeline.call("SET", "foo", "bar") # => nil
  pipeline.call("DOESNOTEXIST", 12) # => nil
  pipeline.call("INCR", "baz") # => nil
end
# results => ["OK", #<RedisClient::CommandError: ERR unknown command 'DOESNOTEXIST', with args beginning with: '12'>, 2]

results.each do |result|
  if result.is_a?(RedisClient::CommandError)
    # Do something with the failed result
  end
end

Transactions

You can use MULTI/EXEC to run a number of commands in an atomic fashion. This is similar to executing a pipeline, but the commands are preceded by a call to MULTI, and followed by a call to EXEC. Like the regular pipeline, the replies to the commands are returned by the #multi method.

redis.multi do |transaction|
  transaction.call("SET", "foo", "bar") # => nil
  transaction.call("INCR", "baz") # => nil
end
# => ["OK", 1]

For optimistic locking, the watched keys can be passed to the #multi method:

redis.multi(watch: ["title"]) do |transaction|
  title = redis.call("GET", "title")
  transaction.call("SET", "title", title.upcase)
end
# => ["OK"] / nil

If the transaction wasn't successful, #multi will return nil.

Note that transactions using optimistic locking aren't automatically retried upon connection errors.

Publish / Subscribe

Pub/Sub related commands must be called on a dedicated PubSub object:

redis = RedisClient.new
pubsub = redis.pubsub
pubsub.call("SUBSCRIBE", "channel-1", "channel-2")

loop do
  if message = pubsub.next_event(timeout)
    message # => ["subscribe", "channel-1", 1]
  else
    # no new message was received in the allocated timeout
  end
end

Note: pubsub connections are stateful, as such they won't ever reconnect automatically. The caller is responsible for reconnecting if the connection is lost and to resubscribe to all channels.

Production

Instrumentation and Middlewares

redis-client offers a public middleware API to aid in monitoring and library extension. Middleware can be registered either globally or on a given configuration instance.

module MyGlobalRedisInstrumentation
  def connect(redis_config)
    MyMonitoringService.instrument("redis.connect") { super }
  end

  def call(command, redis_config)
    MyMonitoringService.instrument("redis.query") { super }
  end

  def call_pipelined(commands, redis_config)
    MyMonitoringService.instrument("redis.pipeline") { super }
  end
end
RedisClient.register(MyGlobalRedisInstrumentation)

Note that RedisClient.register is global and apply to all RedisClient instances.

To add middlewares to only a single client, you can provide them when creating the config:

redis_config = RedisClient.config(middlewares: [AnotherRedisInstrumentation])
redis_config.new_client

If middlewares need a client-specific configuration, Config#custom can be used

module MyGlobalRedisInstrumentation
  def connect(redis_config)
    MyMonitoringService.instrument("redis.connect", tags: redis_config.custom[:tags]) { super }
  end

  def call(command, redis_config)
    MyMonitoringService.instrument("redis.query", tags: redis_config.custom[:tags]) { super }
  end

  def call_pipelined(commands, redis_config)
    MyMonitoringService.instrument("redis.pipeline", tags: redis_config.custom[:tags]) { super }
  end
end
RedisClient.register(MyGlobalRedisInstrumentation)

redis_config = RedisClient.config(custom: { tags: { "environment": Rails.env }})

Timeouts

The client allows you to configure connect, read, and write timeouts. Passing a single timeout option will set all three values:

RedisClient.config(timeout: 1).new

But you can use specific values for each of them:

RedisClient.config(
  connect_timeout: 0.2,
  read_timeout: 1.0,
  write_timeout: 0.5,
).new

All timeout values are specified in seconds.

Reconnection

redis-client support automatic reconnection after network errors via the reconnect_attempts: configuration option.

It can be set as a number of retries:

redis_config = RedisClient.config(reconnect_attempts: 1)

Important Note: Retrying may cause commands to be issued more than once to the server, so in the case of non-idempotent commands such as LPUSH or INCR, it may cause consistency issues.

To selectively disable automatic retries, you can use the #call_once method:

redis_config = RedisClient.config(reconnect_attempts: 3)
redis = redis_config.new_client
redis.call("GET", "counter") # Will be retried up to 3 times.
redis.call_once("INCR", "counter") # Won't be retried.

Note: automatic reconnection doesn't apply to pubsub clients as their connection is stateful.

Exponential backoff

Alternatively, reconnect_attempts accepts a list of sleep durations for implementing exponential backoff:

redis_config = RedisClient.config(reconnect_attempts: [0, 0.05, 0.1])

This configuration is generally used when the Redis server is expected to failover or recover relatively quickly and that it's not really possible to continue without issuing the command.

When the Redis server is used as an ephemeral cache, circuit breakers are generally preferred.

Circuit Breaker

When Redis is used as a cache and a connection error happens, you may not want to retry as it might take longer than to recompute the value. Instead it's likely preferable to mark the server as unavailable and let it recover for a while.

Circuit breakers are a pattern that does exactly that.

Configuation options:

• error_threshold. The amount of errors to encounter within error_threshold_timeout amount of time before opening the circuit, that is to start rejecting requests instantly.
• error_threshold_timeout. The amount of time in seconds that error_threshold errors must occur to open the circuit. Defaults to error_timeout seconds if not set.
• error_timeout. The amount of time in seconds until trying to query the resource again.
• success_threshold. The amount of successes on the circuit until closing it again, that is to start accepting all requests to the circuit.

RedisClient.config(
  circuit_breaker: {
    # Stop querying the server after 3 errors happened in a 2 seconds window
    error_threshold: 3,
    error_threshold_timeout: 2,

    # Try querying again after 1 second
    error_timeout: 1,

    # Stay in half-open state until 3 queries succeeded.
    success_threshold: 3,
  }
)

Drivers

redis-client ships with a pure Ruby socket implementation.

For increased performance, you can enable the hiredis binding by adding hiredis-client to your Gemfile:

gem "hiredis-client"

The hiredis binding is only available on Linux, macOS and other POSIX platforms. You can install the gem on other platforms, but it won't have any effect.

The default driver can be set through RedisClient.default_driver=:

Notable differences with the redis gem

Thread Safety

Contrary to the redis gem, redis-client doesn't protect against concurrent access. To use redis-client in concurrent environments, you MUST use a connection pool, or have one client per Thread or Fiber.

Development

After checking out the repo, run bin/setup to install dependencies. You can also run bin/console for an interactive prompt that will allow you to experiment.

To install this gem onto your local machine, run bundle exec rake install. To release a new version, update the version number in version.rb, and then run bundle exec rake release, which will create a git tag for the version, push git commits and the created tag, and push the .gem file to rubygems.org.

Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/redis-rb/redis-client.