Redis Cluster Client

This library is a client for Redis cluster. It depends on redis-client. So it would be better to read redis-client documents first.

This gem underlies the official gem redis-clustering. The redis-clustering gem was decoupled from the redis gem as of v5. Both are maintained by the repository in the official organization. The redis gem supported cluster mode since the pull request was merged until v4. You can see more details and reasons in the issue if you are interested.

gem 'redis-cluster-client'

Also, the other generic options can be passed. But :url, :host, :port and :path are ignored because they conflict with the :nodes option.

require 'redis_cluster_client'

# The following examples are Docker containers on localhost.
# The client first attempts to connect to redis://127.0.0.1:6379 internally.

# To connect to primary nodes only
RedisClient.cluster.new_client
#=> #<RedisClient::Cluster 172.20.0.2:6379, 172.20.0.6:6379, 172.20.0.7:6379>

# To connect to all nodes to use scale reading feature
RedisClient.cluster(replica: true).new_client
#=> #<RedisClient::Cluster 172.20.0.2:6379, 172.20.0.3:6379, 172.20.0.4:6379, 172.20.0.5:6379, 172.20.0.6:6379, 172.20.0.7:6379>

# To connect to all nodes to use scale reading feature + make reads equally likely from replicas and primary
RedisClient.cluster(replica: true, replica_affinity: :random_with_primary).new_client
#=> #<RedisClient::Cluster 172.20.0.2:6379, 172.20.0.3:6379, 172.20.0.4:6379, 172.20.0.5:6379, 172.20.0.6:6379, 172.20.0.7:6379>

# To connect to all nodes to use scale reading feature prioritizing low-latency replicas
RedisClient.cluster(replica: true, replica_affinity: :latency).new_client
#=> #<RedisClient::Cluster 172.20.0.2:6379, 172.20.0.3:6379, 172.20.0.4:6379, 172.20.0.5:6379, 172.20.0.6:6379, 172.20.0.7:6379>

# With generic options for redis-client
RedisClient.cluster(timeout: 3.0).new_client

# To connect with a subset of nodes for startup
RedisClient.cluster(nodes: %w[redis://node1:6379 redis://node2:6379]).new_client

# To connect with a subset of auth-needed nodes for startup

## with URL:
### User name and password should be URI encoded and the same in every node.
username = 'myuser'
password = URI.encode_www_form_component('!&<123-abc>')
RedisClient.cluster(nodes: %W[redis://#{username}:#{password}@node1:6379 redis://#{username}:#{password}@node2:6379]).new_client

## with options:
RedisClient.cluster(nodes: %w[redis://node1:6379 redis://node2:6379], username: 'myuser', password: '!&<123-abc>').new_client

# To connect to single endpoint
RedisClient.cluster(nodes: 'redis://endpoint.example.com:6379').new_client

# To connect to single endpoint with SSL/TLS (such as Amazon ElastiCache for Redis)
RedisClient.cluster(nodes: 'rediss://endpoint.example.com:6379').new_client

# To connect to NAT-ted endpoint with SSL/TLS (such as Microsoft Azure Cache for Redis)
RedisClient.cluster(nodes: 'rediss://endpoint.example.com:6379', fixed_hostname: 'endpoint.example.com').new_client

# To specify a timeout for "slow" commands (CLUSTER SHARDS, COMMAND)
RedisClient.cluster(slow_command_timeout: 4).new_client

# To specify concurrency settings
RedisClient.cluster(concurrency: { model: :on_demand, size: 6 }).new_client
RedisClient.cluster(concurrency: { model: :pooled, size: 3 }).new_client
RedisClient.cluster(concurrency: { model: :none }).new_client

# The above settings are used by sending commands to multiple nodes like pipelining.
# Please choose the one suited your workloads.

# To reconnect using the original configuration options on error. This can be useful when using a DNS endpoint and the underlying host IPs are all updated
RedisClient.cluster(connect_with_original_config: true).new_client

The following methods are able to be used like redis-client.

• #call
• #call_v
• #call_once
• #call_once_v
• #blocking_call
• #blocking_call_v
• #scan
• #sscan
• #hscan
• #zscan
• #pipelined
• #multi
• #pubsub
• #close

The #scan method iterates all keys around every node seamlessly. The #pipelined method splits and sends commands to each node and aggregates replies. The #multi method supports the transaction feature but you should use a hashtag for your keys. The #pubsub method supports sharded subscriptions. Every interface handles redirections and resharding states internally.

This gem is aimed to keep the compatibility with the public API of redis-client. Hence, there is no plan to add its own public methods beyond the above for the time being.

This gem calls the COMMAND command on startup and decides which node each command should be sent to according to the reply.

• The key positions of the subcommands of a container command such as XINFO STREAM are used if the server reports them. It's available in the Redis 7.0 or later.
• The command tips are used if the server reports them. It's available in the Redis 7.0 or later. A command with request_policy:all_shards is sent to every primary node, and a command with request_policy:all_nodes is sent to every node. The replies are aggregated according to the response_policy tip. It means that a newly added command such as FUNCTION LOAD is routed correctly without waiting for a new release of this gem.

The following cases fall back to the built-in table of this gem:

• The Redis 6.2 or earlier which doesn't report the above information.
• The commands which this gem handles in its own way such as SCAN, KEYS and CLUSTER.
• The request_policy:multi_shard and the request_policy:special tips.
• The response_policy:special tip. Such a command is sent to a single node as before because the aggregation of the replies is undefined. For example, INFO still returns the reply of a single node.

The routing can be overridden per command with the :command_routings option. It takes precedence over both the built-in table and the command tips. The value of each command is the request policy and the response policy which the client should follow, in the same vocabulary as the command tips: request_policy is all_shards or all_nodes, and response_policy is all_succeeded, one_succeeded, agg_sum or omitted. The replies of the nodes are returned as an array if the response policy is omitted. A nil or an empty hash removes the built-in entry of the command, so that the command follows the default resolution: the command tips which the server reports, or the routing by its key. Note that the routing of such a command can vary with the version of the server, because the command tips are what the server reports.

RedisClient.cluster(
  command_routings: {
    'foo' => { request_policy: 'all_shards', response_policy: 'agg_sum' },
    'bar' => { request_policy: 'all_nodes' },
    'dbsize' => nil
  }
).new_client

It affects the direct calls such as #call. The commands inside the #pipelined, #multi and #pubsub blocks are routed by their keys as before. The commands which change the state of a connection such as MULTI, WATCH and SUBSCRIBE can't be overridden.

Also, the option is keyed by a command, not by a subcommand. A container command whose subcommands need different routings such as HIMPORT, whose PREPARE fans out to every primary node but whose SET is routed by its key, can't be expressed with this option. Such a command doesn't need this option in the first place, because this gem follows the key specs and the command tips of the subcommands which the server reports. This option is the escape hatch for the commands without usable command tips.

A subset of commands can be passed multiple keys. In cluster mode, these commands have a constraint that passed keys should belong to the same slot and not just the same node. Therefore, the following error occurs:

$ redis-cli -c mget key1 key2 key3
(error) CROSSSLOT Keys in request don't hash to the same slot

$ redis-cli -c cluster keyslot key1
(integer) 9189

$ redis-cli -c cluster keyslot key2
(integer) 4998

$ redis-cli -c cluster keyslot key3
(integer) 935

For the constraint, Redis cluster provides a feature to be able to bias keys to the same slot with a hash tag.

$ redis-cli -c mget {key}1 {key}2 {key}3
1) (nil)
2) (nil)
3) (nil)

$ redis-cli -c cluster keyslot {key}1
(integer) 12539

$ redis-cli -c cluster keyslot {key}2
(integer) 12539

$ redis-cli -c cluster keyslot {key}3
(integer) 12539

In addition, this gem handles multiple keys without a hash tag in MGET, MSET and DEL commands using pipelining internally automatically. If the first key includes a hash tag, this gem sends the command to the node as is. If the first key doesn't have a hash tag, this gem converts the command into single-key commands and sends them to nodes with pipelining, then gathering replies and returning them.

r = RedisClient.cluster.new_client
#=> #<RedisClient::Cluster 127.0.0.1:6379>

r.call('mget', 'key1', 'key2', 'key3')
#=> [nil, nil, nil]

r.call('mget', '{key}1', '{key}2', '{key}3')
#=> [nil, nil, nil]

This behavior is for higher-level libraries to maintain compatibility with a standalone client. You can exploit this behavior for migrating from a standalone server to a cluster. Although repeated single-key queries are slower than pipelining, pipelined queries are still slower than a single-slot query with multiple keys. Hence, we recommend using a hash tag in this use case for better performance.

This gem supports Redis transactions, including atomicity with MULTI/EXEC, and conditional execution with WATCH. Redis does not support cross-node transactions, so all keys used within a transaction must live in the same key slot. To use transactions, you can use #multi method same as the redis-client:

cli.multi do |tx|
  tx.call('INCR', 'my_key')
  tx.call('INCR', 'my_key')
end

More commonly, however, you will want to perform transactions across multiple keys. To do this, you need to ensure that all keys used in the transaction hash to the same slot; Redis provides a mechanism called hashtags to achieve this. If a key contains a hashtag (e.g. in the key {foo}bar, the hashtag is foo), then it is guaranteed to hash to the same slot (and thus always live on the same node) as other keys which contain the same hashtag.

So, whilst it's not possible in Redis cluster to perform a transaction on the keys foo and bar, it is possible to perform a transaction on the keys {tag}foo and {tag}bar. To perform such transactions on this gem, use the hashtag:

cli.multi do |tx|
  tx.call('INCR', '{user123}coins_spent')
  tx.call('DECR', '{user123}coins_available')
end

# Conditional execution with WATCH can be used to e.g. atomically swap two keys
cli.call('MSET', '{myslot}1', 'v1', '{myslot}2', 'v2')
cli.multi(watch: %w[{myslot}1 {myslot}2]) do |tx|
  old_key1 = cli.call('GET', '{myslot}1')
  old_key2 = cli.call('GET', '{myslot}2')
  tx.call('SET', '{myslot}1', old_key2)
  tx.call('SET', '{myslot}2', old_key1)
end
# This transaction will swap the values of {myslot}1 and {myslot}2 only if no concurrent connection modified
# either of the values

You can early return out of your block with a next statement if you want to cancel your transaction. In this context, don't use break and return statements.

# The transaction isn't executed.
cli.multi do |tx|
  next if some_conditions?

  tx.call('SET', '{key}1', '1')
  tx.call('SET', '{key}2', '2')
end

# The watching state is automatically cleared with an execution of an empty transaction.
cli.multi(watch: %w[{key}1 {key}2]) do |tx|
  next if some_conditions?

  tx.call('SET', '{key}1', '1')
  tx.call('SET', '{key}2', '2')
end

RedisClient::Cluster#multi is aware of redirections and node failures like ordinary calls to RedisClient::Cluster, but because you may have written non-idempotent code inside your block, the block is called once if e.g. the slot it is operating on moves to a different node.

The cluster client internally calls COMMAND and CLUSTER SHARDS commands to operate correctly. Please grant the following permissions.

# The default user is administrator.
cli1 = RedisClient.cluster.new_client

# To create a user with permissions
# Typically, user settings are configured in the config file for the server beforehand.
cli1.call('ACL', 'SETUSER', 'foo', 'ON', '+COMMAND', '+CLUSTER|SHARDS', '+PING', '>mysecret')

# To initialize client with the user
cli2 = RedisClient.cluster(username: 'foo', password: 'mysecret').new_client

# The user can only call the PING command.
cli2.call('PING')
#=> "PONG"

cli2.call('GET', 'key1')
#=> NOPERM this user has no permissions to run the 'get' command (RedisClient::PermissionError)

Otherwise:

RedisClient.cluster(username: 'foo', password: 'mysecret').new_client
#=> Redis client could not fetch cluster information: NOPERM this user has no permissions to run the 'cluster|nodes' command (RedisClient::Cluster::InitialSetupError)

You can use the internal connection pooling feature implemented by redis-client if needed.

# example of docker on localhost
RedisClient.cluster.new_pool(timeout: 1.0, size: 2)
#=> #<RedisClient::Cluster 172.21.0.3:6379, 172.21.0.6:6379, 172.21.0.7:6379>

Please see redis-client.

Please make sure the following tools are installed on your machine.

Please fork this repository and check out the code.

$ git clone git@github.com:your-account-name/redis-cluster-client.git
$ cd redis-cluster-client/
$ git remote add upstream https://github.com/redis-rb/redis-cluster-client.git
$ git fetch -p upstream

Please do the following steps.

• Build a Redis cluster with Docker
• Install gems
• Run basic test cases

## If you use Docker server and your OS is Linux:
$ bundle config set path '.bundle'
$ bundle install --jobs=$(grep process /proc/cpuinfo | wc -l)
$ docker compose up
$ bundle exec rake test

## else:
$ docker compose --profile ruby up
$ docker compose --profile ruby exec ruby bundle install
$ docker compose --profile ruby exec ruby bundle exec rake test

You can see more information in the YAML file for GitHub Actions.

This library might help you if you want to migrate your Redis from a standalone server to a cluster. Here is an example code.

# frozen_string_literal: true

require 'bundler/inline'

gemfile do
  source 'https://rubygems.org'
  gem 'redis-cluster-client'
end

src = RedisClient.config(url: ENV.fetch('REDIS_URL')).new_client
dest = RedisClient.cluster(nodes: ENV.fetch('REDIS_CLUSTER_URL')).new_client
node = dest.instance_variable_get(:@router).instance_variable_get(:@node)

src.scan do |key|
  slot = ::RedisClient::Cluster::KeySlotConverter.convert(key)
  node_key = node.find_node_key_of_primary(slot)
  host, port = ::RedisClient::Cluster::NodeKey.split(node_key)
  src.blocking_call(10, 'MIGRATE', host, port, key, 0, 7, 'COPY', 'REPLACE')
end

Further optimization is needed to perform well in production environments with large numbers of keys. Also, it should handle errors.

• https://redis.io/docs/reference/cluster-spec/
• redis/redis-rb#1070
• redis/redis#8948
• valkey-io/valkey#384
• https://github.com/antirez/redis-rb-cluster
• https://twitter.com/antirez
• https://bsky.app/profile/antirez.bsky.social
• http://antirez.com/latest/0
• https://www.youtube.com/@antirez
• https://www.twitch.tv/thetrueantirez/