Does the redis-py module work with Redis in Cluster mode?
Question:
I’m trying to use redis-py with redis in cluster mode, but I can’t get it to work. I see that redis-py-cluster works, but I have a preference for redis-py as I have been using it and it’s a recommended client.
Answers:
redis-py does not support cluster mode. Clustering has totally different architecture to serve the purpose of horizontal scalability. HA (High Availability) was not a priority in its design. Therefore you can not use one client for the other.
redis-py-cluster seem to have ongoing development / support, and it’s based on redis.py. The client page you linked was not for redis cluster. “redis-py-cluster” is mentioned on redis cluster page (look for “Playing with the cluster”): https://redis.io/topics/cluster-tutorial
Asides from clustering, Redis has sentinel supported setup to provide HA, which redis-py does support.
You can use redis-py in redis cluster, but as different keys are partitioned to different nodes, you need to compute (by crc16/crc32 hash function) which cluster handles which keys.
In order to fully utilize “cluster mode”, where you don’t care about keys’ location, you need to implement “client side partitioning” and “query routing” which redis-py-cluster provides. (https://redis.io/topics/partitioning)
One major disadvantage of redis-py-cluster is that it does not provide a solution for the atomic operation in “pipeline + transaction”
According to redis-py documentation:
redis-py is now supports cluster mode and provides a client for Redis Cluster.
Notice that the redis-py added this feature in version 4.1.0 which has no stable release right now. if you want to have it installed, you should use the command below:
pip install redis==4.1.0-rc1
Maybe when you are reading this answer, it is stable! So just install without the -rc1 postfix.
You can connect to your redis-cluster like the following:
>>> from redis.cluster import RedisCluster as Redis
>>> rc = Redis(host='localhost', port=6379)
>>> print(rc.get_nodes())
[[host=127.0.0.1,port=6379,name=127.0.0.1:6379,server_type=primary,redis_connection=Redis<ConnectionPool<Connection<host=127.0.0.1,port=6379,db=0>>>], [host=127.0.0.1,port=6378,name=127.0.0.1:6378,server_type=primary,redis_connection=Redis<ConnectionPool<Connection<host=127.0.0.1,port=6378,db=0>>>], [host=127.0.0.1,port=6377,name=127.0.0.1:6377,server_type=replica,redis_connection=Redis<ConnectionPool<Connection<host=127.0.0.1,port=6377,db=0>>>]]
I’m trying to use redis-py with redis in cluster mode, but I can’t get it to work. I see that redis-py-cluster works, but I have a preference for redis-py as I have been using it and it’s a recommended client.
redis-py does not support cluster mode. Clustering has totally different architecture to serve the purpose of horizontal scalability. HA (High Availability) was not a priority in its design. Therefore you can not use one client for the other.
redis-py-cluster seem to have ongoing development / support, and it’s based on redis.py. The client page you linked was not for redis cluster. “redis-py-cluster” is mentioned on redis cluster page (look for “Playing with the cluster”): https://redis.io/topics/cluster-tutorial
Asides from clustering, Redis has sentinel supported setup to provide HA, which redis-py does support.
You can use redis-py in redis cluster, but as different keys are partitioned to different nodes, you need to compute (by crc16/crc32 hash function) which cluster handles which keys.
In order to fully utilize “cluster mode”, where you don’t care about keys’ location, you need to implement “client side partitioning” and “query routing” which redis-py-cluster provides. (https://redis.io/topics/partitioning)
One major disadvantage of redis-py-cluster is that it does not provide a solution for the atomic operation in “pipeline + transaction”
According to redis-py documentation:
redis-py is now supports cluster mode and provides a client for Redis Cluster.
Notice that the redis-py added this feature in version 4.1.0 which has no stable release right now. if you want to have it installed, you should use the command below:
pip install redis==4.1.0-rc1
Maybe when you are reading this answer, it is stable! So just install without the -rc1 postfix.
You can connect to your redis-cluster like the following:
>>> from redis.cluster import RedisCluster as Redis
>>> rc = Redis(host='localhost', port=6379)
>>> print(rc.get_nodes())
[[host=127.0.0.1,port=6379,name=127.0.0.1:6379,server_type=primary,redis_connection=Redis<ConnectionPool<Connection<host=127.0.0.1,port=6379,db=0>>>], [host=127.0.0.1,port=6378,name=127.0.0.1:6378,server_type=primary,redis_connection=Redis<ConnectionPool<Connection<host=127.0.0.1,port=6378,db=0>>>], [host=127.0.0.1,port=6377,name=127.0.0.1:6377,server_type=replica,redis_connection=Redis<ConnectionPool<Connection<host=127.0.0.1,port=6377,db=0>>>]]