Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve Deckard integration with Redis Clusters #13

Open
lucasoares opened this issue Feb 16, 2023 · 0 comments
Open

Improve Deckard integration with Redis Clusters #13

lucasoares opened this issue Feb 16, 2023 · 0 comments
Labels
help wanted Extra attention is needed question Further information is requested

Comments

@lucasoares
Copy link
Member

lucasoares commented Feb 16, 2023

Deckard uses redis as a caching mechanism to store queues, priorities, etc. If the entire data is wiped, Deckard will reconstruct it and stay resillient.

Until then, even with thousands of requests per second and millions of messages, it was not necessary to use a Redis Cluster to scale. A single redis node was able to support very high workloads. This can be different for user's environments as they may need it.

A simple solution for different usages is to deploy different instances of deckard for each domain using a dedicated redis instance. This will be enough for the most use cases.

Since we use lua scripts to perform atomic operations in Redis we need to create integration tests using Redis Clusters to ensure that our lua scripts can work with this type of environment, as the documentation states:

Important: to ensure the correct execution of scripts, both in standalone and clustered deployments, all names of keys that a script accesses must be explicitly provided as input key arguments. The script should only access keys whose names are given as input arguments. Scripts should never access keys with programmatically-generated names or based on the contents of data structures stored in the database.

We may need to adapt or use hashing to be able to atomic update different keys to continue with the desired behavior we currently have for a single-node Redis instance.

@lucasoares lucasoares added help wanted Extra attention is needed question Further information is requested labels Feb 16, 2023
@lucasoares lucasoares changed the title Ensure that Deckard works with Redis Clusters Do Deckard works with Redis Clusters? Feb 16, 2023
@lucasoares lucasoares changed the title Do Deckard works with Redis Clusters? Does Deckard works with Redis Clusters? Feb 16, 2023
@lucasoares lucasoares changed the title Does Deckard works with Redis Clusters? Does Deckard work with Redis Clusters? Feb 16, 2023
@lucasoares lucasoares changed the title Does Deckard work with Redis Clusters? Improve Deckard integration with Redis Clusters Jun 23, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted Extra attention is needed question Further information is requested
Projects
None yet
Development

No branches or pull requests

1 participant