Redis Streams Scaler

[abhirockzz]

Redis Streams, a data type introduced with Redis 5.0 is an append-only log data structure. One of its features includes "consumer groups" that allows a group of clients to co-operate consuming a different portion of the same stream of messages.

This scaler can check the Pending Entries List (see XPENDING https://redis.io/commands/xpending) for a specific Consumer Group of a Redis Stream and scale-out the Deployment as per the count returned (by XPENDING command). If you have a Redis Streams consumer application that is unable to keep up with the rate of production, KEDA can help by auto-scaling the no. of instances (part of the same consumer group) so that it can start sharing the processing workload.

Checklist

• A PR is opened to update the documentation on https://github.com/kedacore/keda-docs
• Commits are signed with Developer Certificate of Origin (DCO)
• Tests have been added

Fixes #746

[tomkerkhove]

Thanks man - Let me know when there are docs to review!

[abhirockzz]

Thanks man - Let me know when there are docs to review!

Docs PR - kedacore/keda-docs#147

[zroubalik]

I don't have experience with Redis Streams, but the code looks good

[zroubalik]

@abhirockzz do you think it would be possible to add e2d tests for Redis Streams Scaler?

https://github.com/kedacore/keda/tree/master/tests#adding-tests

[abhirockzz]

@abhirockzz do you think it would be possible to add e2d tests for Redis Streams Scaler?

https://github.com/kedacore/keda/tree/master/tests#adding-tests

will try my best @zroubalik :) let me look into this

[zroubalik]

@abhirockzz just FYI, similar tests for RabbitMQ were just added, so you can check them out: #810

[abhirockzz]

@abhirockzz just FYI, similar tests for RabbitMQ were just added, so you can check them out: #810

Thanks! I am not familiar with TS, so an existing impl should be helpful

[zroubalik]

@abhirockzz any update on this please? Or do you need any help?

[abhirockzz]

@abhirockzz any update on this please? Or do you need any help?

sorry for the delay. I am working on this and will keep you posted

[zroubalik]

@abhirockzz no worries, I was just concerned whether you need some help. Thanks!

[zroubalik]

@idvoretskyi @ahmelsayed FOSSA failed here for missing FOSSA API key again? 🤷‍♂️

[idvoretskyi]

@zroubalik the same issue, I assume. The secrets are not passed to the PR from fork.

[zroubalik]

@idvoretskyi so we should do something about this, failing checks in PR are not ideal. How is the FOSSA done in other projects?

[idvoretskyi]

Ref on the FOSSA check - #808 (comment)