-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Adds support to use regex with HTTP protocol in RabbitMQ Scaler #1957
Conversation
Signed-off-by: jorturfer <jorge_turrado@hotmail.es>
Could you explain the use case for this? Looks like your aim is to scale something consuming from multiple queues? |
Sure! For example, in our use case we have more than 1 queue (near of 70) that are consumed by one hub to do some stuffs. You can imagine it as a "central hub". In this case, you could need to scale in base of all the queues instaed 1. |
Yeah, the use case seems reasonable just wondering the best way to expose it. This hardwires taking the highest values out of any returned queue which I think should probably be fine? Might be some use cases around "mean value" but nothing super obvious comes to mind. And we should make sure that activating it in AMQP mode is a hard error so we don't get silent "it's just not working" behavior. |
You are right, I will update the code to raise an explicit error in case of invalid configuration. Related with the max value or the mean value, I don't have any strong about this point, I did with max value because the rate can't be usable to get the mean value and the maximum yes. I don't have too much time so I will do little changes during the week. I will stay tuned to your comments :) |
Signed-off-by: jorturfer <jorge_turrado@hotmail.es>
Signed-off-by: jorturfer <jorge_turrado@hotmail.es>
Signed-off-by: jorturfer <jorge_turrado@hotmail.es>
Signed-off-by: jorturfer <jorge_turrado@hotmail.es>
Signed-off-by: jorturfer <jorge_turrado@hotmail.es>
Signed-off-by: jorturfer <jorge_turrado@hotmail.es>
Signed-off-by: Jorge Turrado <jorge.turrado@docplanner.com>
Signed-off-by: Jorge Turrado <jorge.turrado@docplanner.com>
Could you approve the CI workflow please? I think that is fixed (at least in my own repo) |
For multi-queue use cases, we add multiple triggers for each queue which will have the same behavior of scaling at the maximum queue size or publish rate, but it will expose each of the scaling metrics in the HPA (instead of the external metric server collapsing it to one metric for the HPA). I don't know that the current setup would support querying the metrics by regex, reporting each of those metrics, then updating the HPA with a metric for each observed queue (or only when there's a change). This would give visibility through the HPA what metrics are being evaluated for scaling decisions. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
Signed-off-by: Jorge Turrado <jorge.turrado@docplanner.com>
Signed-off-by: Jorge Turrado <jorge.turrado@docplanner.com>
Signed-off-by: Jorge Turrado <jorge.turrado@docplanner.com>
Hi @zroubalik |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, @JorTurFer thanks for the contribution!
…core#1957) Signed-off-by: Jorge Turrado <jorge.turrado@docplanner.com> Signed-off-by: nilayasiktoprak <nilayasiktoprak@gmail.com>
Checklist
This PR adds support to use regex as a queue selector in RabbitMQ Scaller. With this changes, complex cases with more than 1 queue in the application topology are covered and can be scaled with RabbitMQ Scaler
Closes #743