-
Notifications
You must be signed in to change notification settings - Fork 1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use approximate queue length for azure queue scaler #4003
Use approximate queue length for azure queue scaler #4003
Conversation
Signed-off-by: Yue Fei <yufei@microsoft.com>
8d0366e
to
d18575f
Compare
By removing the visible count you are not actually achieving what you are suggesting in #4002 |
Removing the visible count will return the approximate message count directly, which contains messages that are queued and being processed. Not sure if there is anything wrong? @tomkerkhove |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
/run-e2e azure_queue |
I have misread the issue, no worries. |
Can you add an entry in the changelog please? I've pinged @ahmelsayed privately as well to see if we can remember why we added that. |
Signed-off-by: Yue Fei <59813791+moria97@users.noreply.github.com>
@ahmelsayed pointed out that this was changed in #2613 by @RamCohen as an optimization. I think the optimization is still OK to have and we can skip this PR but I don't have a strong opinion. |
@tomkerkhove This optimization looks make sense to me. But the issue is not related to the optimization, but the visible message count used to count queue length. Let's say "queue length" parameter is 1, there are 20 messages and scaled out to 20 instances and pulled message from queue, the queue size returned is 0 (visible message account), but the actual value should be 20 (approximate queue length). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@moria97 @tomkerkhove @v-shenoy so what is the status of this PR?
I'm shipping a custom version of this scaler in production since v2.6.1. My custom versions have the exact same code as in this pull request, and they have been working perfectly fine for the past year. ApproximateMessageCount is the way to go. The optimization done with #2613 will be useless, because we should always call GetProperties to get the ApproximateMessageCount. I hope this will be merged soon. |
Signed-off-by: Yue Fei <59813791+moria97@users.noreply.github.com>
@tomkerkhove @zroubalik Can you take a look at this pr? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I approved this earlier also. I think we can (should) merge this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
/run-e2e azure_queue Update: You can check the progress here |
Signed-off-by: Yue Fei <59813791+moria97@users.noreply.github.com>
Signed-off-by: Yue Fei <59813791+moria97@users.noreply.github.com>
/run-e2e azure_queue* |
Thanks team, this is great! |
Use the approximate queue length instead of visible messages, see #4002
Checklist
Fixes #4002
Relates to #