Use approximate queue length for azure queue scaler

[moria97]

Use the approximate queue length instead of visible messages, see #4002

Checklist

• When introducing a new scaler, I agree with the scaling governance policy
• I have verified that my change is according to the deprecations & breaking changes policy
• Tests have been added
• Changelog has been updated and is aligned with our changelog requirements
• A PR is opened to update our Helm chart (repo) (if applicable, ie. when deployment manifests are modified)
• A PR is opened to update the documentation on (repo) (if applicable)
• Commits are signed with Developer Certificate of Origin (DCO - learn more)

Fixes #4002

Relates to #

[tomkerkhove]

By removing the visible count you are not actually achieving what you are suggesting in #4002

[moria97]

By removing the visible count you are not actually achieving what you are suggesting in #4002

Removing the visible count will return the approximate message count directly, which contains messages that are queued and being processed. Not sure if there is anything wrong? @tomkerkhove

[v-shenoy]

LGTM

[v-shenoy]

/run-e2e azure_queue
Update: You can check the progress here

[tomkerkhove]

I have misread the issue, no worries.

[tomkerkhove]

Can you add an entry in the changelog please?

I've pinged @ahmelsayed privately as well to see if we can remember why we added that.

[tomkerkhove]

@ahmelsayed pointed out that this was changed in #2613 by @RamCohen as an optimization.

I think the optimization is still OK to have and we can skip this PR but I don't have a strong opinion.

[moria97]

@tomkerkhove This optimization looks make sense to me. But the issue is not related to the optimization, but the visible message count used to count queue length.

Let's say "queue length" parameter is 1, there are 20 messages and scaled out to 20 instances and pulled message from queue, the queue size returned is 0 （visible message account）, but the actual value should be 20 (approximate queue length).

[zroubalik]

@moria97 @tomkerkhove @v-shenoy so what is the status of this PR?

[antonio-policastro]

I'm shipping a custom version of this scaler in production since v2.6.1. My custom versions have the exact same code as in this pull request, and they have been working perfectly fine for the past year. ApproximateMessageCount is the way to go.
There are multiple issues about this problem(eg: #2385 #3555).

The optimization done with #2613 will be useless, because we should always call GetProperties to get the ApproximateMessageCount.

I hope this will be merged soon.
Thanks

[moria97]

@tomkerkhove @zroubalik Can you take a look at this pr?

[v-shenoy]

I approved this earlier also. I think we can (should) merge this.

@tomkerkhove @JorTurFer

[JorTurFer]

LGTM

[tomkerkhove]

/run-e2e azure_queue

Update: You can check the progress here

[JorTurFer]

/run-e2e azure_queue*
Update: You can check the progress here

[wallflower762]

Thanks team, this is great!