Test Consumption Plan Support

[sebastianburckhardt]

The current implementation already does automatic load balancing over the connected nodes, using the the EventProcessorHost mechanism provided by EventHubs. Also, there is some preliminary logic drafted in ScalingMonitor.cs for making scale recommendations. However, this is not currently hooked up to any infrastructure that performs the scaling.

Eventually I think we would like to be able to support all of the following:

1. Provide auto-scaling for premium plans
2. Provide auto-scaling for consumption plans
3. Provide auto-scaling for KEDA deployments

[sebastianburckhardt]

Is now supported for elastic premium plans.

Still need to update the documentation.

[sebastianburckhardt]

Ok, next step is to support consumption plan.

[sebastianburckhardt]

Waiting on @bachuv who has started to get the existing scale controller logic ported to the consumption plan.

[itpropro]

The official docs state that Consumption Plan support will be introduced with GA. We are already after GA with a 1.x.x version (right?), can this issue be closed then @sebastianburckhardt ?

[sebastianburckhardt]

We are currently testing consumption plan rollout.

[ShubhamGargCTH]

@sebastianburckhardt is this available on the consumption plan now?

[sebastianburckhardt]

It is being rolled out, it is already available in some regions but not all of them. What region are you in?

[ShubhamGargCTH]

i am in Canada Central

[sebastianburckhardt]

I believe the feature is now enabled for Canada Central, as of Thursday (12/29/2022).

[ShubhamGargCTH]

Thanks for the update.. we are seeing the below errors intermittently.
The listener for function '***' was unable to start. Encountered error while fetching the list of EventHub PartitionIds The operation did not complete within the allocated time 00:00:00 for object out-connection39.
Any idea what could be causing this.

let me know if you want me to create a seprate issue for this.

[sebastianburckhardt]

A separate issue would make sense, yes.

The error message indicates that the event hubs sdk was unable to fetch the partition ids from the event hubs namespace.

Some guesses that come to mind:

• The SDK sometimes throws spurious exceptions (network issues) that resolve on their own after a while. Do these errors persist?
• Did you create EventHubs namespace and have sufficient access permissions to connect to it?

[sebastianburckhardt]

We are now officially GA and available worldwide. I will close this issue now, feel free to open another if you have more questions.

[nitinkale]

@sebastianburckhardt does netherite supports auto scaling with KEDA ? If yes then could you please share document\sample application ?

[sebastianburckhardt]

Unfortunately, we don't have a KEDA scaler for Netherite.

[nitinkale]

KEDA supports Eventhub . Is it not possible to scale orchestrator and workers based on the metrics from Eventhub ?

[sebastianburckhardt]

It won't scale accurately since the number of unprocessed messages in event hub (which is how I assume event hubs does the scaling) does not directly correspond to the amount of work in a Netherite task hub (number of orchestrations/entities/activities/timers that need to be processed).