Economy of scale

[yaron2]

From what I can see, for every new deployment that needs to be scaled down to zero, there are two additional deployments that occur:

1. Interceptor
2. External scaler

As these deployments are not scaled down to zero and take resources from the cluster, the benefits of scale to zero are greatly reduced at best (if the user deployment takes equal or more resources than the interceptor and external scaler combined) or completely negated at worst (if the user deployment takes less resources than the interceptor and external scaler combined).

In addition, when the app is not scaled to zero, the interceptor and external scaler when deployed for every app will take resources from the cluster that could otherwise be used to schedule new deployments, and are sitting idle unless scale to zero occurs.

These insights rely on the design doc and my understanding of the code after a brief overview, so excuse misunderstandings.

/cc @tomkerkhove

[arschles]

@yaron2 are you interested in making a change? It would be certainly possible to make the interceptor and scaler multi-tenant if you're interested in that.

[yaron2]

I am certainly interested in that, but I'm not sure it's "me" being interested in that as much as its about every user should be interested in that, because without a multi-tenant interceptor and external scaler I don't see any real benefit to running the HTTP add-on, again, unless I am missing something obvious?

[arschles]

Autoscaling your HTTP workloads would be one major benefit that persists even though you can't (currently) scale the interceptor or external scaler to zero. Regardless, yes, it would certainly be more resource efficient to run multi-tenant interceptors and scalers.

[yaron2]

Autoscaling your HTTP workloads would be one major benefit that persists even though you can't (currently) scale the interceptor or external scaler to zero. Regardless, yes, it would certainly be more resource efficient to run multi-tenant interceptors and scalers.

Yeah I'm talking specifically about the scale to zero part. I didn't see 1:N autoscaling so far, but that can be achieved with the Prometheus scaler today, albeit in a more cumbersome way.

[arschles]

got it. yea, if you're looking to truly scale resources to zero on your cluster, you couldn't use the HTTP addon, yea. Would you be interested in helping design and/or build these multi-tenant components?

[yaron2]

got it. yea, if you're looking to truly scale resources to zero on your cluster, you couldn't use the HTTP addon, yea. Would you be interested in helping design and/or build these multi-tenant components?

I could probably help with design, for sure.

[tomkerkhove]

I haven't thought about it from that angle and that definitely makes sense, we should go multi-tenant for sure.

Yeah I'm talking specifically about the scale to zero part. I didn't see 1:N autoscaling so far, but that can be achieved with the Prometheus scaler today, albeit in a more cumbersome way.

@yaron2 We want to support 1:N scaling as well because we don't want to force people to use Prometheus. For example, my customers always use Azure Monitor and should not have to run Prometheus because of that.

[yaron2]

I haven't thought about it from that angle and that definitely makes sense, we should go multi-tenant for sure.

Yeah I'm talking specifically about the scale to zero part. I didn't see 1:N autoscaling so far, but that can be achieved with the Prometheus scaler today, albeit in a more cumbersome way.

@yaron2 We want to support 1:N scaling as well because we don't want to force people to use Prometheus. For example, my customers always use Azure Monitor and should not have to run Prometheus because of that.

Yeah I understand, I totally support doing 1:N scaling. we should just make sure this is done in a way that isn't too costly on cluster resources.

[arschles]

@yaron2 I've begun a draft design doc: https://hackmd.io/@arschles/mutitenant-keda-http-addon

there are a few places where it's a bit rough - the biggest one is pushing routing table updates to interceptors and verifying that they've updated their in-memory copy.

let me know what you think

[khaosdoctor]

@ajanth97 Raised a concern about the isActive methods in the external scaler interface that will probably be addressed in this too. I'm just waiting for an issue to be created so we can track that down

[khaosdoctor]

@yaron2 I've begun a draft design doc: hackmd.io/@arschles/mutitenant-keda-http-addon

there are a few places where it's a bit rough - the biggest one is pushing routing table updates to interceptors and verifying that they've updated their in-memory copy.

let me know what you think

@arschles added a few comments on the document too, see if they're helpful

[arschles]

Thanks @khaosdoctor !