How different PodAutoScaler configure RPS in Autoscale

[andyyin]

In what area(s)?

/area autoscale

#5141

switch pa.Metric() { case autoscaling.RPS: total = config.RPSTargetDefault tu = config.TargetUtilization default: // Concurrency is used by default total = float64(pa.Spec.ContainerConcurrency) // If containerConcurrency is 0 we'll always target the default. if total == 0 { total = config.ContainerConcurrencyTargetDefault } tu = config.ContainerConcurrencyTargetFraction }

Ask your question here:

Why RPS is handled differently than Concurrency. If so, different podAutoscaler, you can not set rps separately

[andyyin]

@yanweiguo

[markusthoemmes]

RPS is not subject to the containerConcurrency setting so in this case, we don't handle anything related to that in the RPS branch.

The diff of ContainerConcurrencyTargetFraction vs. TargetUtilization is simple that we're going to deprecate the former for the latter eventually. Semantically they are the same.

Does that answer your questions?

[yanweiguo]

Yeah as @markusthoemmes said, TargetUtilization is going to replace ContainerConcurrencyTargetFraction for concurrency usage.

However I have questions: are we going to support multiple metrics for KPA? How do we support custom metrics autoscaling? We may have to change the knobs.

[andyyin]

RPS is not subject to the containerConcurrency setting so in this case, we don't handle anything related to that in the RPS branch.

The diff of ContainerConcurrencyTargetFraction vs. TargetUtilization is simple that we're going to deprecate the former for the latter eventually. Semantically they are the same.

Does that answer your questions?

No, maybe I didn't explain it. I mean the current value (total and tu) of RPS is from autoscaler.Config. This configuration is a global configuration. All Revisons are the same value. How do I set different values ​​for different Revision settings?

[andyyin]

Yeah as @markusthoemmes said, TargetUtilization is going to replace ContainerConcurrencyTargetFraction for concurrency usage.

However I have questions: are we going to support multiple metrics for KPA? How do we support custom metrics autoscaling? We may have to change the knobs.

This is another question for me. In the actual production environment, multiple metrics for KPA will be more reasonable.

[yanweiguo]

RPS is not subject to the containerConcurrency setting so in this case, we don't handle anything related to that in the RPS branch.
The diff of ContainerConcurrencyTargetFraction vs. TargetUtilization is simple that we're going to deprecate the former for the latter eventually. Semantically they are the same.
Does that answer your questions?

No, maybe I didn't explain it. I mean the current value (total and tu) of RPS is from autoscaler.Config. This configuration is a global configuration. All Revisons are the same value. How do I set different values ​​for different Revision settings?

You can use the following annotations on revision to override:

autoscaling.knative.dev/metric: rps 
autoscaling.knative.dev/target: 200 
autoscaling.knative.dev/targetUtilizationPercentage: 70 

The override happens here:

serving/pkg/reconciler/autoscaling/resources/target.go

Line 58 in 7b6e76d

target = math.Max(1, math.Min(target, annotationTarget*tu))

[andyyin]

RPS is not subject to the containerConcurrency setting so in this case, we don't handle anything related to that in the RPS branch.
The diff of ContainerConcurrencyTargetFraction vs. TargetUtilization is simple that we're going to deprecate the former for the latter eventually. Semantically they are the same.
Does that answer your questions?

No, maybe I didn't explain it. I mean the current value (total and tu) of RPS is from autoscaler.Config. This configuration is a global configuration. All Revisons are the same value. How do I set different values ​​for different Revision settings?

You can use the following annotations on revision to override:

autoscaling.knative.dev/metric: rps 
autoscaling.knative.dev/target: 200 
autoscaling.knative.dev/targetUtilizationPercentage: 70 

The override happens here:

serving/pkg/reconciler/autoscaling/resources/target.go

Line 58 in 7b6e76d

target = math.Max(1, math.Min(target, annotationTarget*tu))

I understand this, but math.Min(target, annotationTarget*tu) has a limit, meaning that the final value may not be annotationTarget*tu, but config.RPSTargetDefault*config.TargetUtilization. This means that config.RPSTargetDefault and config.TargetUtilization are difficult to configure a suitable value. I think RPSTargetDefault is more suitable in pa.Spec, similar to pa.Spec.ContainerConcurrency.

[andyyin]

@yanweiguo

[markusthoemmes]

@andyyin you are right indeed! This is a bug! See #5991 for a potential fix.