Scaling of SeldonSeployments

[sasvaritoni]

Seldon 4.0

When I change the replicas field in a SeldonDeployment e.g. from 3 to 4, then I can see that basically 4 new pods are created and the old 3 are terminated.
This behavior seems very strange.

When I do a scaling operation on the deployment level (which belongs to the SeldonDeployment), it is handled smoothly, i.e. only 1 new pod is created in order to achieve the new number of replicas. Although I guess that this kind of scaling should not be performed manually, only via the SeldonDeployment resource.

What is the reason for this? Am I missing some config setting?

Thanks,
Toni

[zwerg19]

Hi,

I saw similar issue.

In case of kubernetes deployments if we use updateStrategy Rolling it works the following way:

• Scaling from 100 to 101 instances it creates 1 new instance and do not replace the old ones
• Changing image version replace the pods according to Rolling update strategy

In case of SeldonDeployment with single model (in 100 instances):

• Scaling from 100 to 101 instances it creates 1 new instance and replace the old ones
• Changing image version (deployment with 100 instances) starts 100 new instances and till they are not ready we have the 100 old pods, so we have 200 pods in the system.

Is it the normal behavior, or I missed something during configuration?

[ryandawsonuk]

Related to #1078

[ryandawsonuk]

I suspect this fits the pattern described in #1110. The predictor spec is fed into all the pods for the engine to read.If that changes, then it prompts a rolling update. And the replicas are part of the predictor spec so this makes sense.

A fix could be to omit the number of replicas from the spec that is fed to the engine as the engine doesn't really need to know that bit.

[zwerg19]

Yes it seems that the value of env variable ENGINE_PREDICTOR changes at every replica change
I checked it with seldon version 1.0.1 and it still reproducible.

[ukclivecox]

The number of replicas should not be in the ENGINE_PREDICTOR. And also we should indeed look at the rolling update strategy. This may be because the Deployment name changes as it based on the hash of the spec. This needs to be reviewed.