Resources and autoscaling fields in BentoDeployment CRD meaning.

[dzhelonkin]

There are spec.autoscaling, spec.resources and spec.runners[].resources, spec.runners[].autoscaling fields. It is not clear for me what spec.autoscaling, spec.resources control (doc: https://docs.bentoml.org/projects/yatai/en/latest/concepts/bentodeployment_crd.html) . Are spec.autoscaling and spec.resources upper bound for spec.runners[].resources and spec.runners[].autoscaling respectively?

[parano]

Hi @dzhelonkin - the spec.resources means the resource allocated for the API servers, essentially the pod spec for running users' service API code. Whereas spec.runners[].resources are pod specs for Runners used in the service. E.g. if you have a model that uses GPU, you should put the GPU resource under spec.runners[].resources. The same applies to the autoscaling field. Note that when deploying BentoDeployment CR with Yatai, runners and API server will be distributed across different pods allowing it to scale and utilize resources more efficiently.

[dzhelonkin]

I got it, thanks for the reply