Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use HPA to support to scale to zero using arbiter #290

Open
nkwangleiGIT opened this issue Nov 26, 2023 · 3 comments
Open

Use HPA to support to scale to zero using arbiter #290

nkwangleiGIT opened this issue Nov 26, 2023 · 3 comments
Assignees
@0xff-dev
Milestone
v0.5.0

Comments

@nkwangleiGIT
Copy link
Contributor

Let's see if we can use arbiter for model serving autoscaling, and maybe scheduling later.

kube-arbiter/arbiter#164
@nkwangleiGIT
Copy link
Contributor Author

@0xff-dev you can have a try for this one.

@bjwswang
Copy link
Collaborator

I think we should work on this in next version.

@nkwangleiGIT nkwangleiGIT added this to the v0.2.0 milestone Dec 23, 2023
@nkwangleiGIT
Copy link
Contributor Author

refer to Ray auto scaler to check if it's a better solution:
https://docs.ray.io/en/master/serve/autoscaling-guide.html

@nkwangleiGIT nkwangleiGIT modified the milestones: v0.2.0, v0.5.0 Jan 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@0xff-dev 0xff-dev

Labels
None yet
Projects
None yet
Milestone
v0.5.0
Development

No branches or pull requests
3 participants
@nkwangleiGIT @0xff-dev @bjwswang