-
Notifications
You must be signed in to change notification settings - Fork 228
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Upgrade RayJob API to v1 #1802
Upgrade RayJob API to v1 #1802
Conversation
✅ Deploy Preview for kubernetes-sigs-kueue ready!
To edit notification comments on pull requests, go to your Netlify site configuration. |
30c62ea
to
0414b33
Compare
Hey, apologies if I'm jumping the gun here, but I'm currently trying to get v1 RayJobs to work with Kueue - I pulled down your PR changes, and installed Kueue through helm, but I'm getting an error whenever I try to create a new RayJob now:
Looking at the Kueue controller logs, it seems like the mutating (and I assume validating) webhook is still for v1alpha1 RayJobs:
Am I doing something wrong here? Possibly installing Kueue in the wrong way to utilize these changes? |
No pb at all :) Thanks a lot for testing this!
That How exactly it is you've tested it? Could you double check the deployment uses the container image that's built with the changes from that PR? |
Ah whoops - realizing now that just running I ran
I'm guessing I should build the new image locally and use that? Are there instructions for testing anywhere? Is |
Oof nevermind, sorry just found this: https://kueue.sigs.k8s.io/docs/installation/#build-and-install-from-source I'll try this again |
Ok, I built & installed from source and confirmed I'm now getting the v1 RayJob mutating & admission webhooks.
|
That error message happens when optimistic locking fails during an update, but the operation is simply retried so it's not an issue. For the RayJob and the associated workload resource, checking their statuses should provide the reason why it's not started / running. |
RayJob is stuck in
Workload status doesn't show anything out of the ordinary:
RayCluster is showing as suspended, though:
Also seeing these errors in the kueue-controller logs (in addition to that other one above):
|
@mbzomowski I've been able to reproduce. That issue is actually unrelated to that PR and should be fixed with #1844. |
Rebased with the new changes & everything works! Thanks so much @astefanutti! |
@mbzomowski great, thanks for the feedback, and the thorough testing! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
/lgtm
LGTM label has been added. Git tree hash: eeb318d227895819982a137aae623f27907664e9
|
@alculquicondor @tenzen-y @andrewsykim @kevin85421 Do we want to get this in the next Kueue minor release, now that KubeRay v1.1.0 has been released? |
Based on this discussion: #1435 (comment) I agree with the migration of KubeRay v1 in Kueue v0.7. |
sgtm, please rebase And put |
PR rebased, and release notes updated. PTAL, thanks. |
@astefanutti Could you address CI errors? |
/retest |
@tenzen-y retesting as I think these were transient failures. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you!
/lgtm
/approve
@astefanutti Could you update documentation and examples in a follow-up?
LGTM label has been added. Git tree hash: 261fce03970d3454a8a9b35fb6516b8050daff94
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: astefanutti, tenzen-y The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
@tenzen-y as far as I can see, the documentation and examples have been updated in the scope of that PR already, unless you have specific changes in mind? On the other, while giving it another look, I've found another follow-up that's covered by #1941. |
Sorry I'm late to review this, @astefanutti do we also want to update the go module version of kuberay? I ran into some corner cases where webhook logic drops fields because the imported type in Kueue doesn't have all the needed fields. |
We should update to v1.1, but make sure it's backwards compatible with a cluster on v1.0 |
Oh ignore me, looks like it's using v1.1 already :) |
Oops, you're right. |
No pb, just wanted to make sure I was not missing anything! |
What this PR does / why we need it:
KubeRay has promoted the RayJob API from v1alpha1 to v1 with KubeRay v1.0.0. While the v1alpha1 version is still served in KubeRay v1.1.0, it'll be removed in v1.2.0.
Which issue(s) this PR fixes:
Fixes #1335.
Special notes for your reviewer:
Supersedes #1435.
Does this PR introduce a user-facing change?