Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug Report: VTOrc is not setting the correct timeout for RPC calls which can get stuck #15990

Closed
GuptaManan100 opened this issue May 22, 2024 · 0 comments · Fixed by #15991
Closed
Labels
Component: VTorc Vitess Orchestrator integration Type: Bug

Comments

@GuptaManan100
Copy link
Member

Overview of the Issue

Description

VTOrc is not setting the correct timeouts for the RPC calls. Since RPC calls can take a long time to run, and can sometimes even get stuck while running queries on MySQL, all RPC calls should be running with a context that expires after RemoteOperationTimeout duration.

Reproduction Steps

Run VTOrc recovery that triggers fixReplica on a tablet that gets stuck on setting read only. This will cause VTOrc to indefinitely hang and unable to fix any other issues in the cluster.

Binary Version

main

Operating System and Environment details

-

Log Fragments

No response

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Component: VTorc Vitess Orchestrator integration Type: Bug
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant