Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

roachtest: auto retry operations when failing due to ssh-related issues #73542

Closed
healthy-pod opened this issue Dec 7, 2021 · 4 comments · Fixed by #90451
Closed

roachtest: auto retry operations when failing due to ssh-related issues #73542

healthy-pod opened this issue Dec 7, 2021 · 4 comments · Fixed by #90451
Labels
C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) O-roachtest

Comments

@healthy-pod
Copy link
Contributor

healthy-pod commented Dec 7, 2021 •
edited by srosenberg
Loading

When a roachprod operation fails due to an ssh error, attempting it again will likely work. This will help in making roachtests less prone to failure due to issues unrelated to the tests such as intermittent SSH errors.

Epic CRDB-10428
Jira issue: CRDB-11630
@healthy-pod healthy-pod added C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) O-roachtest labels Dec 7, 2021
@healthy-pod healthy-pod mentioned this issue Dec 7, 2021
Closed
@healthy-pod healthy-pod mentioned this issue Dec 23, 2021
Closed
This was referenced Mar 28, 2022
Closed
Closed
@srosenberg
Copy link
Member

In case of retry failure the test should be skipped and instead an infra issue should be logged: #66588

@srosenberg
Copy link
Member

Related: #81090 (comment)

@lidorcarmel
Copy link
Contributor

another one: #81091 (comment)

@cockroachdb cockroachdb deleted a comment from mari-crl May 26, 2022
@kenliu-crl
Copy link
Contributor

manually reviewed and updated

@srosenberg srosenberg mentioned this issue Aug 16, 2022
Closed
@srosenberg srosenberg mentioned this issue Oct 6, 2022
Merged
@smg260 smg260 mentioned this issue Nov 10, 2022
Merged
@craig craig bot closed this as completed in 5a24e11 Nov 15, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) O-roachtest
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Roachtest SSH Retries smg260/cockroach
4 participants
@srosenberg @healthy-pod @lidorcarmel @kenliu-crl