Retry pending deployments longer before failing them #13550
Conversation
|
@mfojtik didn't we see this flake somewhere? Failed deployment because of a missing deployer but the deployer actually existed? |
|
[test] |
|
@Kargakis yes, there was one. good catch! |
| @@ -20,6 +20,8 @@ import ( | |||
| "github.com/openshift/origin/pkg/util" | |||
| ) | |||
|
|
|||
| const maxRetryCount = 10 | |||
There was a problem hiding this comment.
please drop a comment here explaining what is this retry count
|
this LGTM |
|
Evaluated for origin test up to df79f89 |
|
[merge] |
|
continuous-integration/openshift-jenkins/test FAILURE (https://ci.openshift.redhat.com/jenkins/job/test_pull_request_origin/476/) (Base Commit: 959be87) |
|
#11114 [merge] |
|
#12072 [merge] |
|
Evaluated for origin merge up to df79f89 |
|
continuous-integration/openshift-jenkins/merge SUCCESS (https://ci.openshift.redhat.com/jenkins/job/merge_pull_request_origin/212/) (Base Commit: 959be87) (Image: devenv-rhel7_6104) |
| // The first requeue is after 5ms and subsequent requeues grow exponentially. | ||
| // This effectively can extend up to 5^10ms which caps to 1000s: | ||
| // | ||
| // 5ms, 25ms, 125ms, 625ms, 3s, 16s, 78s, 390s, 1000s, 1000s |
There was a problem hiding this comment.
thx @Kargakis What determines the rate of backoff for each requeue? Is it configurable?
There was a problem hiding this comment.
Not configurable currently. See here.
There was a problem hiding this comment.
I guess though you can construct your own ratelimiter
@mfojtik ptal