fix: inject artificial delay between sync waves to better support health assessments #4715
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jessesuen
commented
Oct 30, 2020
jessesuen
force-pushed
the
sync-wave-delays
branch
from
c3adaa3 to
7bcc036
Compare
Codecov Report
@@ Coverage Diff @@
## master #4715 +/- ##
==========================================
- Coverage 41.28% 41.11% -0.17%
==========================================
Files 127 127
Lines 17484 17556 +72
==========================================
+ Hits 7218 7219 +1
- Misses 9233 9306 +73
+ Partials 1033 1031 -2
Continue to review full report at Codecov.
|
jessesuen
force-pushed
the
sync-wave-delays
branch
from
7bcc036 to
a08bf26
Compare
wanghong230
approved these changes
Nov 2, 2020
jessesuen
force-pushed
the
sync-wave-delays
branch
5 times, most recently
from
0cd1ec9 to
e084700
Compare
jessesuen
commented
Nov 2, 2020
Comment on lines
-39
to
-49
| Expect(OperationPhaseIs(OperationFailed)). | ||
| Expect(SyncStatusIs(SyncStatusCodeOutOfSync)). | ||
| Expect(HealthIs(health.HealthStatusMissing)). | ||
| Expect(ResourceResultNumbering(1)). | ||
| Expect(ResourceSyncStatusIs("Pod", "pod-1", SyncStatusCodeSynced)). | ||
| Expect(ResourceHealthIs("Pod", "pod-1", health.HealthStatusHealthy)). | ||
| Expect(ResourceSyncStatusIs("Pod", "pod-2", SyncStatusCodeOutOfSync)). | ||
| Expect(ResourceHealthIs("Pod", "pod-2", health.HealthStatusMissing)). | ||
| When(). | ||
| Sync(). | ||
| Then(). |
There was a problem hiding this comment.
The reason this was deleted is because this test had two syncs in a row (even though nothing changed in between those two syncs). The first sync would immediately detect the pod in a ImagePullBackoff state (from the previous state) and always fail.
After my change, the 2-second delay actually allowed the pod time to go from Degraded->Healthy, so that by the time the controller re-reconciled the operation, it saw the pod was healthy and the sync would succeed.
…lth assessments Signed-off-by: Jesse Suen <jesse_suen@intuit.com>
jessesuen
force-pushed
the
sync-wave-delays
branch
from
e084700 to
c954848
Compare
terrycorley
pushed a commit
to terrycorley/argo-cd
that referenced
this pull request
Nov 3, 2020
…zation-generators * 'master' of github.com:argoproj/argo-cd: fix: RevisionFormField component crashes in 'refs' API returns no tags (argoproj#4735) docs: add Opensurvey to USERS.md (argoproj#4727) docs: correct parameters usage in CLI (argoproj#4725) fix: Repo-server has silent unmarshalling errors leading to empty applications (argoproj#4423) (argoproj#4708) fix: inject artificial delay between sync waves to better support health assessments (argoproj#4715) fix: exclude files listed under exclusions (argoproj#4686) feat: support resource actions on CRDs that use status subresources (argoproj#4690) feat: Add autocomplete for repo Revisions (argoproj#4645) (argoproj#4713) fix: webhook don't refresh apps pointing to HEAD (argoproj#4717) feat: Add support for ExecProvider cluster auth (argoproj#4600) (argoproj#4710) fix: adding helm values file in New App (argoproj#4635) docs: Instructions on `make verify-kube-connect` step when using k3d (argoproj#4687) feat: Annotation based app paths detection in webhooks (argoproj#4699) fix: adding commonAnnotations for Kustomize (argoproj#4613)
3 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Resolves #4669
Reduces (but does not eliminate) the possibility of sync waves/hooks firing pre-maturely, by introducing an artificial delay between each sync waves, allowing external controllers to react to the spec changes which were just applied.
Signed-off-by: Jesse Suen jesse_suen@intuit.com