fix(trafficrouting): Fix downtime on initial deployment using Istio DestinationRule Subsets. Fixes #2507 #3602

wmuizelaar · 2024-05-28T09:58:32Z

Checklist:

Either (a) I've created an enhancement proposal and discussed it with the community, (b) this is a bug fix, or (c) this is a chore.
The title of the PR is (a) conventional with a list of types and scopes found here, (b) states what changed, and (c) suffixes the related issues number. E.g. "fix(controller): Updates such and such. Fixes #1234".
I've signed my commits with DCO
I have written unit and/or e2e tests for my change. PRs without these are unlikely to be merged.
My builds are green. Try syncing with master if they are not.
My organization is added to USERS.md.

Fixes #2507

github-actions · 2024-05-28T10:11:53Z

Go Published Test Results

2 166 tests 2 166 ✅ 2m 55s ⏱️
119 suites 0 💤
1 files 0 ❌

Results for commit 582f087.

♻️ This comment has been updated with latest results.

codecov · 2024-05-28T10:12:41Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 83.81%. Comparing base (23e186e) to head (1dcdfcd).
Report is 56 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #3602      +/-   ##
==========================================
- Coverage   83.87%   83.81%   -0.07%     
==========================================
  Files         162      162              
  Lines       18524    18529       +5     
==========================================
- Hits        15537    15530       -7     
- Misses       2119     2125       +6     
- Partials      868      874       +6

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

github-actions · 2024-05-28T10:36:02Z

E2E Tests Published Test Results

4 files 4 suites 3h 28m 0s ⏱️
111 tests 101 ✅ 6 💤 4 ❌
448 runs 420 ✅ 24 💤 4 ❌

For more details on these failures, see this check.

Results for commit 582f087.

♻️ This comment has been updated with latest results.

newtondev

Looks good. I sat with Wietse on a call and we went through it together.

sonarqubecloud · 2024-05-31T15:26:55Z

Quality Gate passed

Issues
1 New issue
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
17.0% Duplication on New Code

See analysis details on SonarCloud

newtondev · 2024-05-31T20:09:58Z

LGTM

wmuizelaar · 2024-06-04T15:19:39Z

@zachaller are you able to take a look at this? It would be much appreciated! 🙏

zachaller · 2024-06-12T21:02:34Z

Yea, at first glance I think I would like to try and not change the traffic router interface, this will cause all plugins to have to update etc etc. I think an ok alternative is to pass the replicasetInformer/Lister/k8s client or just add roCtx.allRSs into the istio reconciler context and use that to get the replicasets. If this where to be a plugin they would have to do something similar as well.

wmuizelaar · 2024-06-17T12:05:28Z

Thanks for your remark, I agree that it would be really nice if the interface could stay the same. Let me experiment if I can get it passed to the Istio Reconciler in a different way.

wmuizelaar · 2024-07-08T11:51:30Z

@zachaller can you take a look again please? This feels indeed a lot simpler and easier way of implementation.

newtondev · 2024-07-08T13:12:57Z

LGTM, working on my test project.

Bennett-Lynch · 2024-07-24T16:57:49Z

@zachaller Gentle bump. Looking to see if this also resolves #3681.

zachaller · 2024-07-29T20:43:23Z

Sorry, it's been a while I will take a look at this soon.

Bennett-Lynch · 2024-08-02T16:08:22Z

rollout/trafficrouting/istio/istio.go

+	// We need to check if the replicasets are ready here as well if we didn't define any services in the rollout
+	// See: https://github.com/argoproj/argo-rollouts/issues/2507
+	if r.rollout.Spec.Strategy.Canary.CanaryService == "" && r.rollout.Spec.Strategy.Canary.StableService == "" {


Do we know for sure this is being properly checked when services are defined? In the example in #3681 I am defining both a canary service and stable service but still experiencing a brief downtime/outage after the rollout ends.

Aaah, if you explicitly define the services, the existing code should actually be working - the fix I built here is explicitly for when you are using the Istio DestionationRule-method to update your traffic (https://argo-rollouts.readthedocs.io/en/stable/features/traffic-management/istio/#subset-level-traffic-splitting to be precise).

If you define the services (I did not see those in your example, so that's why I thought it might be related to this), there are other codepaths that actually check service-readiness. There might be a big/issue in there, but then your issue is at least definitely NOT related to mine.

…estinationRule Subsets. Fixes argoproj#2507 Signed-off-by: Wietse Muizelaar <wmuizelaar@bol.com>

sonarqubecloud · 2024-08-27T13:54:33Z

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
6.8% Duplication on New Code

See analysis details on SonarCloud

github-actions · 2024-08-27T14:31:26Z

Published E2E Test Results

4 files 4 suites 3h 22m 20s ⏱️
113 tests 101 ✅ 7 💤 5 ❌
458 runs 424 ✅ 28 💤 6 ❌

For more details on these failures, see this check.

Results for commit 1dcdfcd.

github-actions · 2024-08-27T14:31:32Z

Published Unit Test Results

2 264 tests 2 264 ✅ 2m 59s ⏱️
128 suites 0 💤
1 files 0 ❌

Results for commit 1dcdfcd.

…estinationRule Subsets. Fixes argoproj#2507 (argoproj#3602) Signed-off-by: Wietse Muizelaar <wmuizelaar@bol.com>

wmuizelaar changed the title ~~fix(istio): Fix downtime on initial deployment using Istio DestinationRule Subsets. Fixes #2507~~ fix(trafficrouting): Fix downtime on initial deployment using Istio DestinationRule Subsets. Fixes #2507 May 28, 2024

wmuizelaar force-pushed the fix_istio_desination_rule_readiness branch from bde071e to 25f0dc8 Compare May 28, 2024 10:02

wmuizelaar force-pushed the fix_istio_desination_rule_readiness branch 2 times, most recently from 54a2ab8 to 3704456 Compare May 31, 2024 14:07

newtondev approved these changes May 31, 2024

View reviewed changes

wmuizelaar force-pushed the fix_istio_desination_rule_readiness branch 2 times, most recently from 2c40d9e to 582f087 Compare July 8, 2024 10:58

wmuizelaar mentioned this pull request Jul 8, 2024

Using canary w/ dynamic stable scale causes failed requests at end of rollout #3681

Closed

zachaller self-assigned this Jul 8, 2024

Bennett-Lynch reviewed Aug 2, 2024

View reviewed changes

zachaller added this to the v1.8 milestone Aug 23, 2024

fix(trafficrouting): Fix downtime on initial deployment using Istio D…

1dcdfcd

…estinationRule Subsets. Fixes argoproj#2507 Signed-off-by: Wietse Muizelaar <wmuizelaar@bol.com>

zachaller force-pushed the fix_istio_desination_rule_readiness branch from 582f087 to 1dcdfcd Compare August 27, 2024 13:54

zachaller approved these changes Aug 27, 2024

View reviewed changes

zachaller merged commit 39764fb into argoproj:master Aug 27, 2024
25 checks passed

ndxsdev mentioned this pull request Jan 27, 2025

Rollout Creation with workloadRef Causes Service Downtime #4065

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix(trafficrouting): Fix downtime on initial deployment using Istio DestinationRule Subsets. Fixes #2507 #3602

fix(trafficrouting): Fix downtime on initial deployment using Istio DestinationRule Subsets. Fixes #2507 #3602

wmuizelaar commented May 28, 2024 •

edited

Loading

github-actions bot commented May 28, 2024 •

edited

Loading

codecov bot commented May 28, 2024 •

edited

Loading

github-actions bot commented May 28, 2024 •

edited

Loading

newtondev left a comment

sonarqubecloud bot commented May 31, 2024

newtondev commented May 31, 2024

wmuizelaar commented Jun 4, 2024

zachaller commented Jun 12, 2024

wmuizelaar commented Jun 17, 2024

wmuizelaar commented Jul 8, 2024

newtondev commented Jul 8, 2024

Bennett-Lynch commented Jul 24, 2024

zachaller commented Jul 29, 2024

Bennett-Lynch Aug 2, 2024

wmuizelaar Aug 6, 2024

sonarqubecloud bot commented Aug 27, 2024

github-actions bot commented Aug 27, 2024

github-actions bot commented Aug 27, 2024

fix(trafficrouting): Fix downtime on initial deployment using Istio DestinationRule Subsets. Fixes #2507 #3602

fix(trafficrouting): Fix downtime on initial deployment using Istio DestinationRule Subsets. Fixes #2507 #3602

Conversation

wmuizelaar commented May 28, 2024 • edited Loading

github-actions bot commented May 28, 2024 • edited Loading

Go Published Test Results

codecov bot commented May 28, 2024 • edited Loading

Codecov Report

github-actions bot commented May 28, 2024 • edited Loading

E2E Tests Published Test Results

newtondev left a comment

Choose a reason for hiding this comment

sonarqubecloud bot commented May 31, 2024

Quality Gate passed

newtondev commented May 31, 2024

wmuizelaar commented Jun 4, 2024

zachaller commented Jun 12, 2024

wmuizelaar commented Jun 17, 2024

wmuizelaar commented Jul 8, 2024

newtondev commented Jul 8, 2024

Bennett-Lynch commented Jul 24, 2024

zachaller commented Jul 29, 2024

Bennett-Lynch Aug 2, 2024

Choose a reason for hiding this comment

wmuizelaar Aug 6, 2024

Choose a reason for hiding this comment

sonarqubecloud bot commented Aug 27, 2024

Quality Gate passed

github-actions bot commented Aug 27, 2024

Published E2E Test Results

github-actions bot commented Aug 27, 2024

Published Unit Test Results

wmuizelaar commented May 28, 2024 •

edited

Loading

github-actions bot commented May 28, 2024 •

edited

Loading

codecov bot commented May 28, 2024 •

edited

Loading

github-actions bot commented May 28, 2024 •

edited

Loading