Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Implement Server-Side Diff #13663

Merged
merged 24 commits into from
Dec 18, 2023
Merged

Conversation

leoluz
Copy link
Collaborator

@leoluz leoluz commented May 19, 2023

Implement Server-Side Diff
fix: #11574

Note on DCO:

If the DCO action in the integration test fails, one or more of your commits are not signed off. Please click on the Details link next to the DCO action for instructions on how to resolve this.

Checklist:

  • Either (a) I've created an enhancement proposal and discussed it with the community, (b) this is a bug fix, or (c) this does not need to be in the release notes.
  • The title of the PR states what changed and the related issues number (used for the release note).
  • The title of the PR conforms to the Toolchain Guide
  • I've included "Closes [ISSUE #]" or "Fixes [ISSUE #]" in the description to automatically close the associated issue.
  • I've updated both the CLI and UI to expose my feature, or I plan to submit a second PR with them.
  • Does this PR require documentation updates?
  • I've updated documentation as required by this PR.
  • Optional. My organization is added to USERS.md.
  • I have signed off all my commits as required by DCO
  • I have written unit and/or e2e tests for my change. PRs without these are unlikely to be merged.
  • My build is green (troubleshooting builds).
  • My new feature complies with the feature status guidelines.
  • I have added a brief description of why this PR is necessary and/or what this PR solves.

Please see Contribution FAQs if you have questions about your pull-request.

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>
Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>
Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>
Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>
Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>
Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>
Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>
Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>
Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>
@leoluz leoluz marked this pull request as ready for review December 6, 2023 22:16
@leoluz leoluz requested review from a team as code owners December 6, 2023 22:16
Copy link
Member

@crenshaw-dev crenshaw-dev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

First pass 🎉

cmd/argocd/commands/admin/app.go Outdated Show resolved Hide resolved
controller/state.go Outdated Show resolved Hide resolved
controller/state.go Outdated Show resolved Hide resolved
controller/sync.go Show resolved Hide resolved
controller/sync.go Outdated Show resolved Hide resolved
util/argo/diff/diff.go Outdated Show resolved Hide resolved
util/argo/diff/diff.go Outdated Show resolved Hide resolved
Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>
controller/sync.go Outdated Show resolved Hide resolved
controller/state.go Outdated Show resolved Hide resolved
Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>
controller/state_test.go Show resolved Hide resolved
cmd/argocd/commands/admin/app.go Outdated Show resolved Hide resolved
Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>
Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>
Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>
Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>
docs/user-guide/diff-strategies.md Outdated Show resolved Hide resolved
docs/user-guide/diff-strategies.md Show resolved Hide resolved
docs/user-guide/diff-strategies.md Outdated Show resolved Hide resolved
docs/user-guide/diff-strategies.md Outdated Show resolved Hide resolved
docs/user-guide/diff-strategies.md Outdated Show resolved Hide resolved
docs/user-guide/diff-strategies.md Outdated Show resolved Hide resolved
docs/user-guide/diff-strategies.md Outdated Show resolved Hide resolved
docs/user-guide/diff-strategies.md Outdated Show resolved Hide resolved
docs/user-guide/diff-strategies.md Outdated Show resolved Hide resolved
docs/user-guide/diff-strategies.md Outdated Show resolved Hide resolved
Copy link
Member

@crenshaw-dev crenshaw-dev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small thing: when I toggle argocd.argoproj.io/compare-options from ServerSideDiff=false to ServerSideDiff=true, I have to manually refresh to get the new diff.

I don't think that's a big deal. But if it's easy to force a recalculation of the diff, I think that would be ideal.

Copy link
Member

@crenshaw-dev crenshaw-dev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should add this to the application-controller line in Procfile:

 --server-side-diff-enabled=${ARGOCD_APPLICATION_CONTROLLER_SERVER_SIDE_DIFF:-''}

@leoluz
Copy link
Collaborator Author

leoluz commented Dec 15, 2023

Small thing: when I toggle argocd.argoproj.io/compare-options from ServerSideDiff=false to ServerSideDiff=true, I have to manually refresh to get the new diff.

I don't think that's a big deal. But if it's easy to force a recalculation of the diff, I think that would be ideal.

I agree and thought about that during implementation. I didn't find an obvious easy way trigger that. I would have to change the logic to decide if it should use the cache or not a bit and I think that could cause different implications.

@leoluz
Copy link
Collaborator Author

leoluz commented Dec 15, 2023

Should add this to the application-controller line in Procfile:

 --server-side-diff-enabled=${ARGOCD_APPLICATION_CONTROLLER_SERVER_SIDE_DIFF:-''}

Added.
Thank you

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>
Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>
Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>
Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>
Copy link

codecov bot commented Dec 17, 2023

Codecov Report

Attention: 46 lines in your changes are missing coverage. Please review.

Comparison is base (4875b02) 49.53% compared to head (579b1ed) 49.48%.
Report is 37 commits behind head on master.

Files Patch % Lines
util/argo/diff/diff.go 37.50% 19 Missing and 1 partial ⚠️
controller/sync.go 0.00% 14 Missing ⚠️
controller/state.go 71.87% 6 Missing and 3 partials ⚠️
cmd/argocd/commands/admin/app.go 25.00% 3 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master   #13663      +/-   ##
==========================================
- Coverage   49.53%   49.48%   -0.05%     
==========================================
  Files         269      271       +2     
  Lines       47321    47726     +405     
==========================================
+ Hits        23441    23619     +178     
- Misses      21576    21772     +196     
- Partials     2304     2335      +31     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>
Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>
Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>
Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>
Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>
@crenshaw-dev crenshaw-dev enabled auto-merge (squash) December 18, 2023 20:13
@crenshaw-dev crenshaw-dev merged commit 82ca7a7 into argoproj:master Dec 18, 2023
25 checks passed
JulienFuix pushed a commit to JulienFuix/argo-cd that referenced this pull request Feb 6, 2024
* feat: Implement Server-Side Diff

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* propagate the refreshtype to the diff config

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* Create the serverSideDiff config

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* chore: add featureflag utility package

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* remove featureflag package

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* add param

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* make ssd configurable with app annotation

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* add server-side-diff flags

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* apply the same logic regardless of the refresh type

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* fix gitops-engine reference

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* address review comments

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* Address review comments

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* docs: add docs related to server-side-diff

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* docs: update doc

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* Add config to include mutation webhooks

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* Address review comments

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* go mod update

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* Add sdd cache test case

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* fix ssd cache unit test

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* Update clidocs

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* update manifests

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* Fix procfile

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* additional doc changes

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* update gitops-engine version

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

---------

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>
penglongli pushed a commit to penglongli/argo-cd that referenced this pull request Mar 6, 2024
* feat: Implement Server-Side Diff

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* propagate the refreshtype to the diff config

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* Create the serverSideDiff config

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* chore: add featureflag utility package

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* remove featureflag package

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* add param

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* make ssd configurable with app annotation

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* add server-side-diff flags

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* apply the same logic regardless of the refresh type

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* fix gitops-engine reference

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* address review comments

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* Address review comments

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* docs: add docs related to server-side-diff

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* docs: update doc

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* Add config to include mutation webhooks

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* Address review comments

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* go mod update

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* Add sdd cache test case

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* fix ssd cache unit test

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* Update clidocs

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* update manifests

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* Fix procfile

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* additional doc changes

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* update gitops-engine version

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

---------

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>
Signed-off-by: penglongli <pelenli@tencent.com>
lyda pushed a commit to lyda/argo-cd that referenced this pull request Mar 28, 2024
* feat: Implement Server-Side Diff

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* propagate the refreshtype to the diff config

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* Create the serverSideDiff config

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* chore: add featureflag utility package

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* remove featureflag package

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* add param

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* make ssd configurable with app annotation

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* add server-side-diff flags

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* apply the same logic regardless of the refresh type

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* fix gitops-engine reference

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* address review comments

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* Address review comments

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* docs: add docs related to server-side-diff

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* docs: update doc

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* Add config to include mutation webhooks

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* Address review comments

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* go mod update

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* Add sdd cache test case

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* fix ssd cache unit test

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* Update clidocs

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* update manifests

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* Fix procfile

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* additional doc changes

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* update gitops-engine version

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

---------

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>
Signed-off-by: Kevin Lyda <kevin@lyda.ie>
Hariharasuthan99 pushed a commit to AmadeusITGroup/argo-cd that referenced this pull request Jun 16, 2024
* feat: Implement Server-Side Diff

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* propagate the refreshtype to the diff config

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* Create the serverSideDiff config

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* chore: add featureflag utility package

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* remove featureflag package

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* add param

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* make ssd configurable with app annotation

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* add server-side-diff flags

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* apply the same logic regardless of the refresh type

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* fix gitops-engine reference

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* address review comments

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* Address review comments

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* docs: add docs related to server-side-diff

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* docs: update doc

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* Add config to include mutation webhooks

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* Address review comments

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* go mod update

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* Add sdd cache test case

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* fix ssd cache unit test

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* Update clidocs

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* update manifests

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* Fix procfile

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* additional doc changes

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

* update gitops-engine version

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>

---------

Signed-off-by: Leonardo Luz Almeida <leonardo_almeida@intuit.com>
@kotyara85
Copy link

kotyara85 commented Oct 4, 2024

@leoluz

error when running on 2.11.7

Unable to save changes: application spec for cilium is invalid: InvalidSpecError: Unable to generate manifests in : rpc error: code = Unknown desc = `helm template . --name-template cilium --namespace kube-system --kube-version 1.30 --values /tmp/e1416a6f-b6c5-4a8c-bb6c-dcd4aa41abc2 <api versions removed> --include-crds` failed exit status 1: Error: template: cilium/templates/cilium-operator/deployment.yaml:141:20: executing "cilium/templates/cilium-operator/deployment.yaml" at <include "k8sServiceHost" .>: error calling include: template: cilium/templates/_helpers.tpl:120:37: executing "k8sServiceHost" at <$configmap.data>: wrong type for value; expected map[string]interface {}; got interface {} Use --debug flag to render out invalid YAML

Template example:

{{- $configmapName := "kubeadm-config" }}
{{- $configmapNamespace := "kube-system" }}
{{- $configmap := (lookup "v1" "ConfigMap" $configmapNamespace $configmapName) }}
apiVersion: v1
data:
    clusterName: {{ ($configmap.data.ClusterConfiguration | fromYaml).clusterName }}
kind: ConfigMap
metadata:
  name: cluster-config

This works fine with helm template . --dry-run=server, but not with argocd

Here's how I did discover this cilium/cilium#35094

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Implement Server-Side Diff in Argo CD controller
3 participants