-
Notifications
You must be signed in to change notification settings - Fork 5.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
chore: transmit only app manifests to the cmp-server for plugin-based applications #18054
chore: transmit only app manifests to the cmp-server for plugin-based applications #18054
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The performance implications of this change are AWESOME. However controlling this based on having a plugin name set seems strange. I think it needs to be a flag in the configmap to start.
It would be nice to configure per app but that could be a future change.
I think a better implementation would be to respect the This implementation would break some apps where manifests reference resources outside the app path. |
If we want to provide this info, we should change RPC contract right? |
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #18054 +/- ##
==========================================
+ Coverage 50.65% 50.68% +0.02%
==========================================
Files 315 316 +1
Lines 43344 43382 +38
==========================================
+ Hits 21958 21986 +28
- Misses 18880 18889 +9
- Partials 2506 2507 +1 ☔ View full report in Codecov by Sentry. |
Possibly... I'm not sure if the repo-server's manifest generation request body currently contains the information from that annotation. If not, we'd have to add it so that the repo-server could evaluate which files to send to the CMP. |
This value is not propagated right now . At first glance we have two options
types.go // ApplicationSource contains all required information about the source of an application
type ApplicationSource struct {
// RepoURL is the URL to the repository (Git or Helm) that contains the application manifests
RepoURL string `json:"repoURL" protobuf:"bytes,1,opt,name=repoURL"`
// Path is a directory path within the Git repository, and is only valid for applications sourced from Git.
Path string `json:"path,omitempty" protobuf:"bytes,2,opt,name=path"`
// TargetRevision defines the revision of the source to sync the application to.
// In case of Git, this can be commit, tag, or branch. If omitted, will equal to HEAD.
// In case of Helm, this is a semver tag for the Chart's version.
TargetRevision string `json:"targetRevision,omitempty" protobuf:"bytes,4,opt,name=targetRevision"`
// Helm holds helm specific options
Helm *ApplicationSourceHelm `json:"helm,omitempty" protobuf:"bytes,7,opt,name=helm"`
// Kustomize holds kustomize specific options
Kustomize *ApplicationSourceKustomize `json:"kustomize,omitempty" protobuf:"bytes,8,opt,name=kustomize"`
// Directory holds path/directory specific options
Directory *ApplicationSourceDirectory `json:"directory,omitempty" protobuf:"bytes,10,opt,name=directory"`
// Plugin holds config management plugin specific options
Plugin *ApplicationSourcePlugin `json:"plugin,omitempty" protobuf:"bytes,11,opt,name=plugin"`
// Chart is a Helm chart name, and must be specified for applications sourced from a Helm repo.
Chart string `json:"chart,omitempty" protobuf:"bytes,12,opt,name=chart"`
// Ref is reference to another source within sources field. This field will not be used if used with a `source` tag.
Ref string `json:"ref,omitempty" protobuf:"bytes,13,opt,name=ref"`
// AnnotationManifestGeneratePaths holds argocd.argoproj.io/manifest-generate-paths annotation value of the Application.
AnnotationManifestGeneratePaths string `json:"annotationManifestGeneratePaths" protobuf:"bytes,14,opt,name=annotationManifestGeneratePaths"`
}
repository.proto // ManifestRequest is a query for manifest generation.
message ManifestRequest {
github.com.argoproj.argo_cd.v2.pkg.apis.application.v1alpha1.Repository repo = 1;
// revision, potentially un-resolved
string revision = 2;
bool noCache = 3;
string appLabelKey = 4;
// Name of the application for which the request is triggered
string appName = 5;
string namespace = 8;
github.com.argoproj.argo_cd.v2.pkg.apis.application.v1alpha1.ApplicationSource applicationSource = 10;
repeated github.com.argoproj.argo_cd.v2.pkg.apis.application.v1alpha1.Repository repos = 11;
// Deprecated: use sidecar plugins instead.
repeated github.com.argoproj.argo_cd.v2.pkg.apis.application.v1alpha1.ConfigManagementPlugin plugins = 12;
github.com.argoproj.argo_cd.v2.pkg.apis.application.v1alpha1.KustomizeOptions kustomizeOptions = 13;
string kubeVersion = 14;
repeated string apiVersions = 15;
// Request to verify the signature when generating the manifests (only for Git repositories)
bool verifySignature = 16;
repeated github.com.argoproj.argo_cd.v2.pkg.apis.application.v1alpha1.RepoCreds helmRepoCreds = 17;
bool noRevisionCache = 18;
string trackingMethod = 19;
map<string, bool> enabledSourceTypes = 20;
github.com.argoproj.argo_cd.v2.pkg.apis.application.v1alpha1.HelmOptions helmOptions = 21;
bool hasMultipleSources = 22;
map<string, github.com.argoproj.argo_cd.v2.pkg.apis.application.v1alpha1.RefTarget> refSources = 23;
// This is used to surface "source not permitted" errors for Helm repositories
repeated string projectSourceRepos = 24;
// This is used to surface "source not permitted" errors for Helm repositories
string projectName = 25;
// argocd.argoproj.io/manifest-generate-paths annotation value of the Application to allow optimize which resources propagated to cmpserver
string AnnotationManifestGeneratePaths = 26;
} IMHO I prefer the second one. Wdyt @crenshaw-dev ? |
I think adding it to just the repository.proto makes sense. |
Thanks mate! Im going with it! 💪 |
Hi @todaywasawesome , Just to confirm if it is the correct approach! |
Hi @todaywasawesome ! Let me know if there is something I can do to push for this change |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I added a few questions. I want to get an extra eye on this because it touches some important areas.
@jsolana can you fix DCO? |
Ey thanks @crenshaw-dev , Im little newbie :( Is this commit the problematic? link is there anyway to fix it easily? (or maybe is better to move all the changes in a new PR?) I was thinking on git rebase --signoff HEAD^^^^^^^^^^^^ # cause are 12 commits
git push -f origin js/transfer-app-manifests-for-plugin Thanks a lot for you help! |
@jsolana If you click "details" on the DCO check it will tell you how to fix it.
|
…st (argoproj#18449) Bumps [@types/node](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/node) from 20.12.12 to 20.12.13. - [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases) - [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/node) --- updated-dependencies: - dependency-name: "@types/node" dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: Javier Solana <javier.solana@cabify.com> Signed-off-by: Javier Solana <javier.solana@cabify.com>
…8450) Bumps [monaco-editor-webpack-plugin](https://github.com/microsoft/monaco-editor) from 7.0.1 to 7.1.0. - [Release notes](https://github.com/microsoft/monaco-editor/releases) - [Changelog](https://github.com/microsoft/monaco-editor/blob/main/CHANGELOG.md) - [Commits](https://github.com/microsoft/monaco-editor/commits) --- updated-dependencies: - dependency-name: monaco-editor-webpack-plugin dependency-type: direct:development update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: Javier Solana <javier.solana@cabify.com> Signed-off-by: Javier Solana <javier.solana@cabify.com>
* feat: removed legacy app tracking label Signed-off-by: Soumya Ghosh Dastidar <gdsoumya@gmail.com> * feat: refactor SetAppInstanceLabel func Signed-off-by: Soumya Ghosh Dastidar <gdsoumya@gmail.com> * feat: refactor SetAppInstance func Signed-off-by: Soumya Ghosh Dastidar <gdsoumya@gmail.com> * feat: refactor SetAppInstanceAnnotation func Signed-off-by: Soumya Ghosh Dastidar <gdsoumya@gmail.com> * feat: refactored SetAppInstance() func Signed-off-by: Soumya Ghosh Dastidar <gdsoumya@gmail.com> --------- Signed-off-by: Soumya Ghosh Dastidar <gdsoumya@gmail.com> Co-authored-by: pasha-codefresh <pavel@codefresh.io> Signed-off-by: Javier Solana <javier.solana@cabify.com> Signed-off-by: Javier Solana <javier.solana@cabify.com>
…D health checks (argoproj#17004) * feat: implement camel.apache.org/Integration CRD health checks Added custom health checks for Camel-K Integration CRDs Signed-off-by: mdebarros <migueld@debarros.me> * chore: cleanup up main health.lua Signed-off-by: mdebarros <miguel@debarros.me> --------- Signed-off-by: mdebarros <migueld@debarros.me> Signed-off-by: mdebarros <miguel@debarros.me> Co-authored-by: mdebarros <migueld@debarros.me> Signed-off-by: Javier Solana <javier.solana@cabify.com> Signed-off-by: Javier Solana <javier.solana@cabify.com>
Signed-off-by: Nabil BENDAFI <nabilbendafi@users.noreply.github.com> Signed-off-by: Javier Solana <javier.solana@cabify.com> Signed-off-by: Javier Solana <javier.solana@cabify.com>
…mgmnt (argoproj#18455) Signed-off-by: Tim Collins <tim@thecollins.team> Signed-off-by: Javier Solana <javier.solana@cabify.com> Signed-off-by: Javier Solana <javier.solana@cabify.com>
* chore: add pipekit to argocd USERS Signed-off-by: Tim Collins <tim@thecollins.team> * Trigger Build Signed-off-by: Tim Collins <tim@thecollins.team> --------- Signed-off-by: Tim Collins <tim@thecollins.team> Signed-off-by: Javier Solana <javier.solana@cabify.com> Signed-off-by: Javier Solana <javier.solana@cabify.com>
…argoproj#18458) This reverts commit 820f4d8. Signed-off-by: jannfis <jann@mistrust.net> Signed-off-by: Javier Solana <javier.solana@cabify.com> Signed-off-by: Javier Solana <javier.solana@cabify.com>
…oj#18462) Bumps [foundation-sites](https://github.com/foundation/foundation-sites) from 6.7.5 to 6.8.1. - [Release notes](https://github.com/foundation/foundation-sites/releases) - [Commits](foundation/foundation-sites@v6.7.5...v6.8.1) --- updated-dependencies: - dependency-name: foundation-sites dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: Javier Solana <javier.solana@cabify.com> Signed-off-by: Javier Solana <javier.solana@cabify.com>
Signed-off-by: Keith Chong <kykchong@redhat.com> Signed-off-by: Javier Solana <javier.solana@cabify.com> Signed-off-by: Javier Solana <javier.solana@cabify.com>
Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com> Signed-off-by: Javier Solana <javier.solana@cabify.com> Signed-off-by: Javier Solana <javier.solana@cabify.com>
…rgoproj#18492) Bumps library/busybox from `5eef5ed` to `9ae97d3`. --- updated-dependencies: - dependency-name: library/busybox dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: Javier Solana <javier.solana@cabify.com> Signed-off-by: Javier Solana <javier.solana@cabify.com>
The timing started before reconciliation timing started including get from the queue, leading to very big times reported, not making sense for what's actually going on. Signed-off-by: Andrii Korotkov <andrii.korotkov@verkada.com> Signed-off-by: Javier Solana <javier.solana@cabify.com> Signed-off-by: Javier Solana <javier.solana@cabify.com>
…er (argoproj#19006) Bumps library/node from 22.4.0 to 22.4.1. --- updated-dependencies: - dependency-name: library/node dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: Javier Solana <javier.solana@cabify.com> Signed-off-by: Javier Solana <javier.solana@cabify.com>
Bumps bitnami/kubectl from `225dae6` to `dc190b7`. --- updated-dependencies: - dependency-name: bitnami/kubectl dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: Javier Solana <javier.solana@cabify.com> Signed-off-by: Javier Solana <javier.solana@cabify.com>
…oproj#19007) Bumps library/node from 22.4.0 to 22.4.1. --- updated-dependencies: - dependency-name: library/node dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: Javier Solana <javier.solana@cabify.com> Signed-off-by: Javier Solana <javier.solana@cabify.com>
Signed-off-by: asjervanasten <asjer94@live.com> Signed-off-by: Javier Solana <javier.solana@cabify.com> Signed-off-by: Javier Solana <javier.solana@cabify.com>
) Bumps [actions/setup-node](https://github.com/actions/setup-node) from 4.0.2 to 4.0.3. - [Release notes](https://github.com/actions/setup-node/releases) - [Commits](actions/setup-node@60edb5d...1e60f62) --- updated-dependencies: - dependency-name: actions/setup-node dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: Javier Solana <javier.solana@cabify.com> Signed-off-by: Javier Solana <javier.solana@cabify.com>
Bumps library/golang from `e94e015` to `a7c0d04`. --- updated-dependencies: - dependency-name: library/golang dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: Javier Solana <javier.solana@cabify.com> Signed-off-by: Javier Solana <javier.solana@cabify.com>
Signed-off-by: Javier Solana <javier.solana@cabify.com> Signed-off-by: Javier Solana <javier.solana@cabify.com>
Bumps [actions/setup-go](https://github.com/actions/setup-go) from 5.0.1 to 5.0.2. - [Release notes](https://github.com/actions/setup-go/releases) - [Commits](actions/setup-go@cdcb360...0a12ed9) --- updated-dependencies: - dependency-name: actions/setup-go dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: Javier Solana <javier.solana@cabify.com> Signed-off-by: Javier Solana <javier.solana@cabify.com>
A while back @csantanapr proposed to add devcontainers to the Argo CD tree (see argoproj#17286). After a few rounds in the contributors' meeting it was proposed to add an issue template, so that for tools like this there would at least be two maintainers that commit to keeping it up-to-date. See [proposal](https://docs.google.com/document/d/1mvfnaeCQgVpCixHchjMPtoW5WdtzLAp8cIr3nWFUSiU/edit) for context. Signed-off-by: Blake Pettersson <blake.pettersson@gmail.com> Signed-off-by: Javier Solana <javier.solana@cabify.com> Signed-off-by: Javier Solana <javier.solana@cabify.com>
* feat(health): support for iam.aws.crossplane.io resources Signed-off-by: Jellyfrog <Jellyfrog@users.noreply.github.com> * feat(health): support for bucket s3.aws.crossplane.io resource Signed-off-by: Jellyfrog <Jellyfrog@users.noreply.github.com> --------- Signed-off-by: Jellyfrog <Jellyfrog@users.noreply.github.com> Signed-off-by: Javier Solana <javier.solana@cabify.com> Signed-off-by: Javier Solana <javier.solana@cabify.com>
…rgoproj#18053) * return the client without performing a matchRepository operation when the application'plugin be configured by its name Signed-off-by: Javier Solana <javier.solana@cabify.com> * add CheckPluginConfiguration pre-flight operation Signed-off-by: Javier Solana <javier.solana@cabify.com> * fix lint Signed-off-by: Javier Solana <javier.solana@cabify.com> * Missing comment Signed-off-by: Javier Solana <javier.solana@cabify.com> * remove WithFields Signed-off-by: Javier Solana <javier.solana@cabify.com> * fix captLocal Signed-off-by: Javier Solana <javier.solana@cabify.com> --------- Signed-off-by: Javier Solana <javier.solana@cabify.com> Co-authored-by: Javier Solana <javier.solana@cabify.com> Signed-off-by: Javier Solana <javier.solana@cabify.com> Signed-off-by: Javier Solana <javier.solana@cabify.com>
* Update USERS.md Added Lely Signed-off-by: lrost <luukrost@gmail.com> * added municipality of The Hague --------- Signed-off-by: lrost <luukrost@gmail.com> Co-authored-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com> Signed-off-by: Javier Solana <javier.solana@cabify.com> Signed-off-by: Javier Solana <javier.solana@cabify.com>
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com> Signed-off-by: Javier Solana <javier.solana@cabify.com> Signed-off-by: Javier Solana <javier.solana@cabify.com>
…ches deadlock (argoproj#19040) This update includes critical fixes that resolve an issue where cluster-scoped resources could have been accidentally deleted during an application sync if pruning is enabled. Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com> Signed-off-by: Javier Solana <javier.solana@cabify.com> Signed-off-by: Javier Solana <javier.solana@cabify.com>
Signed-off-by: CI <ci@argoproj.com> Co-authored-by: CI <ci@argoproj.com> Signed-off-by: Javier Solana <javier.solana@cabify.com> Signed-off-by: Javier Solana <javier.solana@cabify.com>
* feat: Add custom resource actions for flux resources Signed-off-by: Adrian Berger <adrian.berger@bedag.ch> * chore: Add upgrade docs, change annotation text Signed-off-by: Adrian Berger <adrian.berger@bedag.ch> --------- Signed-off-by: Adrian Berger <adrian.berger@bedag.ch> Signed-off-by: Javier Solana <javier.solana@cabify.com> Signed-off-by: Javier Solana <javier.solana@cabify.com>
…goproj#19035) (argoproj#19039) * chore: add optional password setting for headless redis client Signed-off-by: Rachel Sheikh <rsheikh@squareup.com> * fix: remove import cycle Signed-off-by: Rachel Sheikh <rsheikh@squareup.com> * fix: add shared SetOptionalRedisPasswordFromKubeConfig method Signed-off-by: Rachel Sheikh <rsheikh@squareup.com> * fix: export redis consts Signed-off-by: Rachel Sheikh <rsheikh@squareup.com> * test: add test cases for SetOptionalRedisPasswordFromKubeConfig() Signed-off-by: Rachel Sheikh <rsheikh@squareup.com> * chore: go mod tidy Signed-off-by: Rachel Sheikh <rsheikh@squareup.com> * fix: use require instead of assert Signed-off-by: Rachel Sheikh <rsheikh@squareup.com> * fix: Update common/common.go Co-authored-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com> Signed-off-by: Rachel Sheikh <sheikhrachel97@gmail.com> --------- Signed-off-by: Rachel Sheikh <rsheikh@squareup.com> Signed-off-by: Rachel Sheikh <sheikhrachel97@gmail.com> Co-authored-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com> Signed-off-by: Javier Solana <javier.solana@cabify.com> Signed-off-by: Javier Solana <javier.solana@cabify.com>
Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com> Signed-off-by: Javier Solana <javier.solana@cabify.com> Signed-off-by: Javier Solana <javier.solana@cabify.com>
* chore(deps): bump Dex from 2.38.0 to 2.40.0 Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com> * docs Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com> --------- Signed-off-by: Michael Crenshaw <350466+crenshaw-dev@users.noreply.github.com> Signed-off-by: Javier Solana <javier.solana@cabify.com> Signed-off-by: Javier Solana <javier.solana@cabify.com>
Signed-off-by: Javier Solana <javier.solana@cabify.com> Signed-off-by: Javier Solana <javier.solana@cabify.com>
Signed-off-by: Javier Solana <javier.solana@cabify.com> Signed-off-by: Javier Solana <javier.solana@cabify.com>
d290d48
to
e1975d5
Compare
oh 💩 , I ve broken the PR following these instructions :( I think is better to create a new PR with the changes and close this one ok? |
e1975d5
to
dd2842b
Compare
Hi @todaywasawesome! I guess the problem was related to the branch (too old and with merges in the middle). Running these command cracks the PR (not sure why, maybe cause the merges). I ve just created #19209 with the changes and with the DCO fixed. |
Closes #17951
Transmit only the necessary information to the
cmp-server
to improve the performance. More info hereAs a quick win, transmit only
appPath
manifests for plugin-based applications.Checklist: