Support Server-side Dry-run Syncs

[tybook]

Summary

Right now on ArgoCD one can only do client-side dry-runs. Even though I see some code in gitops-engine that shows server-side dry-runs are partially supported already, this bit of code makes it so there's no way to actually trigger them: https://github.com/argoproj/gitops-engine/blob/f15cf615b8141a9f0d8e37cf92bac2d0e7a486da/pkg/sync/sync_context.go#L908-L910
A boolean dryRun comes in, with true mapping to cmdutil.DryRunClient and false mapping to cmdutil.DryRunNone; no way to set cmdutil.DryRunServer or cmdutil.DryRunAll.

(Note that doing argocd app sync --dry-run --server-side does not do a server-side dry-run like I thought it would. It in fact doesn't do a dry-run at all; it does a normal server-side apply that makes changes in the target cluster! A argocd app sync --dry-run --server-side command maps to calls of kubectl apply --dry-run=client --server-side, not calls of kubectl apply --dry-run=server.)

Motivation

I've built a system that validates template/manifest changes via ArgoCD dry-run syncs before allowing them to merge to the main branch for actual sync by ArgoCD. We want to catch as many errors as early as possible, and a server-side dry-run is far more meaningful than a client-side dry-run. I can't easily do direct calls of kubectl apply --dry-run=server on the changed manifests as a workaround, because by design this system doesn't know about every target cluster out there, only our centralized ArgoCD servers do.

Proposal

I suggest changing the argocd app sync boolean --dry-run CLI arg (and the corresponding Application resource sync option bits) to match what kubectl does with a string value: --dry-run=client, --dry-run=server, etc.

[tybook]

Any thoughts on this? Is there a pitfall I'm missing, or would implementation be as simple as wiring up a new --dry-run=server CLI arg to set cmdutil.DryRunServer in gitops-engine?

[tybook]

Also it looks like argoproj/gitops-engine#522 (#11574) added code implementing server-side dry-run applies in gitops-engine, so it should be even easier to implement this now, no?

[blakepettersson]

Indeed, should be resolved with #13663

[jeremych1000]

Is there a way to use this with autosync? I have autosync enabled on all my appsets/apps and that means I can't use argocd app sync --dry-run.

[akust]

Is it possible to turn off ArgoCD notifications for syncs with --dry-run flag?