thread for collecting interesting patterns

[krancour]

I've been saying that Kargo is flexible enough that we will continuously discover new usage patterns that we didn't anticipate. I'm starting this thread as a place for collecting patterns that we'll eventually formalize in a patterns catalog in the documentation.

Very high level descriptions are fine. I'll start us off with some interesting ones.

Please add any interesting patterns you discover!

Not GitOps; Quick and Dirty; But Still Useful

This is a whole class of patterns that involve promotion mechanisms that do not write back to a git repo. Without a repo being the sole source of truth, it's not real GitOps, but that doesn't mean these patterns don't have their place.

Non-exhaustive examples:

• Warehouse subscribes to an image repo. Stages update Kustomize-specific or Helm-specific params in Argo CD App(s) with new image version info. This is a quick and dirty way of keeping one or more deployments of third-party software up to date.

• Warehouse subscribes to a Helm chart repo. Stages update targetRevision in Argo CD App(s) with new chart version info. This is another quick and dirty way of keeping one or more deployments of third-party software up to date.

• Warehouse subscribes to a git repo. Stages update targetRevision in Argo CD App(s) with latest commit ID. This is a quick and dirty way to manage config-only changes (which could technically include image changes also if someone or something else writes updated image information to the git repo). Note: In this case, most everything is in git, but the repo doesn't have a record or what Stage is running what commit. If you lose your App(s), you lose track of what was where.

"Image Updater on Steroids"

Works similarly to Argo CD Image Updater.

Warehouse subscribes to image repo(s) only. Stages write new image version info to:

• Stage-specific path in git repo trunk (e.g. a Kustomize overlay)

  OR

• Stage-specific branch in git repo

Underlying Argo CD App(s) do all the heavy lifting.

Such a pipeline accommodates progressive delivery of new images, but doesn't coordinate other config changes (e.g. flipping a feature flag).

The Common Case

Warehouse subscribes to the trunk of one or more git repos (for config changes) and also to one or more image repos (for software builds). Each stage "mashes" these artifacts together and writes the result to a stage-specific branch. Underlying Argo CD App(s) do all the heavy lifting.

Common Case Variant w/ Pre-Pipeline

In the common case (above), the trunk of a git repo never has image version information. That information resides only in each Stage's branch. The repo still has a complete record of everything, but not having that in the trunk seems to offend some users' sensibilities.

Other users may wish for the simplicity of a Warehouse with a subscription to a git repo only -- which produces simple freight with references to a single commit only.

Both of these can be accommodated by using two pipelines:

1. A short "pre-pipeline" with a warehouse that subscribes to image repo(s) only and a single stage that writes new image version info to a git repo's trunk.

2. A pipeline that subscribes to the git repo's trunk only. Stages either:

   • Copy commits from the trunk to a Stage-specific branch.

   • Render commits from the trunk to a Stage-specific branch.

Composite Patterns

Various patterns above can be combined in numerous ways.

For example, using the "pre-pipeline" pattern with Stages that only update the targetRevision field of each Argo CD App.

[krancour]

## Hotfix Pattern

Although we plan to build features that will make "hotfixes" easier (see #1073 #1074 #1075), there's a pattern that already works today:

Stages can currently subscribe to upstream Stages or to a Warehouse, but not both. (Maybe this will change at some point.)

If you position a Stage (let's call it "hotfix") just upstream from prod, prod can subscribe to the hotfix Stage and whatever other Stage(s) it would normally. The "hotfix" Stage can subscribe directly to a Warehouse -- the same Warehouse that the "far left" Stage ("dev" for instance) subscribes to. This provides a second entry point into the pipeline that's much closer to prod.

Even better, the hotfix stage doesn't even have to do anything -- e.g. No promotion mechanisms.

We have manual Freight approval now.

[krancour]

GitOps Your Pipelines

There's no reason you can't define your projects/pipelines in a GitOps repo and use an Argo CD App (or "App of Apps" even) to deploy the projects/pipelines and keep them up to date.

🐢
🐢
🐢
🐢
🐢
🐢

[yhrn]

Hi @krancour,

I'm trying Kargo out and what you describe here as "The Common Case" is pretty much what we are interested in. However, it doesn't seem as straightforward to me as I expected. The quick start seems to be more of the "Image Updater on Steroids" case and I can't find any other documentation that covers the common case.

The main thing I haven't figured out is how to get Kargo to only create freights for the combination of image version and the git commits from which the image was built and not just combine anything new it finds. I'd like to be able to tell Kargo something like "create a new freight when you see a new commit on the main branch of git repo X and a image with the SHA of that commit as tag in image repo Y". Is there a clean way of doing this or am I misunderstanding what the "common" use case Kargo wants to solve is?

[krancour]

@yhrn please start a separate issue or discussion for this.

[yhrn]

@yhrn please start a separate issue or discussion for this.

Sure, but I should probably rephrase a bit too. What I'm wondering is if my case (git repo with source code and k8s manifests, build/push one or more images during CI, have Kargo inject new image versions into manifests from the corresponding commit and promote to stage) is really what you mean with "the common case" or of it's slightly different and could be added as another pattern here?

I feel like it matches the description, but because it doesn't seem to be straightforward to implement I was wondering if it might be a different pattern after all.

[krancour]

@yhrn no... the common case doesn't pay any regard to the repo that the image was built from. The common case mashes together the "latest" (according to rules you define) image that it has found with manifests from a "gitops repo." Source code never enters into the equation.

[yhrn]

Ok, then I think my pattern could be described something like this (and from my experience this is a common pattern at numerous large software companies):

The DevOps Pattern

A team owns a k8s deployed application end-to-end and the application source code along with the k8s manifests reside in a single Git repo. During the CI process for merging a commit to the main branch, one or more images are built and pushed. The manifests at the main commit now needs to propagate through multiple stages alongside the image versions built from that specific commit.

The DevOps Monorepo Pattern

Same as the "The DevOps Pattern" but the application resides in a monorepo, together with a, potentially large, number of other applications, making it undesirable for each main commit to result in a full rollout/replacement of all pods. The CI process needs to ensure image builds are fully reproducible (not a Kargo concern). The desired behavior from Kargo is that manifest rendering is reproducible (e.g. by using image digests instead of tags) and that expensive/slow (verification) steps kan be omitted during promotion of a new commit that results in no relevant changes (noop promotion).

[krancour]

the application source code along with the k8s manifests reside in a single Git repo

That was not clear initially.

So what you basically want is to subscribe only to manifests (no need to watch the images in this case, really) and at each stage, update the manifests so the image tag matches the sha of the commit containing the manifests.

Kargo can't really do that yet, but it's sensible, and in a future state, this should be possible.

However, you can get almost all the way to what you're looking to do with a non-Kargo workaround. You can add automation to the merges to your main branch that grabs the commit ID at the head of main, builds the image and tags it using the commit ID, then updates the manifests to reference that new tag.

[yhrn]

So what you basically want is to subscribe only to manifests (no need to watch the images in this case, really) and at each stage, update the manifests so the image tag matches the sha of the commit containing the manifests.

Yeah, the Git repo with the manifests is the primary thing to watch in this scenario. However, images would be built after a new commit appears there so some kind of secondary watch on the images could be useful to avoid attempting to promote a change before the images are ready.

Also, optimally I think I would want to inject the image digest instead of the commit sha tag to avoid unnecessary pod churn in cases where a commit leads to an image being built that is identical to the previous commit image. I would still tag the image with the commit sha though so Kargo can find it.

Kargo can't really do that yet, but it's sensible, and in a future state, this should be possible.

I can create a separate issue for this if you'd like. Not 100% sure how I should phrase the request though?

However, you can get almost all the way to what you're looking to do with a non-Kargo workaround. You can add automation to the merges to your main branch that grabs the commit ID at the head of main, builds the image and tags it using the commit ID, then updates the manifests to reference that new tag.

Sure, this is what we do now, but it's a bit messy and part of why we are looking at Kargo.

[krancour]

I can create a separate issue for this if you'd like. Not 100% sure how I should phrase the request though?

I wouldn't bother.

The main feature we are working on now is #2219

Looking beyond the initial implementation of that feature, we will most likely be adding some kind of context that is user-accessible via a simple expression language and I expect that is how something like what you have described would be achievable in the future as it may allow you to access the ID of the relevant commit from your Git repo and use it in performing some image replacement in those manifests.