[Feature request] Add CLI command to dump and merge an image manifest for promotion

[justaugustus]

As we move closer to the vanity domain flip (ref: kubernetes/release#270), Release Managers will be responsible for promoting official images via the promoter, instead of anago.

What I'd love is a "simple" CLI to:

• dump images digests+tags from a staging GCR
• read out the manifest from k/k8s.io
• merge the dumped images into the promotion manifest

Example invocation:

<toolname> promote --project k8s-staging-foo \
--image kube-apiserver \
--staging-tag v1.18.0-extra-stuff \
--prod-tag v1.18.0 \
--output yaml

If the --image is omitted, all images from the staging GCR with a matching --staging-tag would be dumped and merged into the promotion manifest.

From there, someone could PR that new images.yaml to k/k8s.io for promotion.

cc: @listx @tpepper @thockin
ref: group DM, kubernetes/k8s.io#624, kubernetes/release#957

[listx]

Thinking out loud here: the tool could write images back into, e.g., images.yaml, but some things get pretty tricky due to marshaling yaml back out to a string. For example, marshaling the manifest struct back out will (1) most likely result in the dmap field becoming alphabetically-ordered (not all people do this), and (2) result in comments becoming lost. If the underlying images.yaml is tracked in Git (which it should be for our use case) it doesn't become as big of a problem because one could just add the correct changes, but most people just add files wholesale in Git so it's still somewhat of an issue.

We could alternatively change from YAML to something else but that change would entail a ton of code rewriting, something I'm not comfortable doing with golang (Haskell, on the other hand...!).

There are several ways around these issues, but I'm gauging now how difficult they will be. I am considering moving over to v3 of the yaml library because it claims to support comment preservation. It also seems to support hash key order preservation so it might just do the trick.

[listx]

I discovered go-yaml/yaml#459 which suggests that even v3 of the yaml library will alter how comments look.

I could (a) still go with v3's default marshaling function (accepting that some comments will get restyled), or (b) write a custom parser/injector that does not use yaml.Marshal to write things back as yaml and instead just write some new text lines (into the dmap field under images). While I think (b) sounds hacky (and maybe brittle), it would not restyle any comment lines so I think it would lead to a better experience for the user.

Thoughts?

[listx]

If we don't care about preserving comments/style, then the problem space becomes drastically reduced and I can write a patch in a day or two.

[justaugustus]

@listx -- Yep! I think as long as we warn in the command run that comments will get dropped in the manifest, then we should be fine.

I'll leave you to it, since you've already got https://github.com/listx/k8s-container-image-promoter/tree/add-manifest-edit-command in progress. :)

[justaugustus]

/assign @listx
/area release-eng
/sig release
/milestone v1.19
/kind feature

[k8s-ci-robot]

@justaugustus: The label(s) area/release-eng cannot be applied, because the repository doesn't have them

In response to this:

/assign @listx
/area release-eng
/sig release
/milestone v1.19
/kind feature

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[listx]

@listx -- Yep! I think as long as we warn in the command run that comments will get dropped in the manifest, then we should be fine.

I'll leave you to it, since you've already got https://github.com/listx/k8s-container-image-promoter/tree/add-manifest-edit-command in progress. :)

SGTM

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[listx]

This was done in #210