Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add additional output formats (tarball and layout) #134

Merged
merged 7 commits into from
Feb 19, 2020

Conversation

jonjohnsonjr
Copy link
Collaborator

Fixes #128
Closes #95

This is staged in a few commits that should be easy to review independently:

  • Create a MultiPublisher
  • Add publish.{Tarball,Layout}Publisher
  • Consolidate publisher options
  • Add flags for tarball/layout publishers

MultiPublisher mimics io.MultiWriter in that it will publish an image to
multiple publish.Interface implementations.
This adds support for publishing in the tarball format and to an OCI
image layout.

The tarball format isn't great, yet. It only supports writing once
instead of appending.
These were spread all over the place for no reasons. Now all the
publisher related options are grouped together.
Adds --oci-layout-path, --tarball, and --push flags.

--push=false will disable the default behavior of publishing to a
registry.
@jonjohnsonjr
Copy link
Collaborator Author

@bioball @josephlewis42 @jbarrick-mesosphere

Would this work for all of your needs?

@bioball
Copy link

bioball commented Feb 14, 2020

Awesome!! Although, I just tried it, and it doesn't seem like all the images within ko resolve are getting added to the tarball. This is what I see when building https://github.com/tektoncd/pipeline:

$ ko resolve --tarball=output.tar -f config > release.yaml
$ cat output.tar | docker load
# => Loaded image: localhost:5000/mypipelineimages/controller-10a3e32792f33651396d02b6855a6e36:latest

However, I'd expect to find a lot more images in there. These are the images that got built and inserted into release.yaml:

localhost:5000/mypipelineimages/pullrequest-init-4e60f6acf9725cba4c9b0c81d0ba89b8@sha256:1ca483f81029306cfb5aec449a8f8a33f767f7eaa3c91f3709046aa04c5ee9e3
localhost:5000/mypipelineimages/gcs-fetcher-029518c065a5d298216f115c6595f133@sha256:a15e480c72aa283ce25bd1fbee92a6eff9ef3b01fcb05c36724724d67b4ddbce
localhost:5000/mypipelineimages/creds-init-c761f275af7b3d8bea9d50cc6cb0106f@sha256:b3fee8be2abeca4068afeabc13fdcb5239bf4736d2798ecb51b9f7d1e76cecb1
localhost:5000/mypipelineimages/imagedigestexporter-6e7c518e6125f31761ebe0b96cc63971@sha256:4c4240a16e935c8fd4cc6388c33b452ba09c4c14eb4fd8f7c407856e90d9a233
localhost:5000/mypipelineimages/entrypoint-bff0a22da108bc2f16c818c97641a296@sha256:5ef95def223c30b9e88a370512956b09a7a816695b83e3f30d40c1e6a54bb895
localhost:5000/mypipelineimages/kubeconfigwriter-3d37fea0b053ea82d66b7c0bae03dcb0@sha256:29acbfe6b4bc133f58e0664437926ca0bd7a13ce98d5b409c8cf12306cf8fd0b
localhost:5000/mypipelineimages/webhook-d4749e605405422fd87700164e31b2d1@sha256:e2e114e1be7ef2e8ede9e84241b6654287faea1bbdff80f00574b2f4466d3acd
localhost:5000/mypipelineimages/git-init-4874978a9786b6625dd8b6ef2a21aa70@sha256:51f215e953750b51a2b059177b3c285a8baefb8862f29a5e883f6af667ed10e0
localhost:5000/mypipelineimages/controller-10a3e32792f33651396d02b6855a6e36@sha256:e0060a3fde69273eccda239c3c1c174f4272a453c8cb2c764f91be7645fe8

This allows us to defer writing to the tarball until we've collected all
the images that have been published.
@jonjohnsonjr
Copy link
Collaborator Author

jonjohnsonjr commented Feb 14, 2020

doesn't seem like all the images within ko resolve are getting added to the tarball

Ahh yeah I forgot to go back and fix this, thanks. There's not an easy way to append an image to a tarball right now (since the interface supports streaming writes). Hacked a workaround in to just buffer images until the end and write them out in a Close().

Try it now 😄

@bioball
Copy link

bioball commented Feb 14, 2020

Looks like it works!

Copy link

@bioball bioball left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM for the tarball stuff

@josephlewis42
Copy link

This looks like it will fit our purposes nicely!

@jonjohnsonjr jonjohnsonjr merged commit 3c6a907 into ko-build:master Feb 19, 2020
@jonjohnsonjr jonjohnsonjr deleted the output-formats branch February 19, 2020 17:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Tarball output format?
4 participants