Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RFC-0003] Add commands for managing OCI artifacts #2856

Merged
merged 33 commits into from
Aug 11, 2022
Merged

[RFC-0003] Add commands for managing OCI artifacts #2856

merged 33 commits into from
Aug 11, 2022

Conversation

stefanprodan
Copy link
Member

@stefanprodan stefanprodan commented Jun 21, 2022

This PR adds commands to the Flux CLI for managing OCI artifacts as described in the RFC Flux OCI support for Kubernetes manifests.

OCI artifact commands

$ flux push artifact oci://ghcr.io/org/repository/app-config:v0.0.1 \
	--path="./manifests" \
	--source="$(git config --get remote.origin.url)" \
	--revision="$(git branch --show-current)/$(git rev-parse HEAD)"

$ flux tag artifact oci://ghcr.io/org/repository/app-config:v0.0.1 --tag latest --tag production

$ flux list artifacts oci://ghcr.io/org/repository/app-config

$ flux pull artifact oci://ghcr.io/org/repository/app-config:latest --output ./tmp

$ flux build artifact --path ./manifests --output ./tmp/artifact.tgz

For authentication purposes, all flux <verb> artifact commands are using the ~/.docker/config.json config file and the Docker credential helpers.

The artifacts create with flux push artifact are annotated with the standard OCI annotation keys:

  • org.opencontainers.image.source
  • org.opencontainers.image.revision
  • org.opencontainers.image.created

OCI repository commands

$ flux create source oci podinfo-oci \
--url oci://ghcr.io/stefanprodan/manifests/podinfo \
--tag 6.1.6 \
--interval 10m

$ flux create kustomization podinfo-oci \
--source=OCIRepository/podinfo-oci \
--path="./kustomize" \
--prune=true \
--interval=5m \
--target-namespace=default \
--wait=true

$ flux get sources oci
$ flux reconcile source oci podinfo-oci
$ flux suspend source oci podinfo-oci
$ flux resume source oci podinfo-oci
$ flux export source oci podinfo-oci
$ flux delete ks podinfo-oci --silent
$ flux delete source oci podinfo-oci --silent

@stefanprodan stefanprodan added the area/oci OCI related issues and pull requests label Jun 21, 2022
@stefanprodan stefanprodan changed the title Add commands for managing OCI artifacts [POC] Add commands for managing OCI artifacts Jun 21, 2022
internal/oci/build.go Outdated Show resolved Hide resolved
kingdonb pushed a commit to kingdonb/tailscale-k8s that referenced this pull request Jun 22, 2022
@pjbgf pjbgf added this to the GA milestone Jun 30, 2022
@stefanprodan stefanprodan changed the title [POC] Add commands for managing OCI artifacts [RFC-0003] Add commands for managing OCI artifacts Jul 8, 2022
internal/oci/meta.go Outdated Show resolved Hide resolved
somtochiama and others added 7 commits August 9, 2022 12:44
Signed-off-by: Somtochi Onyekwere <somtochionyekwere@gmail.com>
Signed-off-by: Somtochi Onyekwere <somtochionyekwere@gmail.com>
Signed-off-by: Somtochi Onyekwere <somtochionyekwere@gmail.com>
Fixup docs string to match pull command

Signed-off-by: Kingdon Barrett <kingdon@weave.works>
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
@kingdonb
Copy link
Member

kingdonb commented Aug 9, 2022

I will look for somewhere better to put this information, but so as to not lose it for now, since I have written it out, I'll put it here:

Link to the docs that show how to create the free-standing "manifests" package repo and associate it with upstreams that can write OCI artifacts in actions runners from a separate repo.

The best way I know how to do this today is (kludgy, hope there is a better way)

Create a GITHUB_TOKEN (PAT) with packages:write and use docker login and flux push to push a first tag, implicitly creating the package repo. Then go to the page for the package which you can find on your GitHub profile, and set the visibility if needed and the permissions there, to allow actions to write from the source repository, as so:
Screen Shot 2022-08-09 at 1 42 12 PM

Each manifests/podinfo or manifests/... is a separate package repo in GitHub, so you can keep these permissions isolated and granular, rather than free-for-all "everybody write into this bucket" approach.

You still need to set permissions.packages: write as above also. We should explain why this is needed in this example and why it is not needed in the following example, since it's not clear without any comment, or at least point to the docs that explain, here is my impression of why:

Pushes to a branch do not confer "write packages" permission by default in the GitHub permission model, because branches are not releases. Pushes to a tag do confer those permissions because tags are used for releasing.

action/README.md Outdated Show resolved Hide resolved
action/README.md Outdated Show resolved Hide resolved
cmd/flux/build_artifact.go Outdated Show resolved Hide resolved
cmd/flux/build_artifact.go Outdated Show resolved Hide resolved
cmd/flux/build_artifact.go Outdated Show resolved Hide resolved
cmd/flux/build_artifact.go Outdated Show resolved Hide resolved
cmd/flux/create_secret_oci.go Outdated Show resolved Hide resolved
cmd/flux/create_secret_oci.go Outdated Show resolved Hide resolved
cmd/flux/create_source_oci.go Outdated Show resolved Hide resolved
cmd/flux/delete_source_oci.go Outdated Show resolved Hide resolved
cmd/flux/get_source_oci.go Outdated Show resolved Hide resolved
cmd/flux/list.go Outdated Show resolved Hide resolved
cmd/flux/pull_artifact.go Outdated Show resolved Hide resolved
cmd/flux/resume_source_oci.go Outdated Show resolved Hide resolved
cmd/flux/suspend_source_oci.go Outdated Show resolved Hide resolved
cmd/flux/tag.go Outdated Show resolved Hide resolved
cmd/flux/tag_artifact.go Outdated Show resolved Hide resolved
cmd/flux/push_artifact.go Outdated Show resolved Hide resolved
Co-authored-by: Kingdon Barrett <kingdon@weave.works>
Co-authored-by: Sunny <darkowlzz@protonmail.com>
Signed-off-by: Stefan Prodan <stefan.prodan@gmail.com>
makkes and others added 2 commits August 10, 2022 14:37
* Added support for OCIRepositories to `flux trace`
* Changed indentation to compensate new, longer field name "Source
  Revision"
* Added unit tests for the new output

closes #2970

Signed-off-by: Max Jonas Werner <max@e13.dev>
Make `flux trace` work with OCIRepository
Copy link
Contributor

@darkowlzz darkowlzz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/oci OCI related issues and pull requests
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

8 participants