Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add workflow to build and consume kopia binary in kanister-tools #1611

Merged
merged 13 commits into from
Sep 7, 2022

Conversation

ankitjain235
Copy link
Contributor

@ankitjain235 ankitjain235 commented Aug 30, 2022

Signed-off-by: Ankit Jain ankitjain.meone@gmail.com

Change Overview

This PR adds functionality to build and push kopia image to ghcr.io/kanisterio.
When a PR merges, we update the image.

Current commit corresponds to v0.11.3 commit upstream.

Pull request type

Please check the type of change your PR introduces:

  • 🚧 Work in Progress
  • 🌈 Refactoring (no functional changes, no api changes)
  • 🐹 Trivial/Minor
  • 🐛 Bugfix
  • 🌻 Feature
  • 🗺️ Documentation
  • 🤖 Test

Issues

  • fixes #issue-number

Test Plan

  • 💪 Manual
  • ⚡ Unit test
  • 💚 E2E

Signed-off-by: Ankit Jain <ankitjain.meone@gmail.com>
@github-actions
Copy link
Contributor

Thanks for submitting this pull request 🎉. The team will review it soon and get back to you.

If you haven't already, please take a moment to review our project contributing guideline and Code of Conduct document.

@infraq infraq added this to In Progress in Kanister Aug 30, 2022
Makefile Outdated Show resolved Hide resolved
build/update_kopia_image.sh Outdated Show resolved Hide resolved
docker/tools/Dockerfile Outdated Show resolved Hide resolved
@ankitjain235
Copy link
Contributor Author

Working on a follow-up PR to automate the process : #1620

Kanister automation moved this from In Progress to Reviewer approved Sep 5, 2022
@ankitjain235 ankitjain235 removed the kueue label Sep 6, 2022
docker/kopia-build/Dockerfile Outdated Show resolved Hide resolved
docker/kopia-build/Dockerfile Outdated Show resolved Hide resolved
# Build kopia binary from specific commit
RUN git checkout ${kopiaBuildCommit}
# Refers to goreleaser/goreleaser:v1.11.2
COPY --from=docker.io/goreleaser/goreleaser@sha256:b13418f20019fffb29797aff3d03f5e9ca84cea93634f4169a7ed603a95ab198 \
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this the same goreleaser version used during release?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In kanister build, we are using goreleaser/goreleaser:v1.2.4. It's about 8 months old when we last updated it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ankitjain235 (@pavannd1) Can this be simplified?

For example, does goreleaser need to be used here?

It is possible to build kopia directly. For example, see https://github.com/kastenhq/kopia/blob/publish/docker/kopia-alpine/Dockerfile

A few questions:

  • is is necessary to build a container image for kopia? or would it suffice to build the binary, say as part of building the tools image?
  • is it necessary to build the image or binary every time? or can it be cached and reused, and only rebuilt when the kopia version / commit id changes?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@julio-lopez

We used goreleaser just so that we can build the kopia binary the way it is done upstream and we won't have to worry about changing the command if it needs to be changed any time as per change upstream.

In a follow-up PR(#1620) that is WIP, I am adding an action that can be manually triggered whenever we want to build kopia. I kept the process to build a container image for kopia only for this use case. That way, we would only build Kopia image when it's required. Making it as a part of kanister-tools image would result in re-building the binary in scenarios I tried out.

.github/workflows/main.yaml Show resolved Hide resolved
@PrasadG193 PrasadG193 merged commit 0503369 into master Sep 7, 2022
Kanister automation moved this from Reviewer approved to Done Sep 7, 2022
@PrasadG193 PrasadG193 deleted the kopia-image-update branch September 7, 2022 06:56
Copy link
Contributor

@julio-lopez julio-lopez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🥇Thanks for doing this.

Couple of questions to iterate and refine

# Build kopia binary from specific commit
RUN git checkout ${kopiaBuildCommit}
# Refers to goreleaser/goreleaser:v1.11.2
COPY --from=docker.io/goreleaser/goreleaser@sha256:b13418f20019fffb29797aff3d03f5e9ca84cea93634f4169a7ed603a95ab198 \
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ankitjain235 (@pavannd1) Can this be simplified?

For example, does goreleaser need to be used here?

It is possible to build kopia directly. For example, see https://github.com/kastenhq/kopia/blob/publish/docker/kopia-alpine/Dockerfile

A few questions:

  • is is necessary to build a container image for kopia? or would it suffice to build the binary, say as part of building the tools image?
  • is it necessary to build the image or binary every time? or can it be cached and reused, and only rebuilt when the kopia version / commit id changes?

@pavannd1
Copy link
Contributor

pavannd1 commented Sep 7, 2022

@julio-lopez good questions 👍🏼 We have discussed those improvements and will iterate on this. It was crucial to get this in to unblock the upgrades 💯

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Development

Successfully merging this pull request may close these issues.

None yet

6 participants