Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

馃尡 Ensure build images contains correct binary and for correct architecture #9932

Conversation

akshay196
Copy link
Contributor

What this PR does / why we need it:

Ensures build images contains the correct binary and are for the correct architecture.

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):

Fixes #9758

/area ci

@k8s-ci-robot k8s-ci-robot added do-not-merge/invalid-commit-message Indicates that a PR should not merge because it has an invalid commit message. area/ci Issues or PRs related to ci cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Dec 28, 2023
@k8s-ci-robot k8s-ci-robot added the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label Dec 28, 2023
@k8s-ci-robot
Copy link
Contributor

Hi @akshay196. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot k8s-ci-robot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Dec 28, 2023
@akshay196 akshay196 force-pushed the image-contains-correct-binary-9758 branch from b97f3f7 to c658ab0 Compare December 28, 2023 06:20
@k8s-ci-robot k8s-ci-robot removed the do-not-merge/invalid-commit-message Indicates that a PR should not merge because it has an invalid commit message. label Dec 28, 2023
@sbueringer
Copy link
Member

/ok-to-test

/assign @chrischdi

@k8s-ci-robot k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Jan 2, 2024
@akshay196 akshay196 force-pushed the image-contains-correct-binary-9758 branch 2 times, most recently from e68454e to 66f5574 Compare January 3, 2024 10:25
@akshay196
Copy link
Contributor Author

Hey @chrischdi Can you please take a look at this PR? Let me know if you've any comments.
cc @cahillsf

Makefile Outdated Show resolved Hide resolved
Makefile Outdated Show resolved Hide resolved
hack/tools/release/testimage.sh Outdated Show resolved Hide resolved
hack/tools/release/testimage.sh Outdated Show resolved Hide resolved
hack/tools/release/testimage.sh Outdated Show resolved Hide resolved
hack/tools/release/testimage.sh Outdated Show resolved Hide resolved
@chrischdi
Copy link
Member

Thanks for tackling this issue 馃憤 馃弳

@akshay196 akshay196 force-pushed the image-contains-correct-binary-9758 branch from 66f5574 to b268da9 Compare January 11, 2024 16:10
@akshay196
Copy link
Contributor Author

/retest
probably flaky test.

@akshay196
Copy link
Contributor Author

@chrischdi @cahillsf Updated PR to address your comments. PTAL

Copy link
Member

@chrischdi chrischdi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Last nit from my side :-)

hack/tools/release/docker-image-verify.sh Outdated Show resolved Hide resolved
@cahillsf
Copy link
Member

cahillsf commented Jan 15, 2024

do we want this target to fail on verify failures?

i think the current behavior will just echo a failure log and proceed with the rest of the release process, meaning that we will need to go into the Cloudbuild logs for each release to ensure no failures are found prior to promoting the images (and that image verification failures will not be reflected in the prow job state in the event that we merge a change that introduces a problem).

I think it would be better if this target fails if any verify failures are encountered and bubbles them up to the prow job status

wdyt @chrischdi?

@chrischdi
Copy link
Member

do we want this target to fail on verify failures?

i think the current behavior will just echo a failure log and proceed with the rest of the release process, meaning that we will need to go into the Cloudbuild logs for each release to ensure no failures are found prior to promoting the images (and that image verification failures will not be reflected in the prow job state in the event that we merge a change that introduces a problem).

I think it would be better if this target fails if any verify failures are encountered and bubbles them up to the prow job status

wdyt @chrischdi?

Good point, it definitely has to fail when failures are detected.

@sbueringer
Copy link
Member

Agree. Would be good if it fails, especially before pushing the broken images.

@akshay196 akshay196 force-pushed the image-contains-correct-binary-9758 branch from b268da9 to 68f5253 Compare January 16, 2024 14:31
@akshay196
Copy link
Contributor Author

Updated script to fail when image verification fails.
Please take a look.

@chrischdi
Copy link
Member

Good job, thank you.

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jan 17, 2024
@k8s-ci-robot
Copy link
Contributor

LGTM label has been added.

Git tree hash: b846df14dcce68702d2d00caec2a8f1d36cc4ce1

@sbueringer
Copy link
Member

/assign

@akshay196
Copy link
Contributor Author

@sbueringer Checking if you get a chance to take a look at this.

hack/tools/release/docker-image-verify.sh Outdated Show resolved Hide resolved
TESTIMAGE "${REGISTRY}/cluster-api-controller-${arch}:${TAG}" "${arch}" "sigs.k8s.io/cluster-api$"
TESTIMAGE "${REGISTRY}/kubeadm-bootstrap-controller-${arch}:${TAG}" "${arch}" "sigs.k8s.io/cluster-api/bootstrap/kubeadm$"
TESTIMAGE "${REGISTRY}/kubeadm-control-plane-controller-${arch}:${TAG}" "${arch}" "sigs.k8s.io/cluster-api/controlplane/kubeadm$"
TESTIMAGE "${REGISTRY}/capd-manager-${arch}:${TAG}" "${arch}" "command-line-arguments$"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@chrischdi Not super important. Do you have any idea why in CAPD we have command-line-arguments?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe due to difference between building both binaries.

CAPD build:

# Build the CAPD manager using the compiler cache folder
RUN --mount=type=cache,target=/root/.cache/go-build \
    --mount=type=cache,target=/go/pkg/mod \
    CGO_ENABLED=0 GOOS=linux GOARCH=${ARCH} go build -trimpath -a -o /workspace/manager main.go

and

Core controller:

# Do not force rebuild of up-to-date packages (do not use -a) and use the compiler cache folder
RUN --mount=type=cache,target=/root/.cache/go-build \
    --mount=type=cache,target=/go/pkg/mod \
    CGO_ENABLED=0 GOOS=linux GOARCH=${ARCH} \
    go build -trimpath -ldflags "${ldflags} -extldflags '-static'" \
    -o manager ${package}

Not sure if this is the actual reason.

Copy link
Member

@chrischdi chrischdi Jan 22, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reason is in CAPD we refer a file main.go and for the other controllers we pass the package.

Changing main.go to . (to refer the relative package inside the workdir) would fix the path also for capd:

        path    sigs.k8s.io/cluster-api/test/infrastructure/docker

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe worth doing that change and change the test to sigs.k8s.io/cluster-api/test/infrastructure/docker ?

Copy link
Member

@sbueringer sbueringer Jan 22, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah. Let's do it in a separate PR though. Can one of you follow-up please?

(so we unblock @cahillsf)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure, I will raise a separate PR directly as it is small change.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@akshay196 Just fyi @chrischdi already opened the follow-up PR (#10030). So we're good :)

Ensures build images contains the correct binary and are for the
correct architecture.

Signed-off-by: Akshay Gaikwad <akgaikwad001@gmail.com>
@akshay196 akshay196 force-pushed the image-contains-correct-binary-9758 branch from 68f5253 to e55cc52 Compare January 20, 2024 06:22
@k8s-ci-robot k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jan 20, 2024
@akshay196
Copy link
Contributor Author

/retest
probably flaky test.

@sbueringer
Copy link
Member

Thx!

/lgtm
/approve

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jan 22, 2024
@k8s-ci-robot
Copy link
Contributor

LGTM label has been added.

Git tree hash: 9cfe001b80adaf3663549736fbbe6923dbb834e2

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: sbueringer

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jan 22, 2024
@k8s-ci-robot k8s-ci-robot merged commit b3fedd4 into kubernetes-sigs:main Jan 22, 2024
20 checks passed
@k8s-ci-robot k8s-ci-robot added this to the v1.7 milestone Jan 22, 2024
@akshay196 akshay196 deleted the image-contains-correct-binary-9758 branch January 23, 2024 06:03
@akshay196
Copy link
Contributor Author

Thanks @sbueringer @chrischdi @cahillsf for reviewing and merging.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. area/ci Issues or PRs related to ci cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Ensure a built images contains the correct binary
5 participants