Ensure a built images contains the correct binary

[chrischdi]

What would you like to be added (User Story)?

As a maintainer I want to be sure that the images we publish contain the correct binary and are for the correct architecture

Detailed Description

We had to revert #9753 because images which got built using google cloud build (which uses make release-staging -j 8 -O) did not contain the wanted binaries.

To be more concrete: images for CAPD did contain the test-extension binary instead of the one for CAPD.

Anything else you would like to add?

Disclamer: we are not sure where exactly the issue happened, the current idea is it is during building the images, not when pushing the images.

The CI for building an image could try to analyse the image for having the correct binary. The used method needs to work independent of the architecture of the image.

Example which uses docker, go, mkdir and tar:

function TESTIMAGE() {
  IMAGE="${1}"
  ARCH="${2}"
  GREPGOPATH="${3}"
  BINARYPATH="${4:-manager}"

  echo "> Testing $IMAGE"

  docker save ${IMAGE} -o img.tar
  mkdir -p extracted-img extracted
  tar -xf img.tar -C extracted-img/
  for layer in $(find extracted-img/ -name "*.tar"); do
    tar -xf ${layer} -C extracted
  done

  go version -m extracted/${BINARYPATH} | grep -P "\tpath" | grep -E -q -e ${GREPGOPATH} || (echo "FAILED ${IMAGE} expected value for path: \"${GREPVALUE}\""; go version -m extracted/${BINARYPATH} | grep "\tpath")
  go version -m extracted/${BINARYPATH} | grep -q -E "GOARCH=${ARCH}$" || (echo "Failed ${IMAGE} expected GOARCH to be \"$ARCH\""; go version -m extracted/${BINARYPATH} | grep "GOARCH=")
  rm -rf img.tar extracted-img extracted
}

TESTIMAGE "gcr.io/k8s-staging-cluster-api/capd-manager-amd64:dev" "amd64" "command-line-arguments$"
TESTIMAGE "gcr.io/k8s-staging-cluster-api/kubeadm-bootstrap-controller-amd64:dev" "amd64" "sigs.k8s.io/cluster-api/bootstrap/kubeadm$"
TESTIMAGE "gcr.io/k8s-staging-cluster-api/kubeadm-control-plane-controller-amd64:dev" "amd64" "sigs.k8s.io/cluster-api/controlplane/kubeadm$"
TESTIMAGE "gcr.io/k8s-staging-cluster-api/capim-manager-amd64:dev" "amd64" "sigs.k8s.io/cluster-api/test/infrastructure/inmemory$"
TESTIMAGE "gcr.io/k8s-staging-cluster-api/clusterctl-amd64:dev" "amd64" "sigs.k8s.io/cluster-api/cmd/clusterctl$-amd64:dev" "amd64" "clusterctl"
TESTIMAGE "gcr.io/k8s-staging-cluster-api/test-extension-amd64:dev" "amd64" "sigs.k8s.io/cluster-api/test/extension$"

xref: #9752

Label(s) to be applied

/kind ci

[k8s-ci-robot]

@chrischdi: The label(s) kind/ci cannot be applied, because the repository doesn't have them.

In response to this:

What would you like to be added (User Story)?

As a maintainer I want to be sure that the images we publish contain the correct binary and are for the correct architecture

Detailed Description

We had to revert #9753 because images which got built using google cloud build (which uses make release-staging -j 8 -O) did not contain the wanted binaries.

To be more concrete: images for CAPD did contain the test-extension binary instead of the one for CAPD.

Anything else you would like to add?

Disclamer: we are not sure where exactly the issue happened, the current idea is it is during building the images, not when pushing the images.

The CI for building an image could try to analyse the image for having the correct binary. The used method needs to work independent of the architecture of the image.

Example which uses docker, go, mkdir and tar:

function TESTIMAGE() {
 IMAGE="${1}"
 ARCH="${2}"
 GREPGOPATH="${3}"
 BINARYPATH="${4:-manager}"

 echo "> Testing $IMAGE"

 docker save ${IMAGE} -o img.tar
 mkdir -p extracted-img extracted
 tar -xf img.tar -C extracted-img/
 for layer in $(find extracted-img/ -name "*.tar"); do
   tar -xf ${layer} -C extracted
 done

 go version -m extracted/${BINARYPATH} | grep -P "\tpath" | grep -E -q -e ${GREPGOPATH} || (echo "FAILED ${IMAGE} expected value for path: \"${GREPVALUE}\""; go version -m extracted/${BINARYPATH} | grep "\tpath")
 go version -m extracted/${BINARYPATH} | grep -q -E "GOARCH=${ARCH}$" || (echo "Failed ${IMAGE} expected GOARCH to be \"$ARCH\""; go version -m extracted/${BINARYPATH} | grep "GOARCH=")
 rm -rf img.tar extracted-img extracted
}

TESTIMAGE "gcr.io/k8s-staging-cluster-api/capd-manager-amd64:dev" "amd64" "command-line-arguments$"
TESTIMAGE "gcr.io/k8s-staging-cluster-api/kubeadm-bootstrap-controller-amd64:dev" "amd64" "sigs.k8s.io/cluster-api/bootstrap/kubeadm$"
TESTIMAGE "gcr.io/k8s-staging-cluster-api/kubeadm-control-plane-controller-amd64:dev" "amd64" "sigs.k8s.io/cluster-api/controlplane/kubeadm$"
TESTIMAGE "gcr.io/k8s-staging-cluster-api/capim-manager-amd64:dev" "amd64" "sigs.k8s.io/cluster-api/test/infrastructure/inmemory$"
TESTIMAGE "gcr.io/k8s-staging-cluster-api/clusterctl-amd64:dev" "amd64" "sigs.k8s.io/cluster-api/cmd/clusterctl$-amd64:dev" "amd64" "clusterctl"
TESTIMAGE "gcr.io/k8s-staging-cluster-api/test-extension-amd64:dev" "amd64" "sigs.k8s.io/cluster-api/test/extension$"

xref: #9752

Label(s) to be applied

/kind ci

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[sbueringer]

Thx!

/triage accepted

This would be good as a safeguard considering that we only found out about the issue by coincidence.

[sbueringer]

/help

[k8s-ci-robot]

@sbueringer:
This request has been marked as needing help from a contributor.

Guidelines

Please ensure that the issue body includes answers to the following questions:

• Why are we solving this issue?
• To address this issue, are there any code changes? If there are code changes, what needs to be done in the code and what places can the assignee treat as reference points?
• Does this issue have zero to low barrier of entry?
• How can the assignee reach out to you for help?

For more details on the requirements of such an issue, please see here and ensure that they are met.

If this request no longer meets these requirements, the label can be removed
by commenting with the /remove-help command.

In response to this:

/help

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[dharmjit]

I am not sure but is it possible to leverage the SBOM generation work being done in #6153 to identify any ingredients(binary in this case) being part of OCI images. I am not sure if the SBOM specs are supposed to capture the architecture of the binaries as well.

[chrischdi]

I am not sure but is it possible to leverage the SBOM generation work being done in #6153 to identify any ingredients(binary in this case) being part of OCI images. I am not sure if the SBOM specs are supposed to capture the architecture of the binaries as well.

If the sbom generation gets done on top of an image: that data may be a good way to solve this.

If the sbom gets generated from the repo it won't be able to solve the described issue.

[akshay196]

I am interested in this.

/assign