NETOBSERV-847 Operator multi-arch builds (upstream) #285

jpinsonneau · 2023-03-02T13:20:08Z

Implements multi arch builds

$ make help

Usage:
  make <target>

General
  help             Display this help.

Tools
  opm              Download opm locally if necessary.
  controller-gen   Download controller-gen locally if necessary.
  kustomize        Download kustomize locally if necessary.
  envtest          Download envtest-setup locally if necessary.
  crdoc            Download crdoc locally if necessary.
  OPSDK            Download operator-sdk locally if necessary.

Code / files generation
  manifests        Generate WebhookConfiguration, ClusterRole and CustomResourceDefinition objects.
  gencode          Generate code containing DeepCopy, DeepCopyInto, and DeepCopyObject method implementations.
  doc              Generate markdown documentation
  generate-go-conversions  Run all generate-go-conversions
  generate         Run all code/file generators
  clean-generated-conversions  Remove files generated by conversion-gen from the mentioned dirs

Development
  vendors          Refresh vendors directory.
  fmt              Run go fmt against code.
  lint             Run linter (golangci-lint).
  test             Run tests.
  coverage-report  Generate coverage report
  coverage-report-html  Generate HTML coverage report
  build            Build manager binary.

Images
  image-build      Build MULTIARCH_TARGETS images
  image-push       Push MULTIARCH_TARGETS images
  manifest-build   Build MULTIARCH_TARGETS manifest
  manifest-push    Push MULTIARCH_TARGETS manifest

Deployment
  install          Install CRDs into the K8s cluster specified in ~/.kube/config.
  uninstall        Uninstall CRDs from the K8s cluster specified in ~/.kube/config.
  deploy           Deploy controller to the K8s cluster specified in ~/.kube/config.
  undeploy         Undeploy controller from the K8s cluster specified in ~/.kube/config.
  run              Run a controller from your host.

OLM
  bundle-prepare   Generate bundle manifests and metadata, then validate generated files.
  bundle           Generate final bundle files.
  update-bundle    Prepare a clean bundle to be commited
  bundle-build     Build the bundle image.
  shortlived-bundle-build  Build a temporary bundle image, expiring after 2 weeks on quay
  bundle-push      Push the bundle image.
  catalog-build    Build a catalog image.
  shortlived-catalog-build  Build a temporary catalog image, expiring after 2 weeks on quay
  catalog-push     Push a catalog image.
  catalog-deploy   Deploy a catalog image.
  catalog-undeploy  Undeploy a catalog image.

Misc
  related-release-notes  Grab release notes for related components (to be inserted in operator's release note upstream, cf RELEASE.md)

Development helpers
  deploy-loki      Deploy loki.
  undeploy-loki    Undeploy loki.
  undeploy-kafka   Undeploy kafka.
  deploy-grafana   Deploy grafana.
  undeploy-grafana  Undeploy grafana.
  deploy-prometheus  Deploy prometheus.
  undeploy-prometheus  Undeploy prometheus.

Local (Kind)
  install-cert-manager  Install cert manager onto the target kubernetes cluster
  uninstall-cert-manager  Uninstall cert manager from the target kubernetes cluster
  create-kind-cluster  Create kind cluster
  delete-kind-cluster  Delete kind cluster
  local-deploy     Local deploy (kind, loki, grafana, example-cr and sample-workload excluding the operator)
  local-redeploy   Local re-deploy (loki, grafana, example-cr and sample-workload excluding the operator)
  local-undeploy   Local cleanup
  local-run        local-redeploy + run the operator locally
  deploy-kind      Deploy controller to the K8s cluster specified in ~/.kube/config.
  undeploy-kind    Undeploy controller from the K8s cluster specified in ~/.kube/config.

OCP
  ocp-deploy-infra  OCP infra. deploy (only loki and grafana excluding the operator)
  ocp-deploy       OCP deploy (loki, grafana, example-cr and sample-workload excluding the operator)
  ocp-undeploy     OCP cleanup
  ocp-run          OCP-deploy + run the operator locally
  ocp-deploy-operator  run flp from the operator
  undeploy-operator  stop the operator locally

shortcuts helpers
  build-image      Build MULTIARCH_TARGETS images
  push-image       Push MULTIARCH_TARGETS images
  build-manifest   Build MULTIARCH_TARGETS manifest
  push-manifest    Push MULTIARCH_TARGETS manifest
  images           Build and push MULTIARCH_TARGETS images and related manifest
  build-ci-images  Build CI images
  push-ci-images   Push CI images
  ci-images        Build and push CI images
  build-catalog    Build a catalog image
  push-catalog     Push a catalog image
  catalog          Build and push a catalog image
  build-bundle     Build the bundle image
  push-bundle      Push the bundle image
  bundle-all       Build and push the bundle image

Please use netobserv/flowlogs-pipeline#426 for main discussion and this PR only for specific items

DEVELOPMENT.md

msherif1234 · 2023-03-02T14:04:18Z

Makefile

 image-push: ## Push OCI image with the manager.
 ifneq (,$(findstring quay.io/netobserv/,$(IMG)))
 	$(error Do not push to quay.io/netobserv)
 else
 	$(OCI_BIN) push ${IMG}
 endif

+single-multiarch-linux-build/%:
+#The --load option is ignored by podman but required for docker
+	DOCKER_BUILDKIT=1 $(OCI_BIN) buildx build --load --build-arg TARGETPLATFORM=linux/$* --build-arg TARGETARCH=$* --build-arg BUILDPLATFORM=linux/amd64 -t ${IMG}-$* -f Dockerfile .


I don't understand why u set --build-arg BUILDPLATFORM=linux/amd64 while set TARGETPLATFORM=linux/$* ?

BUILDPLATFORM is the platform used to build, se here we build on amd64 while TARGETPLATFORM is the platform where we want the image to run on.

Here, we are cross building on BUILDPLATFORM an image that is intended to run on TARGETPLATFORM.

Makefile

msherif1234 · 2023-03-02T14:07:51Z

Makefile

+
+.PHONY: multiarch-manifest-build
+multiarch-manifest-build: single-multiarch-linux-push/amd64 single-multiarch-linux-push/arm64 single-multiarch-linux-push/ppc64le
+	DOCKER_BUILDKIT=1 $(OCI_BIN) manifest create ${IMG} --amend ${IMG}-amd64 --amend ${IMG}-arm64 --amend ${IMG}-ppc64le


how this change BUILDPLATFORM=linux/amd64 to ppc64le maybe u should change the above to BUILDPLATFORM=linux/$*

See my command above, we never change BUILDPLATFORM because we are always building on amd64, but we change TARGETPLATFORM which is the platform for which is image are built for.

msherif1234 · 2023-03-02T14:11:26Z

Makefile

 image-push: ## Push OCI image with the manager.
 ifneq (,$(findstring quay.io/netobserv/,$(IMG)))
 	$(error Do not push to quay.io/netobserv)
 else
 	$(OCI_BIN) push ${IMG}
 endif

+single-multiarch-linux-build/%:


pls add to all new target ## comment style so they show in make help

Sure, following your comment, I have refactored all the Makefiles and updated these docs 😄

jpinsonneau · 2023-03-02T14:22:26Z

Should we add labels as described here ?

Supported Labels
The following label convention defines the target OSes and architectures supported by an operator:

labels:
    operatorframework.io/arch.<GOARCH>: supported
    operatorframework.io/os.<GOOS>: supported
Where <GOARCH> and <GOOS> are one of the values [listed here](https://github.com/golang/go/blob/master/src/go/build/syslist.go).

Multiple Architectures
Some operators may support multiple node architectures or OSes. In this case, multiple labels can be added. For example, an operator that support both windows and linux workloads will sport the following labels:

labels:
    operatorframework.io/os.windows: supported
    operatorframework.io/os.linux: supported

msherif1234 · 2023-03-02T14:33:03Z

Should we add labels as described here ?

Supported Labels The following label convention defines the target OSes and architectures supported by an operator:
labels:
    operatorframework.io/arch.<GOARCH>: supported
    operatorframework.io/os.<GOOS>: supported
Where <GOARCH> and <GOOS> are one of the values [listed here](https://github.com/golang/go/blob/master/src/go/build/syslist.go).
Multiple Architectures Some operators may support multiple node architectures or OSes. In this case, multiple labels can be added. For example, an operator that support both windows and linux workloads will sport the following labels:
labels:
    operatorframework.io/os.windows: supported
    operatorframework.io/os.linux: supported

Yes that will go to CSV right ?

msherif1234 · 2023-03-02T14:39:22Z

why we didn't pass in the new build-arg make image-build and make the platform env var and user can pass in the arch it needs to build for instead of having the new make targets ?

Makefile

codecov · 2023-03-03T13:31:06Z

Codecov Report

Merging #285 (e3e218e) into main (949256f) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main     #285   +/-   ##
=======================================
  Coverage   50.90%   50.90%           
=======================================
  Files          43       43           
  Lines        5080     5080           
=======================================
  Hits         2586     2586           
  Misses       2293     2293           
  Partials      201      201

Flag	Coverage Δ
unittests	`50.90% <ø> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

jpinsonneau · 2023-04-24T10:29:59Z

Rebased with #300

@jotak @OlivierCazade At the end, it forces me to separate build / push jobs steps and I feel it was better using all at once to ensure arguments are correct.

I wonder if there is a smart way to mock docker push ... comands only from test-workflow.sh without modifying the Makefile.

OlivierCazade

LGTM

jpinsonneau · 2023-05-11T10:58:26Z

/approve

openshift-ci · 2023-05-11T10:58:34Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: jpinsonneau

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [jpinsonneau]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

jpinsonneau requested review from jotak and OlivierCazade March 2, 2023 13:23

msherif1234 reviewed Mar 2, 2023

View reviewed changes

stleerh reviewed Mar 2, 2023

View reviewed changes

Makefile Show resolved Hide resolved

openshift-merge-robot added the needs-rebase label Mar 14, 2023

jpinsonneau mentioned this pull request Apr 20, 2023

NETOBSERV-994 FLP multiarch consistency netobserv/flowlogs-pipeline#426

Merged

jpinsonneau force-pushed the 847 branch from 53474d9 to 576e742 Compare April 20, 2023 11:18

openshift-merge-robot removed the needs-rebase label Apr 20, 2023

jpinsonneau force-pushed the 847 branch 4 times, most recently from 6085580 to 596afab Compare April 20, 2023 12:41

openshift-merge-robot added the needs-rebase label Apr 20, 2023

jpinsonneau force-pushed the 847 branch from 596afab to 24cfe0b Compare April 24, 2023 10:23

openshift-merge-robot removed the needs-rebase label Apr 24, 2023

jpinsonneau added the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Apr 24, 2023

github-actions bot removed the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Apr 24, 2023

jpinsonneau force-pushed the 847 branch from 3410d4e to 9a3890f Compare April 24, 2023 13:17

jpinsonneau added the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Apr 24, 2023

jpinsonneau force-pushed the 847 branch from 9a3890f to 0173d39 Compare April 24, 2023 14:13

github-actions bot removed the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Apr 24, 2023

jpinsonneau added the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Apr 24, 2023

jpinsonneau force-pushed the 847 branch from 0173d39 to 285891a Compare April 25, 2023 11:10

github-actions bot removed the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Apr 25, 2023

jpinsonneau added the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Apr 25, 2023

jpinsonneau force-pushed the 847 branch from 285891a to 7906c12 Compare April 25, 2023 12:31

github-actions bot removed the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Apr 25, 2023

jpinsonneau force-pushed the 847 branch from 7906c12 to b931896 Compare April 25, 2023 12:36

multi arch builds

e3e218e

jpinsonneau force-pushed the 847 branch from b931896 to e3e218e Compare April 25, 2023 13:03

OlivierCazade approved these changes May 10, 2023

View reviewed changes

openshift-ci bot assigned OlivierCazade May 10, 2023

openshift-ci bot added the lgtm label May 10, 2023

openshift-ci bot added the approved label May 11, 2023

openshift-merge-robot merged commit 7ee48aa into netobserv:main May 11, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

NETOBSERV-847 Operator multi-arch builds (upstream) #285

NETOBSERV-847 Operator multi-arch builds (upstream) #285

jpinsonneau commented Mar 2, 2023 •

edited

Loading

msherif1234 Mar 2, 2023

OlivierCazade Mar 27, 2023

msherif1234 Mar 2, 2023

OlivierCazade Mar 27, 2023

msherif1234 Mar 2, 2023

jpinsonneau Apr 20, 2023

jpinsonneau commented Mar 2, 2023

msherif1234 commented Mar 2, 2023

msherif1234 commented Mar 2, 2023

codecov bot commented Mar 3, 2023 •

edited

Loading

jpinsonneau commented Apr 24, 2023

OlivierCazade left a comment

jpinsonneau commented May 11, 2023

openshift-ci bot commented May 11, 2023

NETOBSERV-847 Operator multi-arch builds (upstream) #285

NETOBSERV-847 Operator multi-arch builds (upstream) #285

Conversation

jpinsonneau commented Mar 2, 2023 • edited Loading

msherif1234 Mar 2, 2023

Choose a reason for hiding this comment

OlivierCazade Mar 27, 2023

Choose a reason for hiding this comment

msherif1234 Mar 2, 2023

Choose a reason for hiding this comment

OlivierCazade Mar 27, 2023

Choose a reason for hiding this comment

msherif1234 Mar 2, 2023

Choose a reason for hiding this comment

jpinsonneau Apr 20, 2023

Choose a reason for hiding this comment

jpinsonneau commented Mar 2, 2023

msherif1234 commented Mar 2, 2023

msherif1234 commented Mar 2, 2023

codecov bot commented Mar 3, 2023 • edited Loading

Codecov Report

jpinsonneau commented Apr 24, 2023

OlivierCazade left a comment

Choose a reason for hiding this comment

jpinsonneau commented May 11, 2023

openshift-ci bot commented May 11, 2023

jpinsonneau commented Mar 2, 2023 •

edited

Loading

codecov bot commented Mar 3, 2023 •

edited

Loading