Updates to support multi-arch image builds

[morlay]

related #3505 #10578 #12935

Notice

have to build multi-arch image on amd64 host. some dev tool not updates.
qemu not played well with arm64 host.

Requirements

• https://github.com/docker/buildx#installing
• qemu for RUN other arch binary on amd64 host https://github.com/docker/buildx#building-multi-platform-images
  • qemu not for compiling. multi arch binaries build on amd64 host with GOARCH switch.

Known Issues

• released binaries without arch suffix -linux-${arch}
  • consider use docker image busybox (extends the Dockerfile.binary) to storage binaries instead of google storage （do this in another pr to avoid break CI）
• pg version major upgraded (9.6 to 10.15)
  • now with a hack, to lock amd64 with 9.6, but arm64 with upgrade to 10.15

Changes

• added custom build supports
  • rename goharbor to $(REGISTRYPROJECTNAME)
• compile multi-arch binaries (enabled with MULTIARCH=true, without MULTIARCH=true should not break anything)
  • folder to make/photon/*/binary/ and files with suffix -linux-${arch}
  • Dockerfile updates
    • updates to photon:3.0 (2.0 is not multi-arch image)
    • COPY with ARG TARGETARCH
• local debug enhancement
  • remove "rm binary" end of single job. could cleanup by make cleanall.

Images published by github actions:

see more in https://github.com/querycap/harbor

Live demo https://harbor.rktl.work/ (in k3s aarch64 cluster)

local run

# build & push multi-arch base images
make build_base_docker MULTIARCH=true REGISTRYPROJECTNAME=ghcr.io/querycap/harbor

# build binaries
make compile MULTIARCH=true REGISTRYPROJECTNAME=ghcr.io/querycap/harbor TRIVYFLAG=true CHARTFLAG=true NOTARYFLAG=true

# build & push multi-arch images
make build MULTIARCH=true REGISTRYPROJECTNAME=ghcr.io/querycap/harbor TRIVYFLAG=true CHARTFLAG=true NOTARYFLAG=true

# build & push standalone_db_migrator 
make build_standalone_db_migrator MULTIARCH=true REGISTRYPROJECTNAME=ghcr.io/querycap/harbor

# check file
$ file make/photon/*/binary/*
make/photon/core/binary/harbor_core-linux-amd64:               ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, Go BuildID=N1oqfPNNt-PCzZYZOZEW/fLxpVectfXovZaMQO1vy/gUQnMf1qa071k8_XikKu/nAQMvjp-uaivof8nCi7X, stripped
make/photon/core/binary/harbor_core-linux-arm64:               ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), statically linked, Go BuildID=S7kYrkNLdA5LawmVVxtu/W_bb8ZNtiZI4QxBYKSSh/xrW5NAjPcUtTJQbBZxno/TVinOpvYO4ktg9iNnASA, stripped
make/photon/jobservice/binary/harbor_jobservice-linux-amd64:   ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, Go BuildID=qyJf_2RGtbeslFzIgZmW/MC6vmNoiEZ7NXQ9082mu/6D6T5C2acEAWxPNbyNKW/0DQWN43CV1kZEuRzxKV3, not stripped
make/photon/jobservice/binary/harbor_jobservice-linux-arm64:   ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), statically linked, Go BuildID=Kdagwk74isIdUfbryDN9/DhQL6xB39FENobAZo3kB/-vDGhGK7YN6QCzHFiZsn/W4K53eXyGcno-W_hUkR0, not stripped
make/photon/notary/binary/migrate-linux-amd64:                 ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, Go BuildID=hedrpMN2z33hjotAfGvq/yBGn2YCma7K4tY4de5H2/ZAEKURUUIxw5zO0W6Gjb/j3BQwnxqeexzpJzSYxZc, not stripped
make/photon/notary/binary/migrate-linux-arm64:                 ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), statically linked, Go BuildID=fiowehVHB7jU3T9ATKjW/2LCbklEgncMw2OcXyYuP/kRVFfx0erafQIzB1yux2/zGWiPK1NpPvEhMDZvyQq, not stripped
make/photon/notary/binary/migrate-patch-linux-amd64:           ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, Go BuildID=58WYlEley6Ahz1MVzoZY/bBP_yX-JwqZoZB04_hYV/UOX_tQViOdP9pTc9goqr/enuyOdgawq-ihE5-yKSv, not stripped
make/photon/notary/binary/migrate-patch-linux-arm64:           ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), statically linked, Go BuildID=hGeC0X2akmZ3JCeaF63u/C9AFsYnXMmmnjyhOsn4a/Zch56JqK9l2_Ie9VbnpN/wIOtS7YEZ2ntgxS1SUe8, not stripped
make/photon/notary/binary/migrations:                          directory
make/photon/notary/binary/notary-server-linux-amd64:           ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, Go BuildID=4rq0HGkTbyUXyf5uHsl0/lImCn5b64L6p-aqppHGj/GDGDuYBEeCNcMf_Fxg0O/7opE1OxOhqlC6589_WqQ, not stripped
make/photon/notary/binary/notary-server-linux-arm64:           ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), statically linked, Go BuildID=xK-ei2H7-0P9EkJMGnR1/vHrRYTOuikK3FMTfjK8g/NghAHEv4pxTU5BuLDf0C/MnJwn3qN-JGZ90Va8IvQ, not stripped
make/photon/notary/binary/notary-signer-linux-amd64:           ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, Go BuildID=yuRxD_CgoCR8qksNEs92/pPH7WZbpUBke31OUGXCZ/m-HzKDx88wylvYfwaR7r/0Xhxed--NpZJo25ag5vK, not stripped
make/photon/notary/binary/notary-signer-linux-arm64:           ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), statically linked, Go BuildID=06duwZfHn3f18CZRcL1G/Q1buI5j0jxe-pxXW4kmF/Co3i-Gi-i8CbK1gd0oxz/fTLV79Gl2FUTKNFkcyKe, not stripped
make/photon/registryctl/binary/harbor_registryctl-linux-amd64: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, Go BuildID=q6wqc_oh2MgsMuWkDpdq/kJRFsWMx1AhF4AJbOc1Z/6xKk-crlSEDfuTOZtOhl/cixI4XjBVQTvzjqKZ_fS, not stripped
make/photon/registryctl/binary/harbor_registryctl-linux-arm64: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), statically linked, Go BuildID=3bxESoJZc4QTdQq0IYYy/_QHldXEz905TQtgd5uDl/TXyEZj9ifi_dMbENJuiB/mRQ8VRs7_WfOJMCAQyhI, not stripped
make/photon/trivy-adapter/binary/scanner-trivy-linux-amd64:    ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, Go BuildID=POA7G6NjFu3BxlRDTwFP/G74bQcvyXlQGokTVN6Rz/Lw-kRL9qyghfspMzlcCg/4Gqtsc-qvWRx-rHttbfc, not stripped
make/photon/trivy-adapter/binary/scanner-trivy-linux-arm64:    ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), statically linked, Go BuildID=7cKu7kEtZFHQ-AMSVDvI/rLx5mHoc34S6wk1AnZ5E/O675oj2Q7iwbif6urVGr/FMKYLjbmhy39uffZSBq8, not stripped

[codecov]

Codecov Report

Merging #13788 (07c17de) into master (e2148f9) will increase coverage by 0.03%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master   #13788      +/-   ##
==========================================
+ Coverage   65.60%   65.64%   +0.03%     
==========================================
  Files         923      923              
  Lines       69366    69402      +36     
  Branches     2088     2101      +13     
==========================================
+ Hits        45506    45556      +50     
+ Misses      19918    19897      -21     
- Partials     3942     3949       +7     

Flag	Coverage Δ
unittests	65.64% <ø> (+0.03%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
src/controller/event/handler/auditlog/auditlog.go	55.55% <0.00%> (-5.56%)	⬇️
src/portal/src/app/shared/entities/shared.const.ts	100.00% <0.00%> (ø)
...te-edit-endpoint/create-edit-endpoint.component.ts	36.59% <0.00%> (+1.18%)	⬆️
...es/vulnerability/vulnerability-config.component.ts	52.99% <0.00%> (+4.27%)	⬆️
...project/repository/gridview/grid-view.component.ts	30.68% <0.00%> (+4.54%)	⬆️
...ase/left-side-nav/registries/endpoint.component.ts	44.82% <0.00%> (+6.36%)	⬆️
...roject/repository/repository-gridview.component.ts	37.37% <0.00%> (+8.66%)	⬆️
src/lib/cache/util.go	89.47% <0.00%> (+15.78%)	⬆️

[DerrickMartinez]

Any updates on this?

[reasonerjt]

Given we have reach consensus with commnunity that we will use separate repo to build images on different arch:
goharbor/community#159

I'm closing this PR.

Thanks for your contribution. I wish to suggest you join in the discussion in the slack channel harbor-multi-arch-workgroup to share your valueable experience.