-
Notifications
You must be signed in to change notification settings - Fork 481
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Building images for multi-arch with --load parameter fails #59
Comments
The limitation is temporary as daemon should get support for loading multi-arch with moby/moby#38738 so I'm a bit hesitant to add a custom implementation for it atm. |
Hi, this issue is 5 months old and the linked issue (moby/moby#38738) is still in Draft. Any news? |
Hi, I want to build and execute some local tests before pushing the image to the repository. Build runs fine, but at the end...
Here's my environment: docker version
docker buildx version
|
Depending on the use case it may be sufficient when tests run on the runner's platform only. If so, this issue can be avoided by omitting the
|
@git-developer The current limitation is combination of |
you should use |
The pipeline in the project I'm working on runs tests before pushing. Do I have to build the image once for the test and then a second time for production? Is there any workaround to run a multiarch container before pushing it to a repo? |
@Filius-Patris You're options are:
|
Build ARM docker images in the release workflow. # Changes: - Add a new env key `DOCKER_MULTIARCH` and `DOCKER_PUSH`. When set, it will build multi-arch images and push them to the registry. See docker/buildx#59 for why it must be pushed to the registry. - Usage of `crazy-max/ghaction-docker-buildx ` is necessary as it already configured with the ability to perform cross-compilation (using QEMU) so we can just use it, instead of manually set up it. - Usage of `buildx` now make default global arguments. (See: https://docs.docker.com/engine/reference/builder/#automatic-platform-args-in-the-global-scope) # Follow-up: - Releasing the CLI binary file in ARM architecture. The docker images resulting from these changes already build in the ARM arch. Still, we need to make another adjustment like how to retrieve those binaries and to name it correctly as part of Github Release artifacts. Signed-off-by: Ali Ariff <ali.ariff12@gmail.com>
I also landed here and I'm not sure what the options are, on a practical level it seems like a hack to support "docker build" commands and docker buildx to get around the issue. |
How would I tag an image with a version and latest? |
I'm hitting this as well.. any updates? |
…ocker image (#4721) * feat(docker): add HEALTHCHECK to facilitate testing container startup * feat(build): add orca-integration module to exercise the just-built docker image * feat(gha): run integration test in pr builds multi-arch with --load doesn't work, so add a separate step using the local platform to make an image available for testing. see docker/buildx#59 * feat(gha): run integration test in branch builds (cherry picked from commit b360ad5) # Conflicts: # .github/workflows/build.yml # .github/workflows/pr.yml # Dockerfile.java11.slim # Dockerfile.java11.ubuntu # Dockerfile.slim # Dockerfile.ubuntu
…ocker image (backport #4721) (#4737) * feat(build): add orca-integration module to exercise the just-built docker image (#4721) * feat(docker): add HEALTHCHECK to facilitate testing container startup * feat(build): add orca-integration module to exercise the just-built docker image * feat(gha): run integration test in pr builds multi-arch with --load doesn't work, so add a separate step using the local platform to make an image available for testing. see docker/buildx#59 * feat(gha): run integration test in branch builds (cherry picked from commit b360ad5) # Conflicts: # .github/workflows/build.yml # .github/workflows/pr.yml # Dockerfile.java11.slim # Dockerfile.java11.ubuntu # Dockerfile.slim # Dockerfile.ubuntu * fix(build): resolve conflicts while backporting to release-1.32.x There are Dockerfile.java11.* files on this branch * fix(integration): add useJUnitPlatform to orca-integration.gradle so the tests actually run --------- Co-authored-by: David Byron <82477955+dbyron-sf@users.noreply.github.com> Co-authored-by: David Byron <dbyron@salesforce.com>
…ocker imageTest docker image (#6206) * fix(web): replace deprecated spring.profiles in configuration with spring.config.activate.on-profile to remove these warnings: 2024-05-01 21:29:23.746 WARN 1 --- [ main] o.s.b.c.config.ConfigDataEnvironment : Property 'spring.profiles' imported from location 'class path resource [clouddriver.yml]' is invalid and should be replaced with 'spring.config.activate.on-profile' [origin: class path resource [clouddriver.yml] - 375:13] 2024-05-01 21:29:23.746 WARN 1 --- [ main] o.s.b.c.config.ConfigDataEnvironment : Property 'spring.profiles' imported from location 'class path resource [clouddriver.yml]' is invalid and should be replaced with 'spring.config.activate.on-profile' [origin: class path resource [clouddriver.yml] - 363:13] 2024-05-01 21:29:23.746 WARN 1 --- [ main] o.s.b.c.config.ConfigDataEnvironment : Property 'spring.profiles' imported from location 'class path resource [clouddriver.yml]' is invalid and should be replaced with 'spring.config.activate.on-profile' [origin: class path resource [clouddriver.yml] - 350:13] 2024-05-01 21:29:23.746 WARN 1 --- [ main] o.s.b.c.config.ConfigDataEnvironment : Property 'spring.profiles' imported from location 'class path resource [clouddriver.yml]' is invalid and should be replaced with 'spring.config.activate.on-profile' [origin: class path resource [clouddriver.yml] - 312:13] See https://github.com/spring-projects/spring-boot/wiki/Spring-Boot-Config-Data-Migration-Guide#profile-specific-documents. * feat(docker): add HEALTHCHECK to facilitate testing container startup * feat(build): add clouddriver-integration module to exercise the just-built docker image * feat(gha): run integration test in pr builds multi-arch with --load doesn't work, so add a separate step using the local platform to make an image available for testing. see docker/buildx#59 * feat(gha): run integration test in branch builds * fix(docker): reduce the chance for false positives in the health check In case the health check contains more detailed information where one check could report UP but the overall status is down/out of service/etc. See https://docs.spring.io/spring-boot/docs/2.6.15/reference/html/actuator.html#actuator.endpoints.health for more. (cherry picked from commit 9ea2224)
…ocker imageTest docker image (#6206) * fix(web): replace deprecated spring.profiles in configuration with spring.config.activate.on-profile to remove these warnings: 2024-05-01 21:29:23.746 WARN 1 --- [ main] o.s.b.c.config.ConfigDataEnvironment : Property 'spring.profiles' imported from location 'class path resource [clouddriver.yml]' is invalid and should be replaced with 'spring.config.activate.on-profile' [origin: class path resource [clouddriver.yml] - 375:13] 2024-05-01 21:29:23.746 WARN 1 --- [ main] o.s.b.c.config.ConfigDataEnvironment : Property 'spring.profiles' imported from location 'class path resource [clouddriver.yml]' is invalid and should be replaced with 'spring.config.activate.on-profile' [origin: class path resource [clouddriver.yml] - 363:13] 2024-05-01 21:29:23.746 WARN 1 --- [ main] o.s.b.c.config.ConfigDataEnvironment : Property 'spring.profiles' imported from location 'class path resource [clouddriver.yml]' is invalid and should be replaced with 'spring.config.activate.on-profile' [origin: class path resource [clouddriver.yml] - 350:13] 2024-05-01 21:29:23.746 WARN 1 --- [ main] o.s.b.c.config.ConfigDataEnvironment : Property 'spring.profiles' imported from location 'class path resource [clouddriver.yml]' is invalid and should be replaced with 'spring.config.activate.on-profile' [origin: class path resource [clouddriver.yml] - 312:13] See https://github.com/spring-projects/spring-boot/wiki/Spring-Boot-Config-Data-Migration-Guide#profile-specific-documents. * feat(docker): add HEALTHCHECK to facilitate testing container startup * feat(build): add clouddriver-integration module to exercise the just-built docker image * feat(gha): run integration test in pr builds multi-arch with --load doesn't work, so add a separate step using the local platform to make an image available for testing. see docker/buildx#59 * feat(gha): run integration test in branch builds * fix(docker): reduce the chance for false positives in the health check In case the health check contains more detailed information where one check could report UP but the overall status is down/out of service/etc. See https://docs.spring.io/spring-boot/docs/2.6.15/reference/html/actuator.html#actuator.endpoints.health for more. (cherry picked from commit 9ea2224)
…ocker image (#4721) * feat(docker): add HEALTHCHECK to facilitate testing container startup * feat(build): add orca-integration module to exercise the just-built docker image * feat(gha): run integration test in pr builds multi-arch with --load doesn't work, so add a separate step using the local platform to make an image available for testing. see docker/buildx#59 * feat(gha): run integration test in branch builds (cherry picked from commit b360ad5)
…ocker image (#4721) * feat(docker): add HEALTHCHECK to facilitate testing container startup * feat(build): add orca-integration module to exercise the just-built docker image * feat(gha): run integration test in pr builds multi-arch with --load doesn't work, so add a separate step using the local platform to make an image available for testing. see docker/buildx#59 * feat(gha): run integration test in branch builds (cherry picked from commit b360ad5)
…ocker image (#4721) (#4740) * feat(docker): add HEALTHCHECK to facilitate testing container startup * feat(build): add orca-integration module to exercise the just-built docker image * feat(gha): run integration test in pr builds multi-arch with --load doesn't work, so add a separate step using the local platform to make an image available for testing. see docker/buildx#59 * feat(gha): run integration test in branch builds (cherry picked from commit b360ad5) Co-authored-by: David Byron <82477955+dbyron-sf@users.noreply.github.com>
…ocker imageTest docker image (#6206) (#6228) * fix(web): replace deprecated spring.profiles in configuration with spring.config.activate.on-profile to remove these warnings: 2024-05-01 21:29:23.746 WARN 1 --- [ main] o.s.b.c.config.ConfigDataEnvironment : Property 'spring.profiles' imported from location 'class path resource [clouddriver.yml]' is invalid and should be replaced with 'spring.config.activate.on-profile' [origin: class path resource [clouddriver.yml] - 375:13] 2024-05-01 21:29:23.746 WARN 1 --- [ main] o.s.b.c.config.ConfigDataEnvironment : Property 'spring.profiles' imported from location 'class path resource [clouddriver.yml]' is invalid and should be replaced with 'spring.config.activate.on-profile' [origin: class path resource [clouddriver.yml] - 363:13] 2024-05-01 21:29:23.746 WARN 1 --- [ main] o.s.b.c.config.ConfigDataEnvironment : Property 'spring.profiles' imported from location 'class path resource [clouddriver.yml]' is invalid and should be replaced with 'spring.config.activate.on-profile' [origin: class path resource [clouddriver.yml] - 350:13] 2024-05-01 21:29:23.746 WARN 1 --- [ main] o.s.b.c.config.ConfigDataEnvironment : Property 'spring.profiles' imported from location 'class path resource [clouddriver.yml]' is invalid and should be replaced with 'spring.config.activate.on-profile' [origin: class path resource [clouddriver.yml] - 312:13] See https://github.com/spring-projects/spring-boot/wiki/Spring-Boot-Config-Data-Migration-Guide#profile-specific-documents. * feat(docker): add HEALTHCHECK to facilitate testing container startup * feat(build): add clouddriver-integration module to exercise the just-built docker image * feat(gha): run integration test in pr builds multi-arch with --load doesn't work, so add a separate step using the local platform to make an image available for testing. see docker/buildx#59 * feat(gha): run integration test in branch builds * fix(docker): reduce the chance for false positives in the health check In case the health check contains more detailed information where one check could report UP but the overall status is down/out of service/etc. See https://docs.spring.io/spring-boot/docs/2.6.15/reference/html/actuator.html#actuator.endpoints.health for more. (cherry picked from commit 9ea2224) Co-authored-by: David Byron <82477955+dbyron-sf@users.noreply.github.com>
…ocker imageTest docker image (#6206) (#6227) * fix(web): replace deprecated spring.profiles in configuration with spring.config.activate.on-profile to remove these warnings: 2024-05-01 21:29:23.746 WARN 1 --- [ main] o.s.b.c.config.ConfigDataEnvironment : Property 'spring.profiles' imported from location 'class path resource [clouddriver.yml]' is invalid and should be replaced with 'spring.config.activate.on-profile' [origin: class path resource [clouddriver.yml] - 375:13] 2024-05-01 21:29:23.746 WARN 1 --- [ main] o.s.b.c.config.ConfigDataEnvironment : Property 'spring.profiles' imported from location 'class path resource [clouddriver.yml]' is invalid and should be replaced with 'spring.config.activate.on-profile' [origin: class path resource [clouddriver.yml] - 363:13] 2024-05-01 21:29:23.746 WARN 1 --- [ main] o.s.b.c.config.ConfigDataEnvironment : Property 'spring.profiles' imported from location 'class path resource [clouddriver.yml]' is invalid and should be replaced with 'spring.config.activate.on-profile' [origin: class path resource [clouddriver.yml] - 350:13] 2024-05-01 21:29:23.746 WARN 1 --- [ main] o.s.b.c.config.ConfigDataEnvironment : Property 'spring.profiles' imported from location 'class path resource [clouddriver.yml]' is invalid and should be replaced with 'spring.config.activate.on-profile' [origin: class path resource [clouddriver.yml] - 312:13] See https://github.com/spring-projects/spring-boot/wiki/Spring-Boot-Config-Data-Migration-Guide#profile-specific-documents. * feat(docker): add HEALTHCHECK to facilitate testing container startup * feat(build): add clouddriver-integration module to exercise the just-built docker image * feat(gha): run integration test in pr builds multi-arch with --load doesn't work, so add a separate step using the local platform to make an image available for testing. see docker/buildx#59 * feat(gha): run integration test in branch builds * fix(docker): reduce the chance for false positives in the health check In case the health check contains more detailed information where one check could report UP but the overall status is down/out of service/etc. See https://docs.spring.io/spring-boot/docs/2.6.15/reference/html/actuator.html#actuator.endpoints.health for more. (cherry picked from commit 9ea2224) Co-authored-by: David Byron <82477955+dbyron-sf@users.noreply.github.com>
…ocker image (#4721) (#4739) * feat(docker): add HEALTHCHECK to facilitate testing container startup * feat(build): add orca-integration module to exercise the just-built docker image * feat(gha): run integration test in pr builds multi-arch with --load doesn't work, so add a separate step using the local platform to make an image available for testing. see docker/buildx#59 * feat(gha): run integration test in branch builds (cherry picked from commit b360ad5) Co-authored-by: David Byron <82477955+dbyron-sf@users.noreply.github.com>
I just want to have the ability to load AND build multi-arch in our pipelines:
It's doable in 2 steps (build single arch image, test, then build multi-arch image), it just feels like a waste. |
…ocker imageTest docker image (spinnaker#6206) (spinnaker#6227) * fix(web): replace deprecated spring.profiles in configuration with spring.config.activate.on-profile to remove these warnings: 2024-05-01 21:29:23.746 WARN 1 --- [ main] o.s.b.c.config.ConfigDataEnvironment : Property 'spring.profiles' imported from location 'class path resource [clouddriver.yml]' is invalid and should be replaced with 'spring.config.activate.on-profile' [origin: class path resource [clouddriver.yml] - 375:13] 2024-05-01 21:29:23.746 WARN 1 --- [ main] o.s.b.c.config.ConfigDataEnvironment : Property 'spring.profiles' imported from location 'class path resource [clouddriver.yml]' is invalid and should be replaced with 'spring.config.activate.on-profile' [origin: class path resource [clouddriver.yml] - 363:13] 2024-05-01 21:29:23.746 WARN 1 --- [ main] o.s.b.c.config.ConfigDataEnvironment : Property 'spring.profiles' imported from location 'class path resource [clouddriver.yml]' is invalid and should be replaced with 'spring.config.activate.on-profile' [origin: class path resource [clouddriver.yml] - 350:13] 2024-05-01 21:29:23.746 WARN 1 --- [ main] o.s.b.c.config.ConfigDataEnvironment : Property 'spring.profiles' imported from location 'class path resource [clouddriver.yml]' is invalid and should be replaced with 'spring.config.activate.on-profile' [origin: class path resource [clouddriver.yml] - 312:13] See https://github.com/spring-projects/spring-boot/wiki/Spring-Boot-Config-Data-Migration-Guide#profile-specific-documents. * feat(docker): add HEALTHCHECK to facilitate testing container startup * feat(build): add clouddriver-integration module to exercise the just-built docker image * feat(gha): run integration test in pr builds multi-arch with --load doesn't work, so add a separate step using the local platform to make an image available for testing. see docker/buildx#59 * feat(gha): run integration test in branch builds * fix(docker): reduce the chance for false positives in the health check In case the health check contains more detailed information where one check could report UP but the overall status is down/out of service/etc. See https://docs.spring.io/spring-boot/docs/2.6.15/reference/html/actuator.html#actuator.endpoints.health for more. (cherry picked from commit 9ea2224) Co-authored-by: David Byron <82477955+dbyron-sf@users.noreply.github.com>
…ocker imageTest docker image (spinnaker#6206) (spinnaker#6227) * fix(web): replace deprecated spring.profiles in configuration with spring.config.activate.on-profile to remove these warnings: 2024-05-01 21:29:23.746 WARN 1 --- [ main] o.s.b.c.config.ConfigDataEnvironment : Property 'spring.profiles' imported from location 'class path resource [clouddriver.yml]' is invalid and should be replaced with 'spring.config.activate.on-profile' [origin: class path resource [clouddriver.yml] - 375:13] 2024-05-01 21:29:23.746 WARN 1 --- [ main] o.s.b.c.config.ConfigDataEnvironment : Property 'spring.profiles' imported from location 'class path resource [clouddriver.yml]' is invalid and should be replaced with 'spring.config.activate.on-profile' [origin: class path resource [clouddriver.yml] - 363:13] 2024-05-01 21:29:23.746 WARN 1 --- [ main] o.s.b.c.config.ConfigDataEnvironment : Property 'spring.profiles' imported from location 'class path resource [clouddriver.yml]' is invalid and should be replaced with 'spring.config.activate.on-profile' [origin: class path resource [clouddriver.yml] - 350:13] 2024-05-01 21:29:23.746 WARN 1 --- [ main] o.s.b.c.config.ConfigDataEnvironment : Property 'spring.profiles' imported from location 'class path resource [clouddriver.yml]' is invalid and should be replaced with 'spring.config.activate.on-profile' [origin: class path resource [clouddriver.yml] - 312:13] See https://github.com/spring-projects/spring-boot/wiki/Spring-Boot-Config-Data-Migration-Guide#profile-specific-documents. * feat(docker): add HEALTHCHECK to facilitate testing container startup * feat(build): add clouddriver-integration module to exercise the just-built docker image * feat(gha): run integration test in pr builds multi-arch with --load doesn't work, so add a separate step using the local platform to make an image available for testing. see docker/buildx#59 * feat(gha): run integration test in branch builds * fix(docker): reduce the chance for false positives in the health check In case the health check contains more detailed information where one check could report UP but the overall status is down/out of service/etc. See https://docs.spring.io/spring-boot/docs/2.6.15/reference/html/actuator.html#actuator.endpoints.health for more. (cherry picked from commit 9ea2224) Co-authored-by: David Byron <82477955+dbyron-sf@users.noreply.github.com>
* Fix #245 - Build ARM64 image. * Attempt to work around: docker/buildx#59 * Only build arm64 on release. --------- Co-authored-by: Laurent Quérel <l.querel@f5.com>
Due to Docker limitation docker/buildx#59 Signed-off-by: Anna Rift <anna.rift@hpe.com>
Refactor the Docker nightly testing script to optionally also push release-tagged images. Configured by the `RELEASE_VERSION` environment variable, assumed to get set via Jenkins parameter. If set, the script pushes the images tagged as `latest` and `$RELEASE_VERSION`. All nightly build/pushes still run before release-tagged pushes, so we don't push a release if any image variants are broken. Script aborts without building anything if `RELEASE_VERSION` is set but we're not on the appropriate release branch. As part of this PR I attempted to change our process to test the image before pushing, rather than build, push, then test. However, on `chapelmac-m1` this ran into docker/buildx#59, so I reverted it. Since we only push release images after all nightly images, we're still safe from pushing a broken release-tagged image. Noted this limitation and the requirement to build nightly images first in a comment. Also includes: - delete unused `util/cron/publish-docker-images.bash` which didn't contain as much functionality as `test-docker.bash` - more comments and some small refactors for clarity Resolves Cray/chapel-private#6743. [reviewed by @tzinsky , thanks!] Associated pre-merge tasks: - [x] merge corresponding CI config adjustments PR https://github.hpe.com/hpe/hpc-chapel-ci-config/pull/1291 - [x] disable unused previous job https://chapel-ci.us.cray.com/job/publish-docker-images/ - [x] update Docker release best-practices to tell you to use this script (https://github.hpe.com/hpe/hpc-chapel-docs/commit/d159f7e8a0be122660cd8af39e3b49ddb8b59486) - [x] delete temporarily private Docker repositories created for testing (`chapel-test{,-gasnet,-gasnet-smp}`) Testing: - [x] manual run in non-release mode still works - [x] manual run in release mode (temporarily modified to push to a scratch repo)
While trying to build images for multi-architecture (AMD64 and ARM64), I tried to load them into the Docker daemon with the
--load
parameter but I got an error:I understand that the daemon can't see the manifest lists but I believe there should be a way to tag the images with some variable, like:
docker buildx build --platform linux/arm64,linux/amd64 --load -t carlosedp/test:v1-$ARCH .
To have both images loaded into the daemon and ignoring the manifest list in this case.
The text was updated successfully, but these errors were encountered: