question: crane append doesn't work properly with multi arch images

[frezbo]

Scenario:

A simple go app created for amd64 and arm64 and saved to tarballs named crane_amd64.tar and crane_arm64.tar

package main

import (
	"fmt"
	"runtime"
)

func main() {
	fmt.Printf("Hey crane from %s\n", runtime.GOARCH)
}

❯ ls
crane_amd64.tar  crane_arm64.tar  go.mod  main.go

# amd64 binary
❯ tar xf crane_amd64.tar
❯ file crane 
crane: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, Go BuildID=BPKNPtPsj-yTUUqgjLvy/dk3CVPCLC8x5hWAKpm3d/mirtEfvy7tukCtk_c1-_/baoS6tqiVda69tX4yvwM, not stripped

# arm64 binary
❯ tar xf crane_arm64.tar
❯ file crane
crane: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), statically linked, Go BuildID=7gAY1JRtE59KEIOjJwuI/0Nevb4TycPLJNkNqC00c/NuG6kzQodtdTHB7Vf_qU/ACWVSG_HpTZv6L5pvfMs, not stripped

Now creating an image based on alpine using crane append

❯ crane --platform=linux/amd64 append --base docker.io/library/alpine --new_layer crane_amd64.tar --new_tag ghcr.io/frezbo/crane:0.1.0
2022/01/04 19:11:43 existing blob: sha256:59bf1c3509f33515622619af21ed55bbe26d24913cedbca106468a5fb37a50c3
2022/01/04 19:11:45 pushed blob: sha256:27b163a095d38339fd1337b5233ac366fd85ebafbe6342da0076ef59a4d68569
2022/01/04 19:11:45 pushed blob: sha256:eb23a3f25c9bf62b2b626bda79a42172e34227385e6f06981f85ced4f68ddf35
2022/01/04 19:11:46 ghcr.io/frezbo/crane:0.1.0: digest: sha256:ee5ae6ab0549bfd514c05f17665b7d355b52942cc29ef0775660e64f6c547664 size: 591
ghcr.io/frezbo/crane@sha256:ee5ae6ab0549bfd514c05f17665b7d355b52942cc29ef0775660e64f6c547664

Running the amd64 image

❯ docker container run --rm --platform linux/amd64 ghcr.io/frezbo/crane@sha256:ee5ae6ab0549bfd514c05f17665b7d355b52942cc29ef0775660e64f6c547664 /crane
Unable to find image 'ghcr.io/frezbo/crane@sha256:ee5ae6ab0549bfd514c05f17665b7d355b52942cc29ef0775660e64f6c547664' locally
ghcr.io/frezbo/crane@sha256:ee5ae6ab0549bfd514c05f17665b7d355b52942cc29ef0775660e64f6c547664: Pulling from frezbo/crane
59bf1c3509f3: Already exists 
eb23a3f25c9b: Pull complete 
Digest: sha256:ee5ae6ab0549bfd514c05f17665b7d355b52942cc29ef0775660e64f6c547664
Status: Downloaded newer image for ghcr.io/frezbo/crane@sha256:ee5ae6ab0549bfd514c05f17665b7d355b52942cc29ef0775660e64f6c547664
Hey crane from amd64

Now appending the arm64 edition to a previously created amd64 image tag

❯ crane --platform=linux/arm64 append --base ghcr.io/frezbo/crane@sha256:ee5ae6ab0549bfd514c05f17665b7d355b52942cc29ef0775660e64f6c547664 --new_layer crane_arm64.tar --new_tag ghcr.io/frezbo/crane:0.1.1
2022/01/04 19:12:59 existing blob: sha256:59bf1c3509f33515622619af21ed55bbe26d24913cedbca106468a5fb37a50c3
2022/01/04 19:12:59 existing blob: sha256:eb23a3f25c9bf62b2b626bda79a42172e34227385e6f06981f85ced4f68ddf35
2022/01/04 19:13:04 pushed blob: sha256:a2054f1a0cf4a325c9dc9b80ca4087cffdc867991ebc07bfe24901a41e90e61d
2022/01/04 19:13:05 pushed blob: sha256:47db48988e06243b0a6ecb6fb779fc6e4ace26e73100e0d4b75c147a1f687d8a
2022/01/04 19:13:06 ghcr.io/frezbo/crane:0.1.1: digest: sha256:19c3381936030184e4d692aeb04bee2a4833dfbd2e0a384238844fbdb2a5a034 size: 755
ghcr.io/frezbo/crane@sha256:19c3381936030184e4d692aeb04bee2a4833dfbd2e0a384238844fbdb2a5a034

Running the arm64 image (doesn't work)

❯ docker container run --rm --platform linux/arm64 ghcr.io/frezbo/crane@sha256:19c3381936030184e4d692aeb04bee2a4833dfbd2e0a384238844fbdb2a5a034
Unable to find image 'ghcr.io/frezbo/crane@sha256:19c3381936030184e4d692aeb04bee2a4833dfbd2e0a384238844fbdb2a5a034' locally
ghcr.io/frezbo/crane@sha256:19c3381936030184e4d692aeb04bee2a4833dfbd2e0a384238844fbdb2a5a034: Pulling from frezbo/crane
59bf1c3509f3: Already exists 
eb23a3f25c9b: Already exists 
47db48988e06: Pull complete 
Digest: sha256:19c3381936030184e4d692aeb04bee2a4833dfbd2e0a384238844fbdb2a5a034
Status: Downloaded newer image for ghcr.io/frezbo/crane@sha256:19c3381936030184e4d692aeb04bee2a4833dfbd2e0a384238844fbdb2a5a034
docker: Error response from daemon: image with reference ghcr.io/frezbo/crane@sha256:19c3381936030184e4d692aeb04bee2a4833dfbd2e0a384238844fbdb2a5a034 was found but does not match the specified platform: wanted linux/arm64, actual: linux/amd64.
See 'docker run --help'.

Appending to based on alpine (works)

❯ crane --platform=linux/arm64 append --base docker.io/library/alpine --new_layer crane_arm64.tar --new_tag ghcr.io/frezbo/crane:0.1.1-alpine
2022/01/04 19:14:19 existing blob: sha256:47db48988e06243b0a6ecb6fb779fc6e4ace26e73100e0d4b75c147a1f687d8a
2022/01/04 19:14:19 existing blob: sha256:9b3977197b4f2147bdd31e1271f811319dcd5c2fc595f14e81f5351ab6275b99
2022/01/04 19:14:20 pushed blob: sha256:6487919bba25ef886b4bef3975dd54acc4f5369be76cfa2270a4e1ca8379458b
2022/01/04 19:14:21 ghcr.io/frezbo/crane:0.1.1-alpine: digest: sha256:4c8d37491a5fe5c55b6c2e0cbeaeea3a8cdd6b80e79dfb70098174a32de0b4d7 size: 591
ghcr.io/frezbo/crane@sha256:4c8d37491a5fe5c55b6c2e0cbeaeea3a8cdd6b80e79dfb70098174a32de0b4d7

Running the above created image:

❯ docker container run --rm --platform linux/arm64 ghcr.io/frezbo/crane@sha256:4c8d37491a5fe5c55b6c2e0cbeaeea3a8cdd6b80e79dfb70098174a32de0b4d7 /crane
Unable to find image 'ghcr.io/frezbo/crane@sha256:4c8d37491a5fe5c55b6c2e0cbeaeea3a8cdd6b80e79dfb70098174a32de0b4d7' locally
ghcr.io/frezbo/crane@sha256:4c8d37491a5fe5c55b6c2e0cbeaeea3a8cdd6b80e79dfb70098174a32de0b4d7: Pulling from frezbo/crane
9b3977197b4f: Already exists 
47db48988e06: Pull complete 
Digest: sha256:4c8d37491a5fe5c55b6c2e0cbeaeea3a8cdd6b80e79dfb70098174a32de0b4d7
Status: Downloaded newer image for ghcr.io/frezbo/crane@sha256:4c8d37491a5fe5c55b6c2e0cbeaeea3a8cdd6b80e79dfb70098174a32de0b4d7
Hey crane from arm64

Appending arm64 binary to tag 0.1.0

❯ crane --platform=linux/arm64 append --base ghcr.io/frezbo/crane@sha256:ee5ae6ab0549bfd514c05f17665b7d355b52942cc29ef0775660e64f6c547664 --new_layer crane_arm64.tar --new_tag ghcr.io/frezbo/crane:0.1.0
2022/01/04 19:16:43 existing blob: sha256:eb23a3f25c9bf62b2b626bda79a42172e34227385e6f06981f85ced4f68ddf35
2022/01/04 19:16:43 existing blob: sha256:59bf1c3509f33515622619af21ed55bbe26d24913cedbca106468a5fb37a50c3
2022/01/04 19:16:43 existing blob: sha256:47db48988e06243b0a6ecb6fb779fc6e4ace26e73100e0d4b75c147a1f687d8a
2022/01/04 19:16:43 existing blob: sha256:a2054f1a0cf4a325c9dc9b80ca4087cffdc867991ebc07bfe24901a41e90e61d
2022/01/04 19:16:44 ghcr.io/frezbo/crane:0.1.0: digest: sha256:19c3381936030184e4d692aeb04bee2a4833dfbd2e0a384238844fbdb2a5a034 size: 755
ghcr.io/frezbo/crane@sha256:19c3381936030184e4d692aeb04bee2a4833dfbd2e0a384238844fbdb2a5a034

Trying to run both arm64 and amd64

Here arm64 doesn't work and trying to run amd64 shows that the amd64 manifest has been replaced by the arm64 binary

❯ docker container run --rm --platform linux/arm64 ghcr.io/frezbo/crane@sha256:19c3381936030184e4d692aeb04bee2a4833dfbd2e0a384238844fbdb2a5a034 /crane
Unable to find image 'ghcr.io/frezbo/crane@sha256:19c3381936030184e4d692aeb04bee2a4833dfbd2e0a384238844fbdb2a5a034' locally
ghcr.io/frezbo/crane@sha256:19c3381936030184e4d692aeb04bee2a4833dfbd2e0a384238844fbdb2a5a034: Pulling from frezbo/crane
Digest: sha256:19c3381936030184e4d692aeb04bee2a4833dfbd2e0a384238844fbdb2a5a034
Status: Image is up to date for ghcr.io/frezbo/crane@sha256:19c3381936030184e4d692aeb04bee2a4833dfbd2e0a384238844fbdb2a5a034
docker: Error response from daemon: image with reference ghcr.io/frezbo/crane@sha256:19c3381936030184e4d692aeb04bee2a4833dfbd2e0a384238844fbdb2a5a034 was found but does not match the specified platform: wanted linux/arm64, actual: linux/amd64.
See 'docker run --help'.

# removing the local cache
❯ docker image rm ghcr.io/frezbo/crane@sha256:19c3381936030184e4d692aeb04bee2a4833dfbd2e0a384238844fbdb2a5a034
Untagged: ghcr.io/frezbo/crane@sha256:19c3381936030184e4d692aeb04bee2a4833dfbd2e0a384238844fbdb2a5a034
Deleted: sha256:a2054f1a0cf4a325c9dc9b80ca4087cffdc867991ebc07bfe24901a41e90e61d
Deleted: sha256:193f0d05cb11ebe943aadf29b473f45ef3472ec38832c8d10396351868795170

❯ docker container run --rm --platform linux/amd64 ghcr.io/frezbo/crane@sha256:19c3381936030184e4d692aeb04bee2a4833dfbd2e0a384238844fbdb2a5a034 /crane
Unable to find image 'ghcr.io/frezbo/crane@sha256:19c3381936030184e4d692aeb04bee2a4833dfbd2e0a384238844fbdb2a5a034' locally
ghcr.io/frezbo/crane@sha256:19c3381936030184e4d692aeb04bee2a4833dfbd2e0a384238844fbdb2a5a034: Pulling from frezbo/crane
59bf1c3509f3: Already exists 
eb23a3f25c9b: Already exists 
47db48988e06: Pull complete 
Digest: sha256:19c3381936030184e4d692aeb04bee2a4833dfbd2e0a384238844fbdb2a5a034
Status: Downloaded newer image for ghcr.io/frezbo/crane@sha256:19c3381936030184e4d692aeb04bee2a4833dfbd2e0a384238844fbdb2a5a034
Hey crane from arm64

[imjasonh]

This is a known limitation. crane append currently appends the tarball onto the single-platform image located using --platform (default is linux/amd64), then pushes a single-platform image back up. It doesn't attempt to update the original multi-platform image to reference the new image it pushed, though docker manifest create --amend should be able to do this for you.

In general I agree crane append (and crane mutate, and crane in general, in lots of places) should do better here. I don't even think the code to do it is hard, just the UX for how we want this all to work. Sometimes you really do want to pull a single image from an index, modify it, and push it back, and differentiating that from the multi-arch use case you describe might be tricky.

[jonjohnsonjr]

Come spitball ideas in #1144 😄

[github-actions]

This issue is stale because it has been open for 90 days with no
activity. It will automatically close after 30 more days of
inactivity. Keep fresh with the 'lifecycle/frozen' label.