Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

buildah manifest add --all does not preserve all Image Manifest metadata #5549

Closed
arewm opened this issue May 28, 2024 · 3 comments · Fixed by containers/common#2021
Closed

buildah manifest add --all does not preserve all Image Manifest metadata #5549

arewm opened this issue May 28, 2024 · 3 comments · Fixed by containers/common#2021
Labels
locked - please file new issue/PR

Comments

@arewm
Copy link

arewm commented May 28, 2024 •
edited
Loading

Description

When I am combining multiple Index Images using buildah manifest add --all, the Image Manifest metadata referenced in each of the added Image Indexes is not preserved.

Steps to reproduce the issue:

am@dev-fedora:~/buildah-playground$ buildah manifest create quay.io/arewm/reproducer:index1
b61f9dd58eca49b38d853e65e36a74e3d6ff35531070680af49cd4faa8415f7c
am@dev-fedora:~/buildah-playground$ buildah manifest create quay.io/arewm/reproducer:index2
a779ab7029a8731fec4ea066ea2ba930ed695799ca1f6dfba7ed368510c409b2
am@dev-fedora:~/buildah-playground$ buildah manifest add quay.io/arewm/reproducer:index1 --artifact index1/foo.bar --arch arm64 --os linux --artifact-type application/vnd.reproducer+bar
b61f9dd58eca49b38d853e65e36a74e3d6ff35531070680af49cd4faa8415f7c: sha256:abcefd632b768122eabc84bc82b19e1bb6cc4cb3da722d16ae4415b04e2a2ff6
am@dev-fedora:~/buildah-playground$ buildah manifest add quay.io/arewm/reproducer:index1 --artifact index1/baz.foo --arch amd64 --os darwin --artifact-type application/vnd.reproducer+foo
b61f9dd58eca49b38d853e65e36a74e3d6ff35531070680af49cd4faa8415f7c: sha256:ca133c95d8ac724003424292e2314b35363cc102108ad0fbfaf024194f437458
am@dev-fedora:~/buildah-playground$ buildah manifest add quay.io/arewm/reproducer:index2 --artifact index2/baz.bar --arch s390x --os linux --artifact-type application/vnd.reproducer+bar
a779ab7029a8731fec4ea066ea2ba930ed695799ca1f6dfba7ed368510c409b2: sha256:5d3e80631828ae152e6fa4e9ea1eafb0a52e60a5ac6549dc022de8d9a5f9ed6c
am@dev-fedora:~/buildah-playground$ buildah manifest add quay.io/arewm/reproducer:index2 --artifact index2/bar.foo --arch ppc64le --os linux --artifact-type application/vnd.reproducer+foo
a779ab7029a8731fec4ea066ea2ba930ed695799ca1f6dfba7ed368510c409b2: sha256:ec9b106af76a31aeae1a1dd6fe9037359746387a3f0fa2bc20fdc239b3e94a1d
am@dev-fedora:~/buildah-playground$ buildah manifest push --all quay.io/arewm/reproducer:index1
Getting image list signatures
Copying 2 images generated from 2 images in list
Copying image sha256:abcefd632b768122eabc84bc82b19e1bb6cc4cb3da722d16ae4415b04e2a2ff6 (1/2)
Getting image source signatures
Copying blob e3b0c44298fc done
Copying config 44136fa355 done   |
Writing manifest to image destination
Copying image sha256:ca133c95d8ac724003424292e2314b35363cc102108ad0fbfaf024194f437458 (2/2)
Getting image source signatures
Copying blob e3b0c44298fc skipped: already exists
Copying config 44136fa355 done   |
Writing manifest to image destination
Writing manifest list to image destination
Storing list signatures
am@dev-fedora:~/buildah-playground$ buildah manifest push --all quay.io/arewm/reproducer:index2
Getting image list signatures
Copying 2 images generated from 2 images in list
Copying image sha256:5d3e80631828ae152e6fa4e9ea1eafb0a52e60a5ac6549dc022de8d9a5f9ed6c (1/2)
Getting image source signatures
Copying blob e3b0c44298fc skipped: already exists
Copying config 44136fa355 done   |
Writing manifest to image destination
Copying image sha256:ec9b106af76a31aeae1a1dd6fe9037359746387a3f0fa2bc20fdc239b3e94a1d (2/2)
Getting image source signatures
Copying blob e3b0c44298fc skipped: already exists
Copying config 44136fa355 done   |
Writing manifest to image destination
Writing manifest list to image destination
Storing list signatures
am@dev-fedora:~/buildah-playground$ buildah manifest create quay.io/arewm/reproducer:index3
400b388c0b8ff4568b64a8c963691edb80fd90a24112e0e65e883c9e07d9309a
am@dev-fedora:~/buildah-playground$ buildah manifest add quay.io/arewm/reproducer:index3 docker://quay.io/arewm/reproducer:index1 --all
400b388c0b8ff4568b64a8c963691edb80fd90a24112e0e65e883c9e07d9309a: sha256:abcefd632b768122eabc84bc82b19e1bb6cc4cb3da722d16ae4415b04e2a2ff6
am@dev-fedora:~/buildah-playground$ buildah manifest add quay.io/arewm/reproducer:index3 docker://quay.io/arewm/reproducer:index2 --all
400b388c0b8ff4568b64a8c963691edb80fd90a24112e0e65e883c9e07d9309a: sha256:5d3e80631828ae152e6fa4e9ea1eafb0a52e60a5ac6549dc022de8d9a5f9ed6c
am@dev-fedora:~/buildah-playground$ buildah manifest push --all quay.io/arewm/reproducer:index3
Getting image list signatures
Copying 4 images generated from 4 images in list
Copying image sha256:abcefd632b768122eabc84bc82b19e1bb6cc4cb3da722d16ae4415b04e2a2ff6 (1/4)
Getting image source signatures
Copying blob e3b0c44298fc skipped: already exists
Copying config 44136fa355 done   |
Writing manifest to image destination
Copying image sha256:ca133c95d8ac724003424292e2314b35363cc102108ad0fbfaf024194f437458 (2/4)
Getting image source signatures
Copying blob e3b0c44298fc skipped: already exists
Copying config 44136fa355 done   |
Writing manifest to image destination
Copying image sha256:5d3e80631828ae152e6fa4e9ea1eafb0a52e60a5ac6549dc022de8d9a5f9ed6c (3/4)
Getting image source signatures
Copying blob e3b0c44298fc skipped: already exists
Copying config 44136fa355 done   |
Writing manifest to image destination
Copying image sha256:ec9b106af76a31aeae1a1dd6fe9037359746387a3f0fa2bc20fdc239b3e94a1d (4/4)
Getting image source signatures
Copying blob e3b0c44298fc skipped: already exists
Copying config 44136fa355 done   |
Writing manifest to image destination
Writing manifest list to image destination
Storing list signatures

Describe the results you received:

$ oras manifest fetch quay.io/arewm/reproducer:index1 | jq
{
  "schemaVersion": 2,
  "mediaType": "application/vnd.oci.image.index.v1+json",
  "manifests": [
    {
      "mediaType": "application/vnd.oci.image.manifest.v1+json",
      "digest": "sha256:abcefd632b768122eabc84bc82b19e1bb6cc4cb3da722d16ae4415b04e2a2ff6",
      "size": 478,
      "platform": {
        "architecture": "arm64",
        "os": "linux"
      },
      "artifactType": "application/vnd.reproducer+bar"
    },
    {
      "mediaType": "application/vnd.oci.image.manifest.v1+json",
      "digest": "sha256:ca133c95d8ac724003424292e2314b35363cc102108ad0fbfaf024194f437458",
      "size": 478,
      "platform": {
        "architecture": "amd64",
        "os": "darwin"
      },
      "artifactType": "application/vnd.reproducer+foo"
    }
  ]
}
$ oras manifest fetch quay.io/arewm/reproducer:index2 | jq
{
  "schemaVersion": 2,
  "mediaType": "application/vnd.oci.image.index.v1+json",
  "manifests": [
    {
      "mediaType": "application/vnd.oci.image.manifest.v1+json",
      "digest": "sha256:5d3e80631828ae152e6fa4e9ea1eafb0a52e60a5ac6549dc022de8d9a5f9ed6c",
      "size": 478,
      "platform": {
        "architecture": "s390x",
        "os": "linux"
      },
      "artifactType": "application/vnd.reproducer+bar"
    },
    {
      "mediaType": "application/vnd.oci.image.manifest.v1+json",
      "digest": "sha256:ec9b106af76a31aeae1a1dd6fe9037359746387a3f0fa2bc20fdc239b3e94a1d",
      "size": 478,
      "platform": {
        "architecture": "ppc64le",
        "os": "linux"
      },
      "artifactType": "application/vnd.reproducer+foo"
    }
  ]
}

$ oras manifest fetch quay.io/arewm/reproducer:index3 | jq
{
  "schemaVersion": 2,
  "mediaType": "application/vnd.oci.image.index.v1+json",
  "manifests": [
    {
      "mediaType": "application/vnd.oci.image.manifest.v1+json",
      "digest": "sha256:abcefd632b768122eabc84bc82b19e1bb6cc4cb3da722d16ae4415b04e2a2ff6",
      "size": 478,
      "platform": {
        "architecture": "arm64",
        "os": "linux"
      }
    },
    {
      "mediaType": "application/vnd.oci.image.manifest.v1+json",
      "digest": "sha256:ca133c95d8ac724003424292e2314b35363cc102108ad0fbfaf024194f437458",
      "size": 478,
      "platform": {
        "architecture": "amd64",
        "os": "darwin"
      }
    },
    {
      "mediaType": "application/vnd.oci.image.manifest.v1+json",
      "digest": "sha256:5d3e80631828ae152e6fa4e9ea1eafb0a52e60a5ac6549dc022de8d9a5f9ed6c",
      "size": 478,
      "platform": {
        "architecture": "s390x",
        "os": "linux"
      }
    },
    {
      "mediaType": "application/vnd.oci.image.manifest.v1+json",
      "digest": "sha256:ec9b106af76a31aeae1a1dd6fe9037359746387a3f0fa2bc20fdc239b3e94a1d",
      "size": 478,
      "platform": {
        "architecture": "ppc64le",
        "os": "linux"
      }
    }
  ]
}

Describe the results you expected:

$ oras manifest fetch quay.io/arewm/reproducer:index3 | jq
{
  "schemaVersion": 2,
  "mediaType": "application/vnd.oci.image.index.v1+json",
  "manifests": [
    {
      "mediaType": "application/vnd.oci.image.manifest.v1+json",
      "digest": "sha256:abcefd632b768122eabc84bc82b19e1bb6cc4cb3da722d16ae4415b04e2a2ff6",
      "size": 478,
      "platform": {
        "architecture": "arm64",
        "os": "linux"
      },
      "artifactType": "application/vnd.reproducer+bar"
    },
    {
      "mediaType": "application/vnd.oci.image.manifest.v1+json",
      "digest": "sha256:ca133c95d8ac724003424292e2314b35363cc102108ad0fbfaf024194f437458",
      "size": 478,
      "platform": {
        "architecture": "amd64",
        "os": "darwin"
      },
      "artifactType": "application/vnd.reproducer+foo"
    },
    {
      "mediaType": "application/vnd.oci.image.manifest.v1+json",
      "digest": "sha256:5d3e80631828ae152e6fa4e9ea1eafb0a52e60a5ac6549dc022de8d9a5f9ed6c",
      "size": 478,
      "platform": {
        "architecture": "s390x",
        "os": "linux"
      },
      "artifactType": "application/vnd.reproducer+bar"
    },
    {
      "mediaType": "application/vnd.oci.image.manifest.v1+json",
      "digest": "sha256:ec9b106af76a31aeae1a1dd6fe9037359746387a3f0fa2bc20fdc239b3e94a1d",
      "size": 478,
      "platform": {
        "architecture": "ppc64le",
        "os": "linux"
      },
      "artifactType": "application/vnd.reproducer+foo"
    }
  ]
}

Output of rpm -q buildah or apt list buildah:

$ rpm -q buildah
buildah-1.35.4-1.fc39.aarch64

Output of buildah version:

$ buildah version
Version:         1.35.4
Go Version:      go1.21.9
Image Spec:      1.1.0
Runtime Spec:    1.1.0
CNI Spec:        1.0.0
libcni Version:
image Version:   5.30.1
Git Commit:
Built:           Fri May 10 09:41:31 2024
OS/Arch:         linux/arm64
BuildPlatform:   linux/arm64/v8

Output of cat /etc/*release:

$ cat /etc/*release
Fedora release 39 (Thirty Nine)
NAME="Fedora Linux"
VERSION="39 (Workstation Edition)"
ID=fedora
VERSION_ID=39
VERSION_CODENAME=""
PLATFORM_ID="platform:f39"
PRETTY_NAME="Fedora Linux 39 (Workstation Edition)"
ANSI_COLOR="0;38;2;60;110;180"
LOGO=fedora-logo-icon
CPE_NAME="cpe:/o:fedoraproject:fedora:39"
DEFAULT_HOSTNAME="fedora"
HOME_URL="https://fedoraproject.org/"
DOCUMENTATION_URL="https://docs.fedoraproject.org/en-US/fedora/f39/system-administrators-guide/"
SUPPORT_URL="https://ask.fedoraproject.org/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"
REDHAT_BUGZILLA_PRODUCT="Fedora"
REDHAT_BUGZILLA_PRODUCT_VERSION=39
REDHAT_SUPPORT_PRODUCT="Fedora"
REDHAT_SUPPORT_PRODUCT_VERSION=39
SUPPORT_END=2024-11-12
VARIANT="Workstation Edition"
VARIANT_ID=workstation
Fedora release 39 (Thirty Nine)
Fedora release 39 (Thirty Nine)

Output of uname -a:

$ uname -a
Linux dev-fedora 6.5.6-100.fc37.aarch64 #1 SMP PREEMPT_DYNAMIC Fri Oct  6 19:25:44 UTC 2023 aarch64 GNU/Linux

Output of cat /etc/containers/storage.conf:

$ cat /etc/containers/storage.conf
cat: /etc/containers/storage.conf: No such file or directory
@nalind nalind mentioned this issue May 28, 2024
Merged
@nalind
Copy link
Member

nalind commented May 28, 2024

(The change will probably show up in containers/common v0.60, I'm guessing in a v1.37 release here, if not sooner.)

@arewm
Copy link
Author

arewm commented May 29, 2024

Thanks, do you know what the timeline for those would be?

@arewm arewm changed the title buildad manifest add --all does not preserve all Image Manifest metadata buildah manifest add --all does not preserve all Image Manifest metadata May 29, 2024
@TomSweeneyRedHat
Copy link
Member

TomSweeneyRedHat commented May 29, 2024 •
edited
Loading

@arewm, we just released Buildah v1.36. We don't have set release timelines, but they are generally every couple of months. I suspect v1.37 will be out around July of this year.

@stale-locking-app stale-locking-app bot locked as resolved and limited conversation to collaborators Oct 9, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
locked - please file new issue/PR
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

libimage/manifests.list.Add(): preserve artifactType when all=true nalind/containers-common
3 participants
@nalind @arewm @TomSweeneyRedHat