Add support for OCI artifacts “attached” to an image via subject

[SamirPS]

Hello,

i'm using skopeo version 1.11.1-dev commit: 0fdd10491e154bb9c8021ff80fce29981d59887b and here are the steps I do :

1. Get the ubuntu:lunar image from docker and copy it to an oci image:

skopeo copy --multi-arch all docker://ubuntu:lunar oci:dev:test

Output:

Getting image list signatures
Copying 5 of 5 images in list
Copying image sha256:52293638ba652a2e8f9e1c1cfcc905839b1f2a9e671ddcc9bf77909b6bf527d0 (1/5)
Getting image source signatures
Copying blob db781b8aed49 done  
Copying config beb2152822 done  
Writing manifest to image destination
Storing signatures
Copying image sha256:9fbac10f6899f18c2a614e0d7dc2a38af1559e5f0995d3b70bd8c1d57401ff76 (2/5)
Getting image source signatures
Copying blob 2cf733a376b3 done  
Copying config 173233681d done  
Writing manifest to image destination
Storing signatures
Copying image sha256:0c8e3367a3fe9b703c759e1c148c5809df1a2734f8f37529bd11fbcfd34b1d1c (3/5)
Getting image source signatures
Copying blob 29d183ded65a done  
Copying config eb2d2fb228 done  
Writing manifest to image destination
Storing signatures
Copying image sha256:a8498f0e21515679ba0037a81a4ab642c1d95710be8559a451c53df1d796fe06 (4/5)
Getting image source signatures
Copying blob b536935eabb9 done  
Copying config e3034f5021 done  
Writing manifest to image destination
Storing signatures
Copying image sha256:af9488f1c2d5b6b3d0052c1bba305d8d532dea8c909a67bf8e4ab80fb941eaef (5/5)
Getting image source signatures
Copying blob 24a981375fc3 done  
Copying config 419a9b2f25 done  
Writing manifest to image destination
Storing signatures
Writing manifest list to image destination
Storing list signatures

2. Use syft to create my sbom file

syft ubuntu:lunar -o spdx-json > toattach.spdx.json

3. Using oras to attach the sbom file to the oci layout

 oras attach --oci-layout --artifact-type "application/spdx+json" dev:test toattach.spdx.json -v

Output:

Preparing toattach.spdx.json
Uploading 3ef57bfb7fdc toattach.spdx.json
Uploaded  3ef57bfb7fdc toattach.spdx.json
Uploading 3acbe5645b6c application/vnd.oci.artifact.manifest.v1+json
Uploaded  3acbe5645b6c application/vnd.oci.artifact.manifest.v1+json
Attached to [oci-layout] dev@sha256:678c136e4e918eff1c5e83383859e1ea2227d75ae5fe662c9122711aa6594a2a
Digest: sha256:3acbe5645b6cfe007c4851e77f3afdeab18319639c616d477d19bc3f987c2185

4. check if it's attached

oras discover --oci-layout dev:test -v -o tree

Output:

test
└── application/spdx+json
    └── sha256:3acbe5645b6cfe007c4851e77f3afdeab18319639c616d477d19bc3f987c2185
        └── org.opencontainers.artifact.created: "2023-02-13T17:25:58Z"

5. Use cp to copy this oci image to another oci image

skopeo copy --multi-arch all oci:dev:test oci:dev:issue

Output:

Getting image list signatures
Copying 5 of 5 images in list
Copying image sha256:52293638ba652a2e8f9e1c1cfcc905839b1f2a9e671ddcc9bf77909b6bf527d0 (1/5)
Getting image source signatures
Copying blob db781b8aed49 skipped: already exists  
Copying config beb2152822 done  
Writing manifest to image destination
Storing signatures
Copying image sha256:9fbac10f6899f18c2a614e0d7dc2a38af1559e5f0995d3b70bd8c1d57401ff76 (2/5)
Getting image source signatures
Copying blob 2cf733a376b3 skipped: already exists  
Copying config 173233681d done  
Writing manifest to image destination
Storing signatures
Copying image sha256:0c8e3367a3fe9b703c759e1c148c5809df1a2734f8f37529bd11fbcfd34b1d1c (3/5)
Getting image source signatures
Copying blob 29d183ded65a skipped: already exists  
Copying config eb2d2fb228 done  
Writing manifest to image destination
Storing signatures
Copying image sha256:a8498f0e21515679ba0037a81a4ab642c1d95710be8559a451c53df1d796fe06 (4/5)
Getting image source signatures
Copying blob b536935eabb9 skipped: already exists  
Copying config e3034f5021 done  
Writing manifest to image destination
Storing signatures
Copying image sha256:af9488f1c2d5b6b3d0052c1bba305d8d532dea8c909a67bf8e4ab80fb941eaef (5/5)
Getting image source signatures
Copying blob 24a981375fc3 skipped: already exists  
Copying config 419a9b2f25 done  
Writing manifest to image destination
Storing signatures
Writing manifest list to image destination
Storing list signatures

6. Recheck if the sbom is still attached

oras discover --oci-layout dev:issue -v -o tree

output

issue
└── application/spdx+json
    └── sha256:3acbe5645b6cfe007c4851e77f3afdeab18319639c616d477d19bc3f987c2185
        └── org.opencontainers.artifact.created: "2023-02-13T17:25:58Z"

So right now, it's good.

But now, if I do cp from oci: to docker://, I see this.

skopeo copy --multi-arch all oci:dev:test docker://xxx.azurecr.io/artifactstest:skopeoissuefeb

output

Getting image list signatures
Copying 5 of 5 images in list
Copying image sha256:52293638ba652a2e8f9e1c1cfcc905839b1f2a9e671ddcc9bf77909b6bf527d0 (1/5)
Getting image source signatures
Copying blob db781b8aed49 skipped: already exists  
Copying config beb2152822 done  
Writing manifest to image destination
Storing signatures
Copying image sha256:9fbac10f6899f18c2a614e0d7dc2a38af1559e5f0995d3b70bd8c1d57401ff76 (2/5)
Getting image source signatures
Copying blob 2cf733a376b3 done  
Copying config 173233681d done  
Writing manifest to image destination
Storing signatures
Copying image sha256:0c8e3367a3fe9b703c759e1c148c5809df1a2734f8f37529bd11fbcfd34b1d1c (3/5)
Getting image source signatures
Copying blob 29d183ded65a done  
Copying config eb2d2fb228 done  
Writing manifest to image destination
Storing signatures
Copying image sha256:a8498f0e21515679ba0037a81a4ab642c1d95710be8559a451c53df1d796fe06 (4/5)
Getting image source signatures
Copying blob b536935eabb9 done  
Copying config e3034f5021 done  
Writing manifest to image destination
Storing signatures
Copying image sha256:af9488f1c2d5b6b3d0052c1bba305d8d532dea8c909a67bf8e4ab80fb941eaef (5/5)
Getting image source signatures
Copying blob 24a981375fc3 done  
Copying config 419a9b2f25 done  
Writing manifest to image destination
Storing signatures
Writing manifest list to image destination
Storing list signatures

When I do oras discover xxx.azurecr.io/artifactstest:skopeoissuefeb

I have this output:

Discovered 0 artifact referencing skopeoissuefeb
Digest: sha256:678c136e4e918eff1c5e83383859e1ea2227d75ae5fe662c9122711aa6594a2a

So right now, what I understand skopeo copy artefact when the source and dest are oci but not if the source is oci and dest is docker:// it's normal?

[mtrmac]

Thanks for your report.

(What happens here is that this adds an artifact with a subject field that refers to the dev:test image, per https://github.com/opencontainers/image-spec/blob/main/artifact.md / https://github.com/opencontainers/distribution-spec/blob/main/spec.md#listing-referrers ).

5. Use cp to copy this oci image to another oci image

This really does nothing relevant; Skopeo is completely unaware of the attached artifact, but because this copy within the same oci: repository does not change the manifest digest, the artifact seems to stay attached.

---

The real issue is that Skopeo, and c/image in general, just does not support the concept of OCI artifacts indirectly attached using the subject field.

If you can identify those artifacts manually, you can probably copy them manually with skopeo copy, one artifact at a time, but a skopeo copy of an currently does not find nor copy any of such artifacts automatically.

We have some parts of the infrastructure for this now in c/image, for use-sigstore-attachments (which is a somewhat similar concept); but not exactly this.

---

This would have to be implemented in c/image, then Skopeo would automatically inherit the support. So, moving this RFE there.

[SamirPS]

Okay thanks for your answer

If you can identify those artifacts manually, you can probably copy them manually with skopeo copy, one artifact at a time, but a skopeo copy of an currently does not find nor copy any of such artifacts automatically.

But if I do this, the sbom file will not be attached to my oci image, or did I miss something?

[mtrmac]

The “attachment” is not a separate physical link that needs to be copied; it is a semantic feature purely caused by the artifact having a subject, and the registry, or the consumer of the OCI directory, looking for that link.

So you can upload that artifact to any registry (possibly having to invent a tag for that upload); once it is uploaded, registries that understand the “referrer” concept from the OCI distribution spec are expected to notice the reference, and allow querying it. Registries that don’t understand the “referrer” context would just store the artifact, and it would not be found, or it would have to be found by iterating all tags in that repo.