fix: disable oras tag fallback to tag schema when tagging a referrer #1435

njucjc · 2024-06-28T13:54:53Z

What this PR does / why we need it:

disable oras tag fallback to tag schema when tagging a referrer

Please check the following list:

Does the affected code have corresponding tests, e.g. unit test, E2E test?
Does this change require a documentation update?
Does this introduce breaking changes that would require an announcement or bumping the major version?
Do all new files have an appropriate license header?

Signed-off-by: njucjc <njucjc@gmail.com>

cmd/oras/root/tag.go

qweeah · 2024-06-28T14:03:24Z

Hi @njucjc, thanks for contributing. Can you help explain why --distribution-spec flag need to be added to oras tag?

njucjc · 2024-06-28T14:18:00Z

Hi @njucjc, thanks for contributing. Can you help explain why --distribution-spec flag need to be added to oras tag?

@qweeah I want to tag a referrer, but now if not specify --distribution-spec v1.1-referrers-api, it will fallback to --distribution-spec v1.1-referrers-tag method. The result is that an unexpected addition of a sha256 subject tag in addition to the one I specified.

qweeah · 2024-06-28T14:24:47Z

I want to tag a referrer, but now if not specify --distribution-spec v1.1-referrers-api, it will fallback to --distribution-spec v1.1-referrers-tag method.

That's weird because --distribution-spec should only work for oras attach when adding a referrer. Can you kindly share the error debug log and which registry you are using?

njucjc · 2024-06-28T14:36:20Z

That's weird because --distribution-spec should only work for oras attach when adding a referrer. Can you kindly share the error debug log and which registry you are using?

@qweeah

DEBU[0000] Request #0
> Request method: "GET"
> Request headers:
   "Accept": "application/vnd.docker.distribution.manifest.v2+json, application/vnd.docker.distribution.manifest.list.v2+json, application/vnd.oci.image.manifest.v1+json, application/vnd.oci.image.index.v1+json, application/vnd.oci.artifact.manifest.v1+json"
   "User-Agent": "oras/1.2.0"
DEBU[0000] Response #0
< Response Status: "401 Unauthorized"
< Response headers:
   "Content-Length": "157"
   "Docker-Distribution-Api-Version": "registry/2.0"
   "Date": "Fri, 28 Jun 2024 14:31:27 GMT"
   "Content-Type": "application/json; charset=utf-8"
DEBU[0000] Request #1
> Request method: "GET"
> Request headers:
   "Authorization": "*****"
   "User-Agent": "oras/1.2.0"
DEBU[0000] Response #1
< Response Status: "200 OK"
< Response headers:
   "X-Xss-Protection": "1; mode=block"
   "Cache-Control": "no-cache, no-store, max-age=0, must-revalidate"
   "Pragma": "no-cache"
   "Timing-Allow-Origin": "*"
   "Server": "Tengine"
   "Vary": "Accept-Encoding"
   "X-Content-Type-Options": "nosniff"
   "Date": "Fri, 28 Jun 2024 14:31:27 GMT"
   "Strict-Transport-Security": "max-age=31536000"
   "Eagleeye-Traceid": "0a06dff517195850875481284ea292"
   "Content-Type": "application/json;charset=UTF-8"
   "Set-Cookie": "*****"
   "Expires": "0"
   "X-Frame-Options": "DENY"
DEBU[0000] Request #2
> Request URL: "https:/xxxx/manifests/sha256:bd6e6c6fc03d1f15108de17531082501997335eacfa7ec8cb805430e35fc57eb"
> Request method: "GET"
> Request headers:
   "Accept": "application/vnd.docker.distribution.manifest.v2+json, application/vnd.docker.distribution.manifest.list.v2+json, application/vnd.oci.image.manifest.v1+json, application/vnd.oci.image.index.v1+json, application/vnd.oci.artifact.manifest.v1+json"
   "Authorization": "*****"
   "User-Agent": "oras/1.2.0"
DEBU[0000] Response #2
< Response Status: "200 OK"
< Response headers:
   "Date": "Fri, 28 Jun 2024 14:31:27 GMT"
   "Content-Type": "application/vnd.oci.image.index.v1+json"
   "Content-Length": "736"
   "Docker-Content-Digest": "sha256:bd6e6c6fc03d1f15108de17531082501997335eacfa7ec8cb805430e35fc57eb"
   "Docker-Distribution-Api-Version": "registry/2.0"
   "Etag": "\"sha256:bd6e6c6fc03d1f15108de17531082501997335eacfa7ec8cb805430e35fc57eb\""
Tagging [registry] xxxxx/library/test-2@sha256:bd6e6c6fc03d1f15108de17531082501997335eacfa7ec8cb805430e35fc57eb
DEBU[0000] Request #3
> Request URL: "https://xxxxx/v2/library/test-2/manifests/test"
> Request method: "PUT"
> Request headers:
   "Content-Type": "application/vnd.oci.image.index.v1+json"
   "User-Agent": "oras/1.2.0"
DEBU[0000] Response #3
< Response Status: "401 Unauthorized"
< Response headers:
   "Content-Length": "230"
   "Docker-Distribution-Api-Version": "registry/2.0"
   "Www-Authenticate": "Bearer realm=\
   "Date": "Fri, 28 Jun 2024 14:31:27 GMT"
   "Content-Type": "application/json; charset=utf-8"
DEBU[0000] Request #4
> Request URL: 
> Request method: "GET"
> Request headers:
   "Authorization": "*****"
   "User-Agent": "oras/1.2.0"
DEBU[0001] Response #4
< Response Status: "200 OK"
< Response headers:
   "Date": "Fri, 28 Jun 2024 14:31:28 GMT"
   "X-Frame-Options": "DENY"
   "Strict-Transport-Security": "max-age=31536000"
   "Set-Cookie": "*****"
   "X-Xss-Protection": "1; mode=block"
   "Cache-Control": "no-cache, no-store, max-age=0, must-revalidate"
   "Pragma": "no-cache"
   "Expires": "0"
   "Eagleeye-Traceid": "0a06dff517195850879751310ea292"
   "Timing-Allow-Origin": "*"
   "Server": "Tengine"
   "Content-Type": "application/json;charset=UTF-8"
   "Vary": "Accept-Encoding"
   "X-Content-Type-Options": "nosniff"
DEBU[0001] Request #5
> Request URL: "https://xxxx/v2/library/test-2/manifests/test"
> Request method: "PUT"
> Request headers:
   "Content-Type": "application/vnd.oci.image.index.v1+json"
   "Authorization": "*****"
   "User-Agent": "oras/1.2.0"
DEBU[0001] Response #5
< Response Status: "201 Created"
< Response headers:
   "Date": "Fri, 28 Jun 2024 14:31:28 GMT"
   "Content-Length": "0"
   "Location": "https://xxxx/v2/library/test-2/manifests/sha256:bd6e6c6fc03d1f15108de17531082501997335eacfa7ec8cb805430e35fc57eb"
   "Docker-Content-Digest": "sha256:bd6e6c6fc03d1f15108de17531082501997335eacfa7ec8cb805430e35fc57eb"
   "Docker-Distribution-Api-Version": "registry/2.0"
DEBU[0001] Request #6
> Request URL: "https://xxxxxxx/v2/library/test-2/manifests/sha256-9eaf39d0c368f3ea629401810641cb677c412953e9d0f28e8b26f6cd8653cfde"
> Request method: "GET"
> Request headers:
   "User-Agent": "oras/1.2.0"
   "Accept": "application/vnd.docker.distribution.manifest.v2+json, application/vnd.docker.distribution.manifest.list.v2+json, application/vnd.oci.image.manifest.v1+json, application/vnd.oci.image.index.v1+json, application/vnd.oci.artifact.manifest.v1+json"
   "Authorization": "*****"
DEBU[0001] Response #6
< Response Status: "200 OK"
< Response headers:
   "Date": "Fri, 28 Jun 2024 14:31:28 GMT"
   "Content-Type": "application/vnd.oci.image.index.v1+json"
   "Content-Length": "302"
   "Docker-Content-Digest": "sha256:ce9946afdcd5298cc6da3940e2ebee9583e1c87a71d9ad2e5e0d2792f59626d5"
   "Docker-Distribution-Api-Version": "registry/2.0"
   "Etag": "\"sha256:ce9946afdcd5298cc6da3940e2ebee9583e1c87a71d9ad2e5e0d2792f59626d5\""
Tagged test

the registry (support referrers api as distribution-spec v1.1) appears two tags: test and sha256-9eaf39d0c368f3ea629401810641cb677c412953e9d0f28e8b26f6cd8653cfde

By the way,oras cp subcommand will also use --from-distribution-spec and --to-distribution-spec flag to indicate that the remote registries support the Referrers API.

codecov · 2024-06-28T14:54:18Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 85.22%. Comparing base (ce758bc) to head (8bd4797).

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1435      +/-   ##
==========================================
- Coverage   85.27%   85.22%   -0.05%     
==========================================
  Files         106      106              
  Lines        3796     3798       +2     
==========================================
  Hits         3237     3237              
- Misses        334      336       +2     
  Partials      225      225

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

qweeah · 2024-06-28T15:01:02Z

the registry (support referrers api as distribution-spec v1.1) appears two tags: test and sha256-9eaf39d0c368f3ea629401810641cb677c412953e9d0f28e8b26f6cd8653cfde

Just to confirm, so you are trying to avoid generating sha256-9eaf39d0c368f3ea629401810641cb677c412953e9d0f28e8b26f6cd8653cfde and oras tag --distribution-spec v1.1-referrers-api is the desired behavior in your case, right?

shizhMSFT · 2024-06-28T15:02:42Z

Although tagging a referrer artifact manifest is discouraged, it is still doable. However, re-tagging is implemented as pushing the exact manifest again to the remote server. According to the distribution-spec, if the remote server does not support referrers API, it will go for the referrers tag schema case. However, it is not necessary since the manifest is already there, we are just adding a new tag not really pushing a new manifest.

In summary, tagging a referrers manifest should never go for the tag schema even if the server does not support referrers API since the referrer manifest is already there on the server.

njucjc · 2024-06-28T23:55:29Z

the registry (support referrers api as distribution-spec v1.1) appears two tags: test and sha256-9eaf39d0c368f3ea629401810641cb677c412953e9d0f28e8b26f6cd8653cfde

Just to confirm, so you are trying to avoid generating sha256-9eaf39d0c368f3ea629401810641cb677c412953e9d0f28e8b26f6cd8653cfde and oras tag --distribution-spec v1.1-referrers-api is the desired behavior in your case, right?

@qweeah Yes!

njucjc · 2024-06-28T23:56:00Z

Although tagging a referrer artifact manifest is discouraged, it is still doable. However, re-tagging is implemented as pushing the exact manifest again to the remote server. According to the distribution-spec, if the remote server does not support referrers API, it will go for the referrers tag schema case. However, it is not necessary since the manifest is already there, we are just adding a new tag not really pushing a new manifest.

In summary, tagging a referrers manifest should never go for the tag schema even if the server does not support referrers API since the referrer manifest is already there on the server.

@shizhMSFT @qweeah Is there any suggestions to fix it? The simplest way to disable the tag schema is set referrersAPI = true inside the registry client in oras tag subcommand.

…rrers Signed-off-by: Billy Zha <jinzha1@microsoft.com>

qweeah · 2024-06-29T04:22:59Z

@shizhMSFT @qweeah Is there any suggestions to fix it? The simplest way to disable the tag schema is set referrersAPI = true inside the registry client in oras tag subcommand.

@njucjc Yes, we should disable tag schema without adding --distribution-spec to oras tag. If you are interested in fixing it, you may achieve that using https://pkg.go.dev/oras.land/oras-go/v2@v2.5.0/registry/remote#Repository.SetReferrersCapability.

Signed-off-by: Billy Zha <jinzha1@microsoft.com>

qweeah · 2024-06-29T04:43:29Z

I have added two test specs, one of them ensures that tag schema is not used after tagging to an OCI v1.0 registry (which doesn't support referrers API). @njucjc With a proper fix in oras tag the test specs should pass.

njucjc · 2024-07-01T02:04:12Z

I have added two test specs, one of them ensures that tag schema is not used after tagging to an OCI v1.0 registry (which doesn't support referrers API). @njucjc With a proper fix in oras tag the test specs should pass.

@qweeah PTAL

Signed-off-by: njucjc <njucjc@gmail.com>

Signed-off-by: Billy Zha <jinzha1@microsoft.com>

qweeah

LGTM, thanks @njucjc

qweeah

Oh one required change I have missed: the type assertion should be done after the error handling.

cmd/oras/root/tag.go

Signed-off-by: Billy Zha <jinzha1@microsoft.com>

qweeah

LGTM

…hub.com/njucjc/oras into pr/njucjc/1435

Signed-off-by: Billy Zha <jinzha1@microsoft.com>

shizhMSFT

LGTM

…ras-project#1435) Signed-off-by: njucjc <njucjc@gmail.com> Signed-off-by: Billy Zha <jinzha1@microsoft.com> Co-authored-by: Billy Zha <jinzha1@microsoft.com>

…ras-project#1435) Signed-off-by: njucjc <njucjc@gmail.com> Signed-off-by: Billy Zha <jinzha1@microsoft.com> Co-authored-by: Billy Zha <jinzha1@microsoft.com> Signed-off-by: Billy Zha <jinzha1@microsoft.com>

feat: support oras tag enable distribution spec flag

db53da1

Signed-off-by: njucjc <njucjc@gmail.com>

njucjc requested review from sajayantony, shizhMSFT, SteveLasker, qweeah and TerryHowe as code owners June 28, 2024 13:54

TerryHowe reviewed Jun 28, 2024

View reviewed changes

cmd/oras/root/tag.go Outdated Show resolved Hide resolved

qweeah mentioned this pull request Jun 28, 2024

oras tag should not create referrers tags #1436

Closed

test(e2e): spec to confirm no referrers index added when tagging refe…

6af2b44

…rrers Signed-off-by: Billy Zha <jinzha1@microsoft.com>

qweeah added 3 commits June 29, 2024 04:24

test(e2e): spec clean

e4ea3a1

Signed-off-by: Billy Zha <jinzha1@microsoft.com>

test(e2e): correct repo name

3e46f8c

Signed-off-by: Billy Zha <jinzha1@microsoft.com>

test(e2e): simplify preparation for OCI 1.0 registries

1cd85fe

Signed-off-by: Billy Zha <jinzha1@microsoft.com>

njucjc changed the title ~~feat: support oras tag enable distribution spec flag~~ fix: disable oras tag fallback to tag schema when tagging a referrer Jul 1, 2024

fix: disable oras tag fallback to tag schema when tagging a referrer

e19fdea

Signed-off-by: njucjc <njucjc@gmail.com>

njucjc force-pushed the oras-tag-support-distribution-spec-flag branch from 7599361 to e19fdea Compare July 1, 2024 02:05

test(e2e): avoid tagging twice in ZOT registry

46b1785

Signed-off-by: Billy Zha <jinzha1@microsoft.com>

qweeah approved these changes Jul 1, 2024

View reviewed changes

qweeah requested changes Jul 1, 2024

View reviewed changes

cmd/oras/root/tag.go Outdated Show resolved Hide resolved

test(e2e): avoid tagging twice in ZOT registry

8b7d60e

Signed-off-by: Billy Zha <jinzha1@microsoft.com>

njucjc force-pushed the oras-tag-support-distribution-spec-flag branch from 46b1785 to 8b7d60e Compare July 1, 2024 05:49

qweeah approved these changes Jul 1, 2024

View reviewed changes

qweeah added 3 commits July 1, 2024 08:36

Merge branch 'oras-tag-support-distribution-spec-flag' of https://git…

cd70c5c

…hub.com/njucjc/oras into pr/njucjc/1435

chore: ignore always-nil returned error to improve coverage

e8e4ec5

Signed-off-by: Billy Zha <jinzha1@microsoft.com>

chore: rewording

8bd4797

Signed-off-by: Billy Zha <jinzha1@microsoft.com>

shizhMSFT approved these changes Jul 1, 2024

View reviewed changes

qweeah merged commit ecb7878 into oras-project:main Jul 1, 2024
8 checks passed

FeynmanZhou linked an issue Jul 2, 2024 that may be closed by this pull request

oras tag should not create referrers tags #1436

Closed

qweeah mentioned this pull request Dec 9, 2024

bump: tag and release ORAS CLI v1.3.0-beta.1 #1577

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix: disable oras tag fallback to tag schema when tagging a referrer #1435

fix: disable oras tag fallback to tag schema when tagging a referrer #1435

njucjc commented Jun 28, 2024 •

edited

Loading

qweeah commented Jun 28, 2024

njucjc commented Jun 28, 2024 •

edited

Loading

qweeah commented Jun 28, 2024

njucjc commented Jun 28, 2024 •

edited

Loading

codecov bot commented Jun 28, 2024 •

edited

Loading

qweeah commented Jun 28, 2024

shizhMSFT commented Jun 28, 2024

njucjc commented Jun 28, 2024

njucjc commented Jun 28, 2024

qweeah commented Jun 29, 2024

qweeah commented Jun 29, 2024

njucjc commented Jul 1, 2024

qweeah left a comment

qweeah left a comment

qweeah left a comment

shizhMSFT left a comment

fix: disable oras tag fallback to tag schema when tagging a referrer #1435

fix: disable oras tag fallback to tag schema when tagging a referrer #1435

Conversation

njucjc commented Jun 28, 2024 • edited Loading

qweeah commented Jun 28, 2024

njucjc commented Jun 28, 2024 • edited Loading

qweeah commented Jun 28, 2024

njucjc commented Jun 28, 2024 • edited Loading

codecov bot commented Jun 28, 2024 • edited Loading

Codecov Report

qweeah commented Jun 28, 2024

shizhMSFT commented Jun 28, 2024

njucjc commented Jun 28, 2024

njucjc commented Jun 28, 2024

qweeah commented Jun 29, 2024

qweeah commented Jun 29, 2024

njucjc commented Jul 1, 2024

qweeah left a comment

Choose a reason for hiding this comment

qweeah left a comment

Choose a reason for hiding this comment

qweeah left a comment

Choose a reason for hiding this comment

shizhMSFT left a comment

Choose a reason for hiding this comment

njucjc commented Jun 28, 2024 •

edited

Loading

njucjc commented Jun 28, 2024 •

edited

Loading

njucjc commented Jun 28, 2024 •

edited

Loading

codecov bot commented Jun 28, 2024 •

edited

Loading