Paginate responses for gcrane

[jonjohnsonjr]

If we don't see the GCR-specific response, just continue paging
through results so we can return a reasonable answer for non-Google
registries.

Fixes #984

$ gcrane ls -v quay.io/prometheus/prometheus | wc -l
2021/04/19 17:34:51 No matching credentials were found for "quay.io", falling back on anonymous
2021/04/19 17:34:51 --> GET https://quay.io/v2/
2021/04/19 17:34:51 GET /v2/ HTTP/1.1
Host: quay.io
User-Agent: Go-http-client/1.1
Accept-Encoding: gzip


2021/04/19 17:34:51 <-- 401 https://quay.io/v2/ (352.624355ms)
2021/04/19 17:34:51 HTTP/2.0 401 Unauthorized
Content-Length: 4
Content-Type: text/html; charset=utf-8
Date: Tue, 20 Apr 2021 00:34:51 GMT
Docker-Distribution-Api-Version: registry/2.0
Server: nginx/1.12.1
Www-Authenticate: Bearer realm="https://quay.io/v2/auth",service="quay.io"

true
2021/04/19 17:34:51 --> GET https://quay.io/v2/auth?scope=repository%3Aprometheus%2Fprometheus%3Apull&service=quay.io [body redacted: basic token response contains credentials]
2021/04/19 17:34:51 GET /v2/auth?scope=repository%3Aprometheus%2Fprometheus%3Apull&service=quay.io HTTP/1.1
Host: quay.io
User-Agent: go-containerregistry/(devel)
Accept-Encoding: gzip


2021/04/19 17:34:51 <-- 200 https://quay.io/v2/auth?scope=repository%3Aprometheus%2Fprometheus%3Apull&service=quay.io (100.662959ms) [body redacted: basic token response contains credentials]
2021/04/19 17:34:51 HTTP/2.0 200 OK
Content-Length: 877
Cache-Control: no-cache, no-store, must-revalidate
Content-Type: application/json
Date: Tue, 20 Apr 2021 00:34:51 GMT
Server: nginx/1.12.1
Strict-Transport-Security: max-age=63072000; preload
X-Frame-Options: DENY


2021/04/19 17:34:51 --> GET https://quay.io/v2/prometheus/prometheus/tags/list?n=1000
2021/04/19 17:34:51 GET /v2/prometheus/prometheus/tags/list?n=1000 HTTP/1.1
Host: quay.io
User-Agent: go-containerregistry/(devel)
Authorization: <redacted>
Accept-Encoding: gzip


2021/04/19 17:34:51 <-- 200 https://quay.io/v2/prometheus/prometheus/tags/list?n=1000 (94.928814ms)
2021/04/19 17:34:51 HTTP/2.0 200 OK
Content-Length: 598
Content-Type: application/json
Date: Tue, 20 Apr 2021 00:34:51 GMT
Link: </v2/prometheus/prometheus/tags/list?next_page=gAAAAABgfiGraV3DWDarUWy71JWT5kI95UudnctN04hLMA_k686zJ_cUIYiKV0NnmwOSQ9E1YvzfjE0CixNN6noSDsQIs0AQQ6nm747LlSsI6WdR4mpUqDo%3D&n=50>; rel="next"
Server: nginx/1.12.1
Strict-Transport-Security: max-age=63072000; preload
X-Frame-Options: DENY

{"name":"prometheus/prometheus","tags":["v2.5.0","v2.6.0-rc.0","v2.6.0-rc.1","v2.6.0","v2.6.1","v2.7.0-rc.0","v2.7.0-rc.1","v2.7.0-rc.2","v2.7.0","v2.7.1","v2.0.0-rc.2","v1.3.0-beta.0","v2.0.0","v2.0.0-alpha.2","v1.8.1","v1.1.1","v1.2.1","v1.0.0-rc.0","v1.5.1","v2.0.0-rc.0","v2.0.0-beta.2","v1.7.2","v2.0.0-rc.3","0.20.0","0.19.2","v2.1.0","v1.2.0","v2.0.0-beta.1","v1.0.2","v2.2.1","v2.0.0-rc.1","v2.2.0","0.19.3","v1.8.2","v2.2.0-rc.0","v2.0.0-beta.4","v1.1.0","v1.8.0","0.19.0","v1.6.1","v1.4.0","v1.0.1","v2.0.0-alpha.1","v2.4.1","v1.5.0","v1.0.0","v1.1.2","v1.6.3","v2.0.0-beta.5","v1.5.2"]}

2021/04/19 17:34:51 saw non-google tag listing response, falling back to pagination
2021/04/19 17:34:51 --> GET https://quay.io/v2/prometheus/prometheus/tags/list?next_page=gAAAAABgfiGraV3DWDarUWy71JWT5kI95UudnctN04hLMA_k686zJ_cUIYiKV0NnmwOSQ9E1YvzfjE0CixNN6noSDsQIs0AQQ6nm747LlSsI6WdR4mpUqDo%3D&n=50
2021/04/19 17:34:51 GET /v2/prometheus/prometheus/tags/list?next_page=gAAAAABgfiGraV3DWDarUWy71JWT5kI95UudnctN04hLMA_k686zJ_cUIYiKV0NnmwOSQ9E1YvzfjE0CixNN6noSDsQIs0AQQ6nm747LlSsI6WdR4mpUqDo%3D&n=50 HTTP/1.1
Host: quay.io
User-Agent: go-containerregistry/(devel)
Authorization: <redacted>
Accept-Encoding: gzip


2021/04/19 17:34:51 <-- 200 https://quay.io/v2/prometheus/prometheus/tags/list?next_page=gAAAAABgfiGraV3DWDarUWy71JWT5kI95UudnctN04hLMA_k686zJ_cUIYiKV0NnmwOSQ9E1YvzfjE0CixNN6noSDsQIs0AQQ6nm747LlSsI6WdR4mpUqDo%3D&n=50 (87.768131ms)
2021/04/19 17:34:51 HTTP/2.0 200 OK
Content-Length: 602
Content-Type: application/json
Date: Tue, 20 Apr 2021 00:34:51 GMT
Link: </v2/prometheus/prometheus/tags/list?next_page=gAAAAABgfiGrpdV49dMVR-yUO8f4TaC-jwl3N-IsbApoKDwRFgq4iaHHQcqReRu704hg7Ni-ZgppKvD-DKgVEdtGC1kHkL0iY3huODMxnr7AybxQ4ho4lG4%3D&n=50>; rel="next"
Server: nginx/1.12.1
Strict-Transport-Security: max-age=63072000; preload
X-Frame-Options: DENY

{"name":"prometheus/prometheus","tags":["v1.2.2","v1.6.2","v2.3.2","v2.2.0-rc.1","v1.3.0","v2.4.2","v2.4.0","v2.3.0","v1.1.3","v2.4.0-rc.0","v2.0.0-alpha.3","v2.0.0-alpha.0","v1.7.0","v1.5.3","v1.7.1","v1.6.0","v2.4.3","v1.3.1","v1.4.1","0.19.1","v2.0.0-beta.3","v2.3.1","v2.5.0-rc.0","v2.5.0-rc.1","v2.0.0-beta.0","v1.2.3","v2.5.0-rc.2","v2.7.2","v2.8.0-rc.0","v2.8.0","v2.8.1","v2.9.0-rc.0","v2.9.0","v2.9.1","v2.9.2","v2.10.0-rc.0","v2.10.0","v2.11.0-rc.0","v2.11.0","v2.11.1","v2.11.2","v2.12.0-rc.0","v2.12.0","v2.13.0-rc.0","v2.13.0","v2.13.1","v2.14.0-rc.0","v2.14.0","v2.15.0-rc.0","v2.15.0"]}

2021/04/19 17:34:51 saw non-google tag listing response, falling back to pagination
2021/04/19 17:34:51 --> GET https://quay.io/v2/prometheus/prometheus/tags/list?next_page=gAAAAABgfiGrpdV49dMVR-yUO8f4TaC-jwl3N-IsbApoKDwRFgq4iaHHQcqReRu704hg7Ni-ZgppKvD-DKgVEdtGC1kHkL0iY3huODMxnr7AybxQ4ho4lG4%3D&n=50
2021/04/19 17:34:51 GET /v2/prometheus/prometheus/tags/list?next_page=gAAAAABgfiGrpdV49dMVR-yUO8f4TaC-jwl3N-IsbApoKDwRFgq4iaHHQcqReRu704hg7Ni-ZgppKvD-DKgVEdtGC1kHkL0iY3huODMxnr7AybxQ4ho4lG4%3D&n=50 HTTP/1.1
Host: quay.io
User-Agent: go-containerregistry/(devel)
Authorization: <redacted>
Accept-Encoding: gzip


2021/04/19 17:34:52 <-- 200 https://quay.io/v2/prometheus/prometheus/tags/list?next_page=gAAAAABgfiGrpdV49dMVR-yUO8f4TaC-jwl3N-IsbApoKDwRFgq4iaHHQcqReRu704hg7Ni-ZgppKvD-DKgVEdtGC1kHkL0iY3huODMxnr7AybxQ4ho4lG4%3D&n=50 (89.701974ms)
2021/04/19 17:34:52 HTTP/2.0 200 OK
Content-Length: 597
Content-Type: application/json
Date: Tue, 20 Apr 2021 00:34:51 GMT
Server: nginx/1.12.1
Strict-Transport-Security: max-age=63072000; preload
X-Frame-Options: DENY

{"name":"prometheus/prometheus","tags":["v2.15.1","v2.15.2","v2.16.0-rc.0","v2.16.0-rc.1","v2.16.0","v2.17.0-rc.0","v2.17.0-rc.1","v2.17.0-rc.3","v2.17.0-rc.4","v2.17.0","v2.17.1","v2.17.2","v2.18.0-rc.0","v2.18.0-rc.1","v2.18.0","v2.18.1","v2.19.0-rc.0","v2.18.2","v2.19.0","v2.19.1","v2.19.2","v2.20.0-rc.0","v2.20.0-rc.1","v2.20.0","v2.19.3","v2.20.1","v2.21.0-rc.0","v2.21.0-rc.1","v2.21.0","v2.22.0-rc.0","v2.22.0","v2.22.1","v2.22.2","v2.23.0-rc.0","v2.23.0","v2.24.0-rc.0","v2.24.0","v2.24.1","v2.25.0-rc.0","v2.25.0","master","v2.25.1","v2.25.2","v2.26.0-rc.0","v2.26.0","latest","main"]}

2021/04/19 17:34:52 saw non-google tag listing response, falling back to pagination
147

[codecov-commenter]

Codecov Report

Merging #985 (da4f3f0) into main (23990da) will increase coverage by 0.12%.
The diff coverage is 73.91%.

@@            Coverage Diff             @@
##             main     #985      +/-   ##
==========================================
+ Coverage   74.61%   74.74%   +0.12%     
==========================================
  Files         107      107              
  Lines        5027     5060      +33     
==========================================
+ Hits         3751     3782      +31     
- Misses        721      723       +2     
  Partials      555      555              

Impacted Files	Coverage Δ
pkg/v1/google/list.go	71.31% <73.91%> (+8.39%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 23990da...da4f3f0. Read the comment docs.

[imjasonh]

What's the use case for using gcrane against non-GCR? Can we just fail if the requested registry is not GCR?

[jonjohnsonjr]

What's the use case

This was also my initial reaction. See the attached bug.

[jonjohnsonjr]

I'm tempted to merge this regardless because of this existing behavior:

go-containerregistry/cmd/gcrane/cmd/list.go

Lines 66 to 72 in 23990da

	if len(tags.Manifests) == 0 && len(tags.Children) == 0 {
	// If we didn't see any GCR extensions, just list the tags like normal.
	for _, tag := range tags.Tags {
	fmt.Printf("%s:%s\n", repo, tag)
	}
	return
	}