Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

add --blobs-to-disk to 'crane registry serve' #1731

Merged
merged 5 commits into from
Jun 15, 2023
Merged

add --blobs-to-disk to 'crane registry serve' #1731

merged 5 commits into from
Jun 15, 2023

Conversation

imjasonh
Copy link
Collaborator

@imjasonh imjasonh commented Jun 15, 2023 •
edited
Loading

This only writes blobs to disk, and keeps manifests in memory. This is mostly a short-term bandaid to be able to store more/larger images while we work on a storage backend that writes to an OCI layout.

$ PORT=8080 go run ./cmd/crane registry serve --blobs-to-disk
2023/06/15 09:28:36 storing blobs in /var/folders/gw/86t1jg2109s5j8vzszj281kc0000gn/T/
2023/06/15 09:28:36 serving on port 8080
...

...do some pushes...

$ ls /var/folders/gw/86t1jg2109s5j8vzszj281kc0000gn/T/sha256:*
/var/folders/gw/86t1jg2109s5j8vzszj281kc0000gn/T/sha256:0d7529a407858eb4f76aa7087887a3d0e854a52dde475938080ae6c43297a929
/var/folders/gw/86t1jg2109s5j8vzszj281kc0000gn/T/sha256:31e352740f534f9ad170f75378a84fe453d6156e40700b882d737a8f4a6988a3
/var/folders/gw/86t1jg2109s5j8vzszj281kc0000gn/T/sha256:35d589f20301ca0a99f301c3c92567d50fb0aea3a95294b630f8ae13e49b98d1
/var/folders/gw/86t1jg2109s5j8vzszj281kc0000gn/T/sha256:5053b247d78b5e43b5543fec77c856ce70b8dc705d9f38336fa77736f25ff47c

@imjasonh
add --blobs-to-disk to 'crane registry serve'
7dab7ad 
Signed-off-by: Jason Hall <jason@chainguard.dev>
github-advanced-security[bot]
github-advanced-security bot found potential problems Jun 15, 2023
View reviewed changes
pkg/registry/blobs_disk.go
m.lock.Lock()
defer m.lock.Unlock()

fi, err := os.Stat(filepath.Join(m.dir, h.String()))

Check failure

Code scanning / CodeQL

Uncontrolled data used in path expression High

This path depends on a
user-provided value
Loading
.
pkg/registry/blobs_disk.go
m.lock.Lock()
defer m.lock.Unlock()

return os.Open(filepath.Join(m.dir, h.String()))

Check failure

Code scanning / CodeQL

Uncontrolled data used in path expression High

This path depends on a
user-provided value
Loading
.
pkg/registry/blobs_disk.go
m.lock.Lock()
defer m.lock.Unlock()

f, err := os.Create(filepath.Join(m.dir, h.String()))

Check failure

Code scanning / CodeQL

Uncontrolled data used in path expression High

This path depends on a
user-provided value
Loading
.
pkg/registry/blobs_disk.go
m.lock.Lock()
defer m.lock.Unlock()

return os.Remove(filepath.Join(m.dir, h.String()))

Check failure

Code scanning / CodeQL

Uncontrolled data used in path expression High

This path depends on a
user-provided value
Loading
.
mattmoor
mattmoor approved these changes Jun 15, 2023
View reviewed changes
Copy link
Collaborator

@mattmoor mattmoor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add test coverage for this?

@imjasonh
add tests
ff93b1e 
Signed-off-by: Jason Hall <jason@chainguard.dev>
@imjasonh
mark flag hidden
31c7f8c 
Signed-off-by: Jason Hall <jason@chainguard.dev>
@imjasonh
tasty boilerplate
0bd43d1 
Signed-off-by: Jason Hall <jason@chainguard.dev>
@imjasonh
boilerplate mmm
a42c0f1 
Signed-off-by: Jason Hall <jason@chainguard.dev>
@codecov-commenter
Copy link

codecov-commenter commented Jun 15, 2023 •
edited
Loading

Codecov Report

Merging #1731 (a42c0f1) into main (4e4b03a) will decrease coverage by 0.14%.
The diff coverage is 54.34%.

@@            Coverage Diff             @@
##             main    #1731      +/-   ##
==========================================
- Coverage   72.02%   71.88%   -0.14%     
==========================================
  Files         121      122       +1     
  Lines        9777     9818      +41     
==========================================
+ Hits         7042     7058      +16     
- Misses       2060     2080      +20     
- Partials      675      680       +5
Impacted Files Coverage Δ
pkg/registry/blobs_disk.go 44.44% <44.44%> (ø)
pkg/registry/blobs.go 61.74% <83.33%> (-0.19%) ⬇️
pkg/registry/registry.go 84.21% <100.00%> (+0.87%) ⬆️

... and 1 file with indirect coverage changes

@imjasonh imjasonh merged commit 03ad2ac into google:main Jun 15, 2023
17 of 18 checks passed
@jonjohnsonjr jonjohnsonjr mentioned this pull request Jul 27, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jonjohnsonjr jonjohnsonjr jonjohnsonjr left review comments

@mattmoor mattmoor mattmoor approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@imjasonh @codecov-commenter @mattmoor @jonjohnsonjr