-
Notifications
You must be signed in to change notification settings - Fork 26
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
🐛 check the underlying storage for existing cluster catalog content #290
🐛 check the underlying storage for existing cluster catalog content #290
Conversation
896eed7
to
0f6d857
Compare
pkg/storage/localdir.go
Outdated
@@ -60,6 +61,13 @@ func (s LocalDir) StorageServerHandler() http.Handler { | |||
return mux | |||
} | |||
|
|||
func (s LocalDir) ContentExists(catalog string) bool { | |||
if _, err := os.Stat(filepath.Join(s.RootDir, catalog)); errors.Is(err, os.ErrNotExist) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If there is some error other than os.ErrNotExist
returned from Stat
, I think we need to return it to the caller, which would likely want to attempt to delete whatever is there and unpack again.
Also, I think we should be explicitly checking for an all.json
entry and that it is a file.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Re: return to the caller I'm not sure if we should because it might lead to some ugly code like this inside the needsUnpacking(catalog)
func -
exists, err := r.Storage.ContentExists(catalog.Name)
if err != nil {
// content is corrupted. delete and unpack again
r.Storage.Delete(catalog.Name)
return true
}
IMO the Unpacker itself should be clearing out the path in case it's not empty, and it does - with Cleanup()
.
The needsUnpacking(catalog)
func should only indicate if the action should take place, not do any actions on its own.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'll add the all.json file check shortly.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
PTAL
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #290 +/- ##
==========================================
+ Coverage 33.12% 33.53% +0.40%
==========================================
Files 15 15
Lines 646 656 +10
==========================================
+ Hits 214 220 +6
- Misses 411 413 +2
- Partials 21 23 +2 ☔ View full report in Codecov by Sentry. |
@everettraven, an aside: I see a failure in go-apidiff. I wouldn't expect this code to be in our public API. In preparation for a v1.0, should we take a moment to make sure that the only code in our public API are the catalogd types? |
@joelanford i think that is reasonable. I imagine that should fall under the v1.0 epic to evaluate catalogd's public API surface? |
0f6d857
to
9e2611c
Compare
pkg/storage/localdir.go
Outdated
@@ -60,6 +60,18 @@ func (s LocalDir) StorageServerHandler() http.Handler { | |||
return mux | |||
} | |||
|
|||
func (s LocalDir) ContentExists(catalog string) bool { | |||
file, err := os.Stat(s.ContentURL(catalog)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Does s.ContentURL
actually work as expected here? IIUC that would return something like http://<url>/<catalog.Name>/all.json
which I would expect to be an invalid filepath due to it having the http://
or https://
scheme present.
I think we may need to instead do something along the lines of filepath.Join(s.RootDir, catalog, "all.json")
Related, could we add some unit tests for this function to ensure it is functioning as expected?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You're right. I got confused because saw a place where the extension is being stripped after calling.
Signed-off-by: Igor Troyanovsky <itroyano@redhat.com>
9e2611c
to
70d91c0
Compare
91e899f
Fixes #288