Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

remove dependencies on resumable #2707

Merged
merged 6 commits into from
Sep 20, 2018
Merged

Conversation

davidswu
Copy link
Contributor

closes #2697

@GordonTheTurtle
Copy link

Please sign your commits following these rules:
https://github.com/moby/moby/blob/master/CONTRIBUTING.md#sign-your-work
The easiest way to do this is to amend the last commit:

$ git clone -b "go-1.11" git@github.com:davidswu/distribution.git somewhere
$ cd somewhere
$ git rebase -i HEAD~842354293216
editor opens
change each 'pick' to 'edit'
save the file and quit
$ git commit --amend -s --no-edit
$ git rebase --continue # and repeat the amend for each commit
$ git push -f

Amending updates the existing PR. You DO NOT need to open a new one.

David Wu added 2 commits September 11, 2018 08:21
Signed-off-by: David Wu <david.wu@docker.com>
Signed-off-by: David Wu <david.wu@docker.com>
@davidswu davidswu changed the title [WIP] remove dependencies on resumable remove dependencies on resumable Sep 11, 2018
David Wu added 2 commits September 11, 2018 16:14
Signed-off-by: David Wu <david.wu@docker.com>
Signed-off-by: David Wu <david.wu@docker.com>
@codecov
Copy link

codecov bot commented Sep 11, 2018

Codecov Report

Merging #2707 into master will increase coverage by 0.21%.
The diff coverage is 77.77%.

Impacted file tree graph

@@            Coverage Diff            @@
##           master   #2707      +/-   ##
=========================================
+ Coverage   60.18%   60.4%   +0.21%     
=========================================
  Files         103     103              
  Lines        8015    8003      -12     
=========================================
+ Hits         4824    4834      +10     
+ Misses       2548    2527      -21     
+ Partials      643     642       -1
Flag Coverage Δ
#linux 60.4% <77.77%> (+0.21%) ⬆️
Impacted Files Coverage Δ
context/http.go 64.18% <ø> (+5.18%) ⬆️
registry/handlers/helpers.go 46.15% <100%> (-0.52%) ⬇️
registry/storage/linkedblobstore.go 69.95% <100%> (ø) ⬆️
registry/storage/blobwriter.go 42.37% <66.66%> (+6.53%) ⬆️
registry/storage/blobwriter_resumable.go 52.17% <66.66%> (-2.24%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a5c2fdc...a927fbd. Read the comment docs.

Signed-off-by: David Wu <david.wu@docker.com>
@dmcgowan
Copy link
Collaborator

This is great, glad we can use the upstream hash libraries again!

Can you point to the 1.11 change related to http.CloseNotifier? Curious what changed there

@davidswu
Copy link
Contributor Author

@dmcgowan staticchecks was failing the build about it since it is deprecated
Since we can just switch to Request.Context() I opted to clean it up rather than remove the static checks from gometalinter. I can revert the change and remove the checks instead if you think it's too disruptive

Copy link
Contributor

@caervs caervs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice start! But I have some questions and comments so far.

} else {
storedState, err := bw.driver.GetContent(ctx, hashStateMatch.path)
if err != nil {
return err
}

if err = h.Restore(storedState); err != nil {
// This type assertion is safe since we already did an assertion at the beginning
if err = h.(encoding.BinaryUnmarshaler).UnmarshalBinary(storedState); err != nil {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the type assertion necessary? h is already asserted to be a encoded.BinaryMarsheler, right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a encoding.BinaryUnmarshaler

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh derp 😛

return err
}

state, err = h.(encoding.BinaryMarshaler).MarshalBinary()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same as above

@@ -104,18 +104,6 @@ func GetRequestID(ctx context.Context) string {
// WithResponseWriter returns a new context and response writer that makes
// interesting response statistics available within the context.
func WithResponseWriter(ctx context.Context, w http.ResponseWriter) (context.Context, http.ResponseWriter) {
if closeNotifier, ok := w.(http.CloseNotifier); ok {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you provide a little more context in the commit message as to why the notifier is being removed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

staticchecks was failing since http.CloseNotifier is deprecated. So we either remove the notifier and rely on Request.Context instead or remove the checks. I opted for the former

offset := bw.fileWriter.Size()
if offset == h.Len() {
if offset == int64(len(state)) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this is right. In a resumable hash then Len() is the number of bytes that have been written to the hash so far. I believe this is just giving you the len of the digest itself.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No this is the length of the state not the length of the digest.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is wrong though since the state length is not the offset we are looking for. We may need to implement a digester to track the offset written to the hash

return err
}
}

// Mind the gap.
if gapLen := offset - h.Len(); gapLen > 0 {
if gapLen := offset - int64(len(state)); gapLen > 0 {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above

Signed-off-by: David Wu <david.wu@docker.com>
Copy link
Collaborator

@dmcgowan dmcgowan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

@caervs caervs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

@dmp42 dmp42 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dmp42 dmp42 merged commit 16128bb into distribution:master Sep 20, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Building private registry with go1.11 fails
5 participants