-
Notifications
You must be signed in to change notification settings - Fork 45
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Bundle of changes #112
Bundle of changes #112
Conversation
in one of the PRs above you said you wanted the PR bundled and the changes to go-ipfs ready. This is now done! I'm pretty pleased with the result, especially since it removes some odd stuff like that function The changes to go-ipfs are in ipfs/kubo#5035. |
cli/responseemitter.go
Outdated
@@ -70,26 +74,52 @@ func (re *responseEmitter) SetEncoder(enc func(io.Writer) cmds.Encoder) { | |||
re.enc = enc(re.stdout) | |||
} | |||
|
|||
func (re *responseEmitter) SetError(v interface{}, errType cmdkit.ErrorType) { | |||
func (re *responseEmitter) CloseWithError(err error) error { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It would be really great if this method didnt return an error. I can't think of any situation where the caller wants to do something with the error, or even can. I think it's likely better to just log an error and move on.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The reason I let made this return an error is that CloseWithError(error) error
is idiomatic and is used throughout the entire Go ecosystem. I don't see a reason to diverge - maybe we don't really need it in ipfs, but possible somewhere else. And now that the command's Run
function just returns an error we don't call this so often that it's too much of a pain for the caller to do the logging themselves.
Hey @whyrusleeping judging by Stebalien's contribution activity graph it seems they are currently not avalailable, so could you maybe review this PR or delegate this? Thanks! |
Also if you have questions on the structure of this bundle PR, shoot! |
@keks yeah, he's just now getting back from a vacation. I'll still want his review, but i'll take a pass here :) |
chan.go
Outdated
r.err = val | ||
return nil, ErrRcvdError | ||
// TODO keks remove logging | ||
log.Error("unexpected error value:", val) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
rm ?
chan.go
Outdated
|
||
func (re *chanResponseEmitter) Emit(v interface{}) error { | ||
if e, ok := v.(*cmdkit.Error); ok { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this is because we shouldnt be emitting errors ever anymore, right?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Exactly. Should I remove this or are we not confident enough this doesn't happen? Or should I return that as an error, so it will definitely fail?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i think we should keep it, and maybe even add a check for the non-pointerized version too just in case. Then also add a comment explaining why this bit of code is here.
Hey @Stebalien, do you think you'll find some time to CR this PR and the one in go-ipfs? Would love to merge this soon |
chan.go
Outdated
@@ -70,6 +82,20 @@ func (r *chanResponse) Length() uint64 { | |||
return r.length | |||
} | |||
|
|||
func (re *chanResponse) Head() Head { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Whats this method supposed to do?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's not new, I just moved it from here: https://github.com/ipfs/go-ipfs-cmds/pull/112/files/15c1508126b5934ffae976e4a09c4353fa90956e#diff-d1319de8df19ba21c67a62fa1fde5f89L201
But I think it's needed for HEAD requests, it's even in the ResponseEmitter interface; but it looks like it's not called from anywhere in go-ipfs-cmds/...
, so maybe we can drop it. I'd suggest doing that in a new PR though.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I decided to do it in this PR.
http/response.go
Outdated
@@ -73,7 +76,24 @@ func (res *Response) RawNext() (interface{}, error) { | |||
err = io.EOF | |||
} | |||
|
|||
return m.Get(), err | |||
if err != nil { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
maybe put this inside the same if err != nil
as above? might make it a bit clearer as to whats going on here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
First pass.
@@ -57,7 +61,15 @@ func (r *chanResponse) Error() *cmdkit.Error { | |||
return nil |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we get rid of both of these nil tests? I'd rathe panic if something is wrong (and we'll panic in the second case anyways).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
well the r == nil
check obviously doesn't help because we already access r.wait
before. But what's wrong with the r.err == nil
check?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We call r.err.Error()
below, so r.err
better not be nil.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry, I was refering to both of the r == nil
checks (in this function and the one above). Unfortunately, git hides that part of the patch.
chan.go
Outdated
func (re *chanResponse) Head() Head { | ||
<-re.wait | ||
|
||
err, ok := re.err.(*cmdkit.Error) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This will treat EOF as an error. Maybe we should just have a separate closed flag if we need that and not write EOF to the error field.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actually, we can just check <-re.closed
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You are right that this is not correct. However, there can be an error even if re.closed
already is closed, so I don't think it's a good indicator. I think I fixed this locally and will push the changes once I'm through you first pass.
chan.go
Outdated
case v, ok := <-r.ch: | ||
if !ok { | ||
r.ch = nil |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Are you sure this isn't racy? Also, this'll cause this method to hang forever if called again.
chan.go
Outdated
return nil, io.EOF | ||
} | ||
|
||
if err, ok := v.(cmdkit.Error); ok { | ||
if err, ok := v.(*cmdkit.Error); ok { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm pretty sure we just want to remove this entirely.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We're already checking if something is an error before putting it into the channel. If we leave this here, we'll need to leve the cmdkit.Error
case in the switch statement below. However, IMO, we should just remove both (there shouldn't be any errors here).
chan.go
Outdated
} | ||
if e, ok := v.(*cmdkit.Error); ok { | ||
log.Errorf("unexpected error value emitted: %s at\n%s", e.Error(), debug.Stack()) | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Shouldn't this call CloseWithError
and return?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nah, we don't emit errors anymore. Doing so is a programmer error, so we want them to know. But panicking because of this feels too strong, so I opted for logging an error.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That won't cause a panic, it'll just "do the right thing" (redirect the error). As it stands, we'll end up sending the error on the value channel.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh yes, I meant to return. Anyway, I change it to panic now. I hope we can then in the near future remove those panics because we are more confident we don't do that mistake anymore.
chan.go
Outdated
}) | ||
|
||
re.l.Lock() | ||
defer re.l.Unlock() | ||
|
||
if re.ch == nil { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think this is a valid "closed" check anymore. Probably want to check:
select {
case <-re.closed:
return ...
default:
}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You're right it's not a valid close check anymore, but I think your proposal is also racy: It might occur that re.closed is not yet closed but re.ch is, which would result in a write to closed channel error. Thinking this through...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It should be if we only close the channel while holding the lock.
This is actually a good case for removing the closeOnce
sync.Once
(a user may think that closeOnce
is protecting the close while really the lock is).
cli/responseemitter.go
Outdated
|
||
return nil | ||
|
||
log.Errorf("got an error: %v\n%s", err, debug.Stack()) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Again, we should probably forward these errors to CloseWithError
for now.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I believe we should do away with emitting errors and make sure doing so is an error.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should. However, we should either panic in cases like this or "do the right thing". Simply logging and dropping the error on the floor is not going to end well.
case io.Reader: | ||
_, err = io.Copy(re.stdout, t) | ||
if err != nil { | ||
re.SetError(err, cmdkit.ErrNormal) | ||
err = nil | ||
return err |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm wondering if this should also call CloseWithError
. We could also say that it's the callers responsibility to return the error from Run
. Thoughts?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Calling CloseWithError
here would close the emitter and only print io.Copy
's error. I think it's better to allow the caller to specify a more intelligible error.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fair enough.
cli/responseemitter_test.go
Outdated
//"io" | ||
"testing" | ||
|
||
"github.com/ipfs/go-ipfs-cmdkit" | ||
//"github.com/ipfs/go-ipfs-cmdkit" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should probably remove before final merge.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yup.
command.go
Outdated
defer re.Close() | ||
var err error | ||
|
||
defer func() { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How about putting all the logic in Call
in a separate call
function that turns an error. That way, we don't need this defer? The shared err
variable makes be uncomfortable.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Second round.
executor.go
Outdated
return nil | ||
err = cmd.Run(req, re, env) | ||
log.Debugf("cmds.Execute: Run returned %q with response emitter of type %T", err, re) | ||
if err == nil { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can just call CloseWithError
(with a nil error).
|
||
re, res = NewChanResponsePair(req) | ||
|
||
go func() { |
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
http/client.go
Outdated
return re.CloseWithError(err) | ||
} | ||
|
||
return re.Close() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Again, can probably just call CloseWithError
.
http/response.go
Outdated
e := res.err | ||
res.err = nil | ||
return e | ||
if res.err == io.EOF { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Impossible condition.
http/response.go
Outdated
} | ||
|
||
return m.Get(), err | ||
return v, err | ||
} | ||
|
||
func (res *Response) Next() (interface{}, error) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
THis function checks if the value is an error, I assume this isn't possible given RawNext?
http/responseemitter.go
Outdated
e = &cmdkit.Error{Message: err.Error()} | ||
} | ||
|
||
if re.streaming || e.Code == cmdkit.ErrFatal { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Aren't all errors "fatal" now? That is, can't we just always use the trailer now?
writer.go
Outdated
case Single: | ||
return val.Value, nil | ||
case *cmdkit.Error: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Again, I assume this case is impossible.
Nice! This is starting to look a lot more robust. |
@Stebalien thank you for the thorough feedback! I tried to incorporate all of it and made some additional changes:
Right now Travis complains that Also, in my branch of go-ipfs that has the changes for this library there is a test (this one) that doesn't complete currently. Investigating that now. Also my branch fell behind a bit, so I'll still need to rebase that on master -- but I first want to get that test to pass. |
Okay, I fixed all the tests in the go-ipfs branch and rebased it is it's en par with master. I still have no clue with Travis let's this PR fail though. It thinks that writer.go is not gofmt'd, even though it is -- at least for me calling go fmt doesn't change anything. Rebuilding doesn't change anything. |
cc @Stebalien |
chan.go
Outdated
defer re.wl.Unlock() | ||
|
||
if _, ok := v.(Single); ok { | ||
defer re.closeWithError(nil) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This'll close without error even if we fail somewhere down the line. It can also panic if we emit a single value after closing.
chan.go
Outdated
|
||
func (r *chanResponse) Request() *Request { | ||
return r.req | ||
} | ||
|
||
func (r *chanResponse) Error() *cmdkit.Error { | ||
<-r.wait |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It seems kind of funny for this method to wait for at least one value to be emitted. I'd expect it to either wait until the end or always return immediately (preferably the latter to reduce the chance of a deadlock).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I understand the concern about deadlocks, but returning immediately raises the question of what to return when the command is still running and we didn't see an error yet. So I went for wait-until-close, I'm currently testing this in go-ipfs to see if it makes problems.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Okay, if we block on error until the stream is closed and change nothing else, we get a deadlock. Now I just return nil if the connection is not closed yet.
chan.go
Outdated
return r.req | ||
} | ||
|
||
func (r *chanResponse) Error() *cmdkit.Error { | ||
<-r.wait | ||
|
||
if r == nil { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is racy as we set err
under a lock (and can set it at any time).
cli/responseemitter.go
Outdated
Err: re.err, | ||
} | ||
} | ||
|
||
func (re *responseEmitter) Emit(v interface{}) error { | ||
// unwrap | ||
if val, ok := v.(cmds.Single); ok { | ||
v = val.Value |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we not need to close in this case?
executor.go
Outdated
|
||
err := cmd.PostRun[typer.Type()](res, lower) | ||
closeErr = lower.CloseWithError(err) | ||
errCh <- closeErr |
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
// if they are errors | ||
if e, ok := v.(error); ok { | ||
if err, ok := v.(error); ok { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is throwing the error away, not returning it. It's creating a new error variable.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I know it's creating a new error variable, and I use it as the argument when calling CloseWithError
. I don't think I'm throwing it away?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah. Got it. I thought you were trying to overwrite the named return value err
.
executor.go
Outdated
@@ -105,6 +132,12 @@ func (x *executor) Execute(req *Request, re ResponseEmitter, env Environment) (e | |||
} | |||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So, Run
panics, we won't wait for PostRun
to exit. Not much of an issue but we may want to consider waiting on errCh
in here instead.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd still want to return the error from errCh
, but yes the defer should also wait for it. So let's wait in both places.
http/responseemitter.go
Outdated
ch, isChan = value.(chan interface{}) | ||
if single, ok := value.(cmds.Single); ok { | ||
value = single.Value | ||
defer re.closeWithError(nil) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Probably needs to go after the lock.
http/responseemitter.go
Outdated
if _, ok := value.(cmdkit.Error); ok { | ||
log.Warning("fixme: got Error not *Error: ", value) | ||
value = &value | ||
if err, ok := value.(cmdkit.Error); ok { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Use the debug error assertion?
writer.go
Outdated
return errors.New("provided closer does not support CloseWithError") | ||
} | ||
|
||
func (re *writerResponseEmitter) SetError(v interface{}, errType cmdkit.ErrorType) { | ||
err := re.Emit(&cmdkit.Error{Message: fmt.Sprint(v), Code: errType}) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Shouldn't we remove this function?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We can't just remove it, because then it wouldn't satisfy the interface anymore. But I made it simply log an error.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We need to consider the story around calling Close
and CloseWithError
. We could tell users not to do that and to simply return the errors from Run
or we could say the first one wins.
If we say the first one wins, we'll need to fix several of the CloseWithError
implementations so that they don't barf when called twice. We'll also want to avoid logging to noisily when we accidentally close twice.
If we do say "don't close, well do it for you", then we'd need to figure out how to handle Single
as that does close.
// if they are errors | ||
if e, ok := v.(error); ok { | ||
if err, ok := v.(error); ok { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah. Got it. I thought you were trying to overwrite the named return value err
.
chan.go
Outdated
@@ -155,6 +151,10 @@ func (re *chanResponseEmitter) Emit(v interface{}) error { | |||
|
|||
select { | |||
case re.ch <- v: | |||
if _, ok := v.(Single); ok { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nit: We could unwrap this before sending to simplify the Next()
implementation. However, that's up to you (and may not save any code).
chan.go
Outdated
// waitErr is closed when the stream is closed. | ||
// Error checks if the stream has been closed by checking if this channes is closed. | ||
// Its closing is protected by wl. | ||
waitErr chan struct{} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nit: I'd just call this closedCh
or something like that. waitErr
implies some relation to the error.
case Single: | ||
return val.Value, nil | ||
default: | ||
return v, nil | ||
} | ||
case <-ctx.Done(): | ||
close(r.done) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So, there's a subtle but probably harmless issue here: Next()
can return an error while re.Error()
won't. We may want to close here (with ctx.Err()
?).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not sure. That would mean that if one call to Next()
gets cancelled, the stream will end. Intuitively I would say that the stream should live on and the caller should be able to do another call to Next()
later. Don't you think?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, no, you're totally right.
chan.go
Outdated
|
||
if re.ch == nil { | ||
return fmt.Errorf("emitter closed") | ||
// unblock Length() and Error() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Doesn't unblock Error()
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should still fix this comment.
cli/responseemitter.go
Outdated
return re.Close() | ||
} | ||
|
||
e, ok := err.(*cmdkit.Error) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should probably be consistent with the cmdkit.Error
-> *cmdkit.Error
dance. We should either do that everywhere or nowhere.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This still looks like an issue. We should probably have a "if cmdkit.Error, convert to *cmdkit.Error" check here.
cli/responseemitter.go
Outdated
@@ -200,6 +211,10 @@ func (re *responseEmitter) Emit(v interface{}) error { | |||
} | |||
} | |||
|
|||
if isSingle { | |||
return re.Close() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This'll return success to the user if, e.g., encoding fails. Should probably call re.CloseWithErr(err)
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This still looks like an issue.
writer.go
Outdated
// old error emitting semantics. | ||
// Also errors may occur both as pointers and as plain values, so we need to | ||
// check both. | ||
debug.AssertNotError(v) | ||
|
||
if _, ok := v.(Single); ok { | ||
defer re.Close() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Will close without error if encoding fails.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think I would send the encoding error down the stream. If I try to send an unencodable value through a writer, the stream should live on and I should get an error so I can send something else.
However, we should return the Close()
error, if there is one.
writer.go
Outdated
func (re *WriterResponseEmitter) SetLength(length uint64) { | ||
func (re *writerResponseEmitter) SetError(v interface{}, errType cmdkit.ErrorType) { | ||
err := &cmdkit.Error{Message: fmt.Sprint(v), Code: errType} | ||
log.Errorf("error setting error %q: cannot set error with writerResponseEmitter", err) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If we can't do the right thing here, maybe we should just remove this from the interface. We really don't want the user to try calling this and then return a "success" to the user.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actually, hasn't this been removed from the interface?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, looks like I forgot deleting it. done.
http/responseemitter.go
Outdated
|
||
if single, ok := value.(cmds.Single); ok { | ||
value = single.Value | ||
defer re.closeWithError(nil) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Will throw away encoding errors.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
IMHO encoding errors should be passed to the caller instead of be used to close the stream. And closeWithError
always returns nil
so I figured we don't need it to handle the error.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The issue is that we're closing the response stream as if the command succeeded when, really, the command failed.
FYI, if I take more than a week to get to a review, the notification has been buried in my inbox. At that point, just bug me on IRC (really, feel free to bug me on IRC repeatedly as soon as you have something for me to review). |
Thank you for the review. I was traveling and will work on these over the next week. And okay, next time I'll be more stubborn :) |
Hey @Stebalien I now commented or acted on everything except the thoughts on |
I think the most idiomatic way is to ignore double close errors when getting an error on err = re.Close()
if err != ErrClosingClosedEmitter && err != nil {
// handle error
} Progress:
|
And possibly some places in go-ipfs. Warning, @whyrusleeping is grumbling about this being a massive breaking change. My plan is:
|
- setting the error is moved into the preamble - we call the preamble before the err!=nil branch because we send to the conn in there
This reverts commit a011832. Do not remove tests that reveal race conditions.
shadow err
So, it turns out I was wrong about how we currently handle HTTP headers/errors. We always handle trailing errors using the Given this, I believe we can switch to sending errors-in-trailers only for all 200 responses (don't additionally include the value). Thoughts? This should cover both the old client talking to a new server and a new server talking to an old client cases. I'd like to eventually move to:
But we can tackle this later. |
@Stebalien done! Want to give it another round of review? |
http/parse.go
Outdated
@@ -218,7 +218,10 @@ func parseResponse(httpRes *http.Response, req *cmds.Request) (cmds.Response, er | |||
return nil, fmt.Errorf("unknown error content type: %s", contentType) | |||
default: | |||
// handle errors from headers | |||
e.Message = httpRes.Header.Get(StreamErrHeader) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should we not be returning an error in this case?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As it stands now the error will be returned on the first call to Next()
, but I can also make it return immediately. I believe that made the test case more complex, but maybe that's okay.
http/responseemitter.go
Outdated
err = re.closeWithError(err) | ||
// always close singles with nil error. | ||
// encoding errors go to caller. | ||
err = re.closeWithError(nil) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If we return the error to the "caller", expecting them to do something, we can't close. This should be something like isSingle && err != nil
. Am I missing something?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It should have been isSingle && err == nil
. If an encoding or copy error occurs, we didn't send the value so it doesn't make sense to close the emitter but we should return it. If encoding went fine, gracefully close the emitter.
You know, I think this is pretty much ready (modulo those two questions/issues). This is going to be awesome! |
Hey @Stebalien, please CR :) |
@keks all ready to go! I'll merge this as soon as ipfs/kubo#5035 is ready. |
This branch bundles a couple of changes so we can test against all of them at once in go-ipfs