Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
transport: fix error handling on Stream deletion (#1275)
This patch writes client-side error before closing the active stream to fix blocking `RecvMsg` issue on `grpc.ClientStream` [1]. Previous gRPC client stream just exits on `ClientTransport.Error` [2]. And latest gRPC added another select case on client connection context cancel [3]. Now when client stream closes from client connection context cancel, it calls `CloseStream` with `ErrClientConnClosing` error. And then the stream gets deleted from `*http2Client.activeStreams`, without processing the error [4]. Then in-flight `RecvMsg` call on this client will block on `*parser.Reader.recvMsg` [5]. In short, 1. `ClientConn.Close`. 2. in-flight streams will receive case `<-cc.ctx.Done()` https://github.com/grpc/grpc-go/blob/master/stream.go#L253-L255. 3. `cs.closeTransportStream(ErrClientConnClosing)` calls `cs.t.CloseStream(cs.s, err)`. 4. `CloseStream(cs.s, err)` calls `delete(t.activeStreams, s.id)` without handling the error. 5. in-flight streams will never receive error, left hanging. I can reproduce in etcd tests with in-flight `recvMsg` calls to `Observe` RPC. --- [1] etcd-io/etcd#7896 (comment) [2] https://github.com/grpc/grpc-go/blob/v1.2.x/stream.go#L235-L238 [3] #1136 [4] https://github.com/grpc/grpc-go/blob/master/transport/http2_client.go#L569 [5] https://github.com/grpc/grpc-go/blob/master/rpc_util.go#L280 Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
- Loading branch information