-
Notifications
You must be signed in to change notification settings - Fork 9.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
clientv3: fix health balancer, retry #8688
Conversation
why this is still wip? |
@@ -473,8 +473,8 @@ func TestKVNewAfterClose(t *testing.T) { | |||
|
|||
donec := make(chan struct{}) | |||
go func() { | |||
if _, err := cli.Get(context.TODO(), "foo"); err != context.Canceled { | |||
t.Fatalf("expected %v, got %v", context.Canceled, err) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is added because previously, no matter what, retry.go
switches endpoint (resets connection) so didn't hit grpc.ErrClientConnClosing
path. Now, only does switch when it's transient error, so grpc.ErrClientConnClosing
is expected when concurrent goroutine closes client.
68be468
to
a56d5f4
Compare
clientv3/retry.go
Outdated
@@ -181,7 +190,7 @@ type retryLeaseClient struct { | |||
func RetryLeaseClient(c *Client) pb.LeaseClient { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
comment needs to be fixed.
clientv3/retry.go
Outdated
@@ -181,7 +190,7 @@ type retryLeaseClient struct { | |||
func RetryLeaseClient(c *Client) pb.LeaseClient { | |||
retry := &retryLeaseClient{ | |||
pb.NewLeaseClient(c.conn), | |||
c.newRetryWrapper(isReadStopError), | |||
c.newRetryWrapper(false), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this is incorrect. leaseGrant is write, but list leases is read.
clientv3/retry.go
Outdated
type retryClusterClient struct { | ||
pb.ClusterClient | ||
retryf retryRpcFunc | ||
} | ||
|
||
// RetryClusterClient implements a ClusterClient that uses the client's FailFast retry policy. | ||
func RetryClusterClient(c *Client) pb.ClusterClient { | ||
return &retryClusterClient{pb.NewClusterClient(c.conn), c.newRetryWrapper(isWriteStopError)} | ||
return &retryClusterClient{pb.NewClusterClient(c.conn), c.newRetryWrapper(true)} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
not correct. memerlist is read.
we need to make the wrapper to understand read/write type per RPC, not per Service. A service might have both read/write type RPCs. |
d915681
to
b0dee09
Compare
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
Set grpc.FailFast==true, so that RPCs fail immediately on broken connections or unreachable servers. And let client balancer handle the retries. Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
…erClose Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
9396c35
to
db175aa
Compare
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
c.f. efd7800