Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

clientv3/integration: add TestBalancerUnderServerShutdownMutable* #8772

Merged
merged 1 commit into from
Oct 26, 2017

Conversation

gyuho
Copy link
Contributor

@gyuho gyuho commented Oct 26, 2017


timeout := time.Second
failed := false
for i := 0; i < 30; i++ {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do we need 30 retries? we should expect exact one retry, right?

Copy link
Contributor Author

@gyuho gyuho Oct 26, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had 30 retries in case request never fails (try up to ~3-sec), or the error is transport is closing which takes more than 100ms to recover(endpoint switch)--so it takes more than 1-retry to succeed.

Changed the logic to test with only one retry after failure, by giving more wait time.

// shut down eps[0]
clus.Members[0].Terminate(t)

// switched to others when eps[0] was shut down
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

was explicitly shut down

@xiang90
Copy link
Contributor

xiang90 commented Oct 26, 2017

lgtm

@xiang90
Copy link
Contributor

xiang90 commented Oct 26, 2017

/cc @jpbetz

}

// testBalancerUnderServerShutdownMutable expects when the member of the
// pinned endpoint is shut down, balancer switch its endpoints and following
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

grammar nit: "...expects that when the member of the pinned endpoint is shut down, the balancer switches its endpoints and all subsequent..."


// switched to others when eps[0] was explicitly shut down
// and following request should succeed
time.Sleep(time.Second)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Potential source of test flakiness? Is there a condition we can poll for that would be more reliable?

Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
@gyuho
Copy link
Contributor Author

gyuho commented Oct 26, 2017

@jpbetz All fixed. And added TODO to remove sleeps. We have ConnectNotify internally for balancer, but not good enough to detect endpoint switch.

@jpbetz
Copy link
Contributor

jpbetz commented Oct 26, 2017

lgtm

Thanks!

@gyuho gyuho merged commit 0160cd7 into etcd-io:master Oct 26, 2017
@gyuho gyuho deleted the shutdown branch October 26, 2017 23:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants