Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

TestCluster_Recover_peer_client_manual_TLS_scheme hangs #216

Closed
gyuho opened this issue Apr 18, 2017 · 1 comment · Fixed by etcd-io/etcd#7768
Closed

TestCluster_Recover_peer_client_manual_TLS_scheme hangs #216

gyuho opened this issue Apr 18, 2017 · 1 comment · Fixed by etcd-io/etcd#7768
Assignees
@gyuho
Labels
Backend Bug

Comments

@gyuho
Copy link
Contributor

gyuho commented Apr 18, 2017 

=== RUN   TestCluster_Recover_peer_client_manual_TLS_scheme



starting cluster
2017-04-18 11:11:30.828522 I | cluster: starting 3 Members (root directory "/var/folders/bf/3l3rdj1d2293ng3jsk71dlsc0000gn/T/cluster-test048905948", root port :1300)
2017-04-18 11:11:30.828618 I | cluster: removing root directory "/var/folders/bf/3l3rdj1d2293ng3jsk71dlsc0000gn/T/cluster-test048905948"
2017-04-18 11:11:30.828694 I | cluster: getting default host
2017-04-18 11:11:30.828699 W | cluster: default host not supported on darwin_amd64
2017-04-18 11:11:30.828703 W | cluster: overwriting default host with 'localhost
2017-04-18 11:11:30.828706 I | cluster: detected default host "localhost"
2017-04-18 11:11:30.828736 I | cluster: removed "/var/folders/bf/3l3rdj1d2293ng3jsk71dlsc0000gn/T/cluster-test048905948/node1.data-dir-etcd"
2017-04-18 11:11:30.828747 I | cluster: removed "/var/folders/bf/3l3rdj1d2293ng3jsk71dlsc0000gn/T/cluster-test048905948/node1.data-dir-etcd/wal"
2017-04-18 11:11:30.828752 I | cluster: "node1" is set up to listen on client url "https://localhost:1300"
2017-04-18 11:11:30.828767 I | cluster: removed "/var/folders/bf/3l3rdj1d2293ng3jsk71dlsc0000gn/T/cluster-test048905948/node2.data-dir-etcd"
2017-04-18 11:11:30.828776 I | cluster: removed "/var/folders/bf/3l3rdj1d2293ng3jsk71dlsc0000gn/T/cluster-test048905948/node2.data-dir-etcd/wal"
2017-04-18 11:11:30.828780 I | cluster: "node2" is set up to listen on client url "https://localhost:1302"
2017-04-18 11:11:30.828793 I | cluster: removed "/var/folders/bf/3l3rdj1d2293ng3jsk71dlsc0000gn/T/cluster-test048905948/node3.data-dir-etcd"
2017-04-18 11:11:30.828801 I | cluster: removed "/var/folders/bf/3l3rdj1d2293ng3jsk71dlsc0000gn/T/cluster-test048905948/node3.data-dir-etcd/wal"
2017-04-18 11:11:30.828807 I | cluster: "node3" is set up to listen on client url "https://localhost:1304"
2017-04-18 11:11:30.828880 I | embed: peerTLS: cert = ../test-certs/test-cert.pem, key = ../test-certs/test-cert-key.pem, ca = , trusted-ca = ../test-certs/trusted-ca.pem, client-cert-auth = true
2017-04-18 11:11:30.828886 I | embed: peerTLS: cert = ../test-certs/test-cert.pem, key = ../test-certs/test-cert-key.pem, ca = , trusted-ca = ../test-certs/trusted-ca.pem, client-cert-auth = true
2017-04-18 11:11:30.828903 I | embed: peerTLS: cert = ../test-certs/test-cert.pem, key = ../test-certs/test-cert-key.pem, ca = , trusted-ca = ../test-certs/trusted-ca.pem, client-cert-auth = true
2017-04-18 11:11:30.832414 I | embed: listening for peers on https://localhost:1305
2017-04-18 11:11:30.832438 I | embed: listening for peers on https://localhost:1303
2017-04-18 11:11:30.832454 I | embed: listening for peers on https://localhost:1301
2017-04-18 11:11:30.832925 I | embed: listening for client requests on localhost:1300
2017-04-18 11:11:30.832935 I | embed: listening for client requests on localhost:1302
2017-04-18 11:11:30.832955 I | embed: listening for client requests on localhost:1304
2017-04-18 11:11:30.915906 I | etcdserver: name = node3
2017-04-18 11:11:30.915922 I | etcdserver: data dir = /var/folders/bf/3l3rdj1d2293ng3jsk71dlsc0000gn/T/cluster-test048905948/node3.data-dir-etcd
2017-04-18 11:11:30.915927 I | etcdserver: member dir = /var/folders/bf/3l3rdj1d2293ng3jsk71dlsc0000gn/T/cluster-test048905948/node3.data-dir-etcd/member
2017-04-18 11:11:30.915930 I | etcdserver: dedicated WAL dir = /var/folders/bf/3l3rdj1d2293ng3jsk71dlsc0000gn/T/cluster-test048905948/node3.data-dir-etcd/wal
2017-04-18 11:11:30.915933 I | etcdserver: heartbeat = 100ms
2017-04-18 11:11:30.915936 I | etcdserver: election = 1000ms
2017-04-18 11:11:30.915938 I | etcdserver: snapshot count = 100000
2017-04-18 11:11:30.915945 I | etcdserver: advertise client URLs = https://localhost:1304
2017-04-18 11:11:30.915949 I | etcdserver: initial advertise peer URLs = https://localhost:1305
2017-04-18 11:11:30.915958 I | etcdserver: initial cluster = node1=https://localhost:1301,node2=https://localhost:1303,node3=https://localhost:1305
2017-04-18 11:11:30.932199 I | etcdserver: name = node2
2017-04-18 11:11:30.932233 I | etcdserver: data dir = /var/folders/bf/3l3rdj1d2293ng3jsk71dlsc0000gn/T/cluster-test048905948/node2.data-dir-etcd
2017-04-18 11:11:30.932240 I | etcdserver: member dir = /var/folders/bf/3l3rdj1d2293ng3jsk71dlsc0000gn/T/cluster-test048905948/node2.data-dir-etcd/member
2017-04-18 11:11:30.932243 I | etcdserver: dedicated WAL dir = /var/folders/bf/3l3rdj1d2293ng3jsk71dlsc0000gn/T/cluster-test048905948/node2.data-dir-etcd/wal
2017-04-18 11:11:30.932246 I | etcdserver: heartbeat = 100ms
2017-04-18 11:11:30.932249 I | etcdserver: election = 1000ms
2017-04-18 11:11:30.932252 I | etcdserver: snapshot count = 100000
2017-04-18 11:11:30.932258 I | etcdserver: advertise client URLs = https://localhost:1302
2017-04-18 11:11:30.932261 I | etcdserver: initial advertise peer URLs = https://localhost:1303
2017-04-18 11:11:30.932270 I | etcdserver: initial cluster = node1=https://localhost:1301,node2=https://localhost:1303,node3=https://localhost:1305
2017-04-18 11:11:30.933000 I | etcdserver: name = node1
2017-04-18 11:11:30.933026 I | etcdserver: data dir = /var/folders/bf/3l3rdj1d2293ng3jsk71dlsc0000gn/T/cluster-test048905948/node1.data-dir-etcd
2017-04-18 11:11:30.933036 I | etcdserver: member dir = /var/folders/bf/3l3rdj1d2293ng3jsk71dlsc0000gn/T/cluster-test048905948/node1.data-dir-etcd/member
2017-04-18 11:11:30.933042 I | etcdserver: dedicated WAL dir = /var/folders/bf/3l3rdj1d2293ng3jsk71dlsc0000gn/T/cluster-test048905948/node1.data-dir-etcd/wal
2017-04-18 11:11:30.933047 I | etcdserver: heartbeat = 100ms
2017-04-18 11:11:30.933052 I | etcdserver: election = 1000ms
2017-04-18 11:11:30.933056 I | etcdserver: snapshot count = 100000
2017-04-18 11:11:30.933064 I | etcdserver: advertise client URLs = https://localhost:1300
2017-04-18 11:11:30.933071 I | etcdserver: initial advertise peer URLs = https://localhost:1301
2017-04-18 11:11:30.933085 I | etcdserver: initial cluster = node1=https://localhost:1301,node2=https://localhost:1303,node3=https://localhost:1305
2017-04-18 11:11:31.094618 I | etcdserver: starting member 3eb1e71170c6eb84 in cluster 8899aea05aa44bd2
2017-04-18 11:11:31.094667 I | etcdserver: starting member 7d0d073235bdca52 in cluster 8899aea05aa44bd2
2017-04-18 11:11:31.094685 I | raft: 3eb1e71170c6eb84 became follower at term 0
2017-04-18 11:11:31.094695 I | raft: 7d0d073235bdca52 became follower at term 0
2017-04-18 11:11:31.094708 I | raft: newRaft 3eb1e71170c6eb84 [peers: [], term: 0, commit: 0, applied: 0, lastindex: 0, lastterm: 0]
2017-04-18 11:11:31.094715 I | raft: newRaft 7d0d073235bdca52 [peers: [], term: 0, commit: 0, applied: 0, lastindex: 0, lastterm: 0]
2017-04-18 11:11:31.094724 I | raft: 3eb1e71170c6eb84 became follower at term 1
2017-04-18 11:11:31.094731 I | raft: 7d0d073235bdca52 became follower at term 1
2017-04-18 11:11:31.101859 I | etcdserver: starting member 9195b0bacc666c6 in cluster 8899aea05aa44bd2
2017-04-18 11:11:31.101913 I | raft: 9195b0bacc666c6 became follower at term 0
2017-04-18 11:11:31.101949 I | raft: newRaft 9195b0bacc666c6 [peers: [], term: 0, commit: 0, applied: 0, lastindex: 0, lastterm: 0]
2017-04-18 11:11:31.101958 I | raft: 9195b0bacc666c6 became follower at term 1
2017-04-18 11:11:31.103436 W | auth: simple token is not cryptographically signed
2017-04-18 11:11:31.104283 W | auth: simple token is not cryptographically signed
2017-04-18 11:11:31.106043 W | auth: simple token is not cryptographically signed
2017-04-18 11:11:31.108674 I | rafthttp: starting peer 9195b0bacc666c6...
2017-04-18 11:11:31.108712 I | rafthttp: started HTTP pipelining with peer 9195b0bacc666c6
2017-04-18 11:11:31.109062 I | rafthttp: started streaming with peer 9195b0bacc666c6 (writer)
2017-04-18 11:11:31.109767 I | rafthttp: started streaming with peer 9195b0bacc666c6 (writer)
2017-04-18 11:11:31.109813 I | rafthttp: starting peer 9195b0bacc666c6...
2017-04-18 11:11:31.109864 I | rafthttp: started HTTP pipelining with peer 9195b0bacc666c6
2017-04-18 11:11:31.110597 I | rafthttp: starting peer 3eb1e71170c6eb84...
2017-04-18 11:11:31.110651 I | rafthttp: started HTTP pipelining with peer 3eb1e71170c6eb84
2017-04-18 11:11:31.111302 I | rafthttp: started peer 9195b0bacc666c6
2017-04-18 11:11:31.111330 I | rafthttp: added peer 9195b0bacc666c6
2017-04-18 11:11:31.111340 I | rafthttp: starting peer 7d0d073235bdca52...
2017-04-18 11:11:31.111352 I | rafthttp: started HTTP pipelining with peer 7d0d073235bdca52
2017-04-18 11:11:31.111401 I | rafthttp: started streaming with peer 9195b0bacc666c6 (stream MsgApp v2 reader)
2017-04-18 11:11:31.111416 I | rafthttp: started streaming with peer 9195b0bacc666c6 (stream Message reader)
2017-04-18 11:11:31.111439 I | rafthttp: started streaming with peer 9195b0bacc666c6 (writer)
2017-04-18 11:11:31.111788 I | rafthttp: started streaming with peer 9195b0bacc666c6 (writer)
2017-04-18 11:11:31.111872 I | rafthttp: started streaming with peer 7d0d073235bdca52 (writer)
2017-04-18 11:11:31.112919 I | rafthttp: started streaming with peer 3eb1e71170c6eb84 (writer)
2017-04-18 11:11:31.113109 I | rafthttp: started streaming with peer 7d0d073235bdca52 (writer)
2017-04-18 11:11:31.114737 I | rafthttp: started peer 9195b0bacc666c6
2017-04-18 11:11:31.114767 I | rafthttp: started streaming with peer 9195b0bacc666c6 (stream MsgApp v2 reader)
2017-04-18 11:11:31.114781 I | rafthttp: started streaming with peer 9195b0bacc666c6 (stream Message reader)
2017-04-18 11:11:31.115385 I | rafthttp: started streaming with peer 3eb1e71170c6eb84 (writer)
2017-04-18 11:11:31.115407 I | rafthttp: started peer 7d0d073235bdca52
2017-04-18 11:11:31.115420 I | rafthttp: added peer 7d0d073235bdca52
2017-04-18 11:11:31.115435 I | rafthttp: added peer 9195b0bacc666c6
2017-04-18 11:11:31.115452 I | rafthttp: starting peer 3eb1e71170c6eb84...
2017-04-18 11:11:31.115463 I | rafthttp: started streaming with peer 7d0d073235bdca52 (stream MsgApp v2 reader)
2017-04-18 11:11:31.115500 I | rafthttp: started streaming with peer 7d0d073235bdca52 (stream Message reader)
2017-04-18 11:11:31.115527 I | rafthttp: started HTTP pipelining with peer 3eb1e71170c6eb84
2017-04-18 11:11:31.115546 I | etcdserver: starting server... [version: 3.2.0+git, cluster version: to_be_decided]
2017-04-18 11:11:31.115573 I | embed: ClientTLS: cert = ../test-certs/test-cert.pem, key = ../test-certs/test-cert-key.pem, ca = , trusted-ca = ../test-certs/trusted-ca.pem, client-cert-auth = true
2017-04-18 11:11:31.115693 E | etcdserver: cannot monitor file descriptor usage (cannot get FDUsage on darwin)
2017-04-18 11:11:31.116247 I | etcdserver/membership: added member 9195b0bacc666c6 [https://localhost:1303] to cluster 8899aea05aa44bd2
2017-04-18 11:11:31.116391 I | etcdserver/membership: added member 3eb1e71170c6eb84 [https://localhost:1305] to cluster 8899aea05aa44bd2
2017-04-18 11:11:31.116468 I | etcdserver/membership: added member 7d0d073235bdca52 [https://localhost:1301] to cluster 8899aea05aa44bd2
2017-04-18 11:11:31.116631 I | rafthttp: started streaming with peer 3eb1e71170c6eb84 (writer)
2017-04-18 11:11:31.117562 I | rafthttp: started peer 3eb1e71170c6eb84
2017-04-18 11:11:31.117622 I | rafthttp: added peer 3eb1e71170c6eb84
2017-04-18 11:11:31.117639 I | rafthttp: starting peer 7d0d073235bdca52...
2017-04-18 11:11:31.117896 I | rafthttp: started HTTP pipelining with peer 7d0d073235bdca52
2017-04-18 11:11:31.120079 I | rafthttp: started peer 7d0d073235bdca52
2017-04-18 11:11:31.120107 I | rafthttp: added peer 7d0d073235bdca52
2017-04-18 11:11:31.120128 I | etcdserver: starting server... [version: 3.2.0+git, cluster version: to_be_decided]
2017-04-18 11:11:31.120169 I | embed: ClientTLS: cert = ../test-certs/test-cert.pem, key = ../test-certs/test-cert-key.pem, ca = , trusted-ca = ../test-certs/trusted-ca.pem, client-cert-auth = true
2017-04-18 11:11:31.122605 I | rafthttp: started peer 3eb1e71170c6eb84
2017-04-18 11:11:31.122641 I | rafthttp: added peer 3eb1e71170c6eb84
2017-04-18 11:11:31.122667 I | etcdserver: starting server... [version: 3.2.0+git, cluster version: to_be_decided]
2017-04-18 11:11:31.122700 I | embed: ClientTLS: cert = ../test-certs/test-cert.pem, key = ../test-certs/test-cert-key.pem, ca = , trusted-ca = ../test-certs/trusted-ca.pem, client-cert-auth = true
2017-04-18 11:11:31.123923 I | rafthttp: started streaming with peer 3eb1e71170c6eb84 (stream MsgApp v2 reader)
2017-04-18 11:11:31.124143 I | rafthttp: started streaming with peer 3eb1e71170c6eb84 (writer)
2017-04-18 11:11:31.124222 I | rafthttp: started streaming with peer 3eb1e71170c6eb84 (stream Message reader)
2017-04-18 11:11:31.124268 I | rafthttp: started streaming with peer 3eb1e71170c6eb84 (stream MsgApp v2 reader)
2017-04-18 11:11:31.124415 I | rafthttp: started streaming with peer 3eb1e71170c6eb84 (stream Message reader)
2017-04-18 11:11:31.124431 I | rafthttp: started streaming with peer 7d0d073235bdca52 (writer)
2017-04-18 11:11:31.124455 I | rafthttp: started streaming with peer 7d0d073235bdca52 (writer)
2017-04-18 11:11:31.125165 I | rafthttp: started streaming with peer 7d0d073235bdca52 (stream MsgApp v2 reader)
2017-04-18 11:11:31.125263 I | rafthttp: started streaming with peer 7d0d073235bdca52 (stream Message reader)
2017-04-18 11:11:31.125314 E | etcdserver: cannot monitor file descriptor usage (cannot get FDUsage on darwin)
2017-04-18 11:11:31.125497 I | etcdserver/membership: added member 9195b0bacc666c6 [https://localhost:1303] to cluster 8899aea05aa44bd2
2017-04-18 11:11:31.125653 I | etcdserver/membership: added member 3eb1e71170c6eb84 [https://localhost:1305] to cluster 8899aea05aa44bd2
2017-04-18 11:11:31.125685 E | etcdserver: cannot monitor file descriptor usage (cannot get FDUsage on darwin)
2017-04-18 11:11:31.125724 I | etcdserver/membership: added member 7d0d073235bdca52 [https://localhost:1301] to cluster 8899aea05aa44bd2
2017-04-18 11:11:31.125843 I | etcdserver/membership: added member 9195b0bacc666c6 [https://localhost:1303] to cluster 8899aea05aa44bd2
2017-04-18 11:11:31.125925 I | etcdserver/membership: added member 3eb1e71170c6eb84 [https://localhost:1305] to cluster 8899aea05aa44bd2
2017-04-18 11:11:31.125997 I | etcdserver/membership: added member 7d0d073235bdca52 [https://localhost:1301] to cluster 8899aea05aa44bd2
2017-04-18 11:11:31.604160 I | raft: 3eb1e71170c6eb84 is starting a new election at term 1
2017-04-18 11:11:31.604255 I | raft: 3eb1e71170c6eb84 became candidate at term 2
2017-04-18 11:11:31.604278 I | raft: 3eb1e71170c6eb84 received MsgVoteResp from 3eb1e71170c6eb84 at term 2
2017-04-18 11:11:31.604310 I | raft: 3eb1e71170c6eb84 [logterm: 1, index: 3] sent MsgVote request to 9195b0bacc666c6 at term 2
2017-04-18 11:11:31.604325 I | raft: 3eb1e71170c6eb84 [logterm: 1, index: 3] sent MsgVote request to 7d0d073235bdca52 at term 2
2017-04-18 11:11:31.713836 I | raft: 7d0d073235bdca52 is starting a new election at term 1
2017-04-18 11:11:31.713997 I | raft: 7d0d073235bdca52 became candidate at term 2
2017-04-18 11:11:31.714014 I | raft: 7d0d073235bdca52 received MsgVoteResp from 7d0d073235bdca52 at term 2
2017-04-18 11:11:31.714024 I | raft: 7d0d073235bdca52 [logterm: 1, index: 3] sent MsgVote request to 9195b0bacc666c6 at term 2
2017-04-18 11:11:31.714093 I | raft: 7d0d073235bdca52 [logterm: 1, index: 3] sent MsgVote request to 3eb1e71170c6eb84 at term 2
2017-04-18 11:11:31.905171 I | raft: 9195b0bacc666c6 is starting a new election at term 1
2017-04-18 11:11:31.905206 I | raft: 9195b0bacc666c6 became candidate at term 2
2017-04-18 11:11:31.905213 I | raft: 9195b0bacc666c6 received MsgVoteResp from 9195b0bacc666c6 at term 2
2017-04-18 11:11:31.905219 I | raft: 9195b0bacc666c6 [logterm: 1, index: 3] sent MsgVote request to 3eb1e71170c6eb84 at term 2
2017-04-18 11:11:31.905225 I | raft: 9195b0bacc666c6 [logterm: 1, index: 3] sent MsgVote request to 7d0d073235bdca52 at term 2
2017-04-18 11:11:33.104357 I | raft: 3eb1e71170c6eb84 is starting a new election at term 2
2017-04-18 11:11:33.104376 I | raft: 3eb1e71170c6eb84 became candidate at term 3
2017-04-18 11:11:33.104383 I | raft: 3eb1e71170c6eb84 received MsgVoteResp from 3eb1e71170c6eb84 at term 3
2017-04-18 11:11:33.104389 I | raft: 3eb1e71170c6eb84 [logterm: 1, index: 3] sent MsgVote request to 9195b0bacc666c6 at term 3
2017-04-18 11:11:33.104394 I | raft: 3eb1e71170c6eb84 [logterm: 1, index: 3] sent MsgVote request to 7d0d073235bdca52 at term 3
2017-04-18 11:11:33.202971 I | raft: 7d0d073235bdca52 is starting a new election at term 2
2017-04-18 11:11:33.202998 I | raft: 7d0d073235bdca52 became candidate at term 3
2017-04-18 11:11:33.203006 I | raft: 7d0d073235bdca52 received MsgVoteResp from 7d0d073235bdca52 at term 3
2017-04-18 11:11:33.203017 I | raft: 7d0d073235bdca52 [logterm: 1, index: 3] sent MsgVote request to 9195b0bacc666c6 at term 3
2017-04-18 11:11:33.203026 I | raft: 7d0d073235bdca52 [logterm: 1, index: 3] sent MsgVote request to 3eb1e71170c6eb84 at term 3
2017-04-18 11:11:33.604603 I | raft: 9195b0bacc666c6 is starting a new election at term 2
2017-04-18 11:11:33.604622 I | raft: 9195b0bacc666c6 became candidate at term 3
2017-04-18 11:11:33.604630 I | raft: 9195b0bacc666c6 received MsgVoteResp from 9195b0bacc666c6 at term 3
2017-04-18 11:11:33.604636 I | raft: 9195b0bacc666c6 [logterm: 1, index: 3] sent MsgVote request to 3eb1e71170c6eb84 at term 3
2017-04-18 11:11:33.604641 I | raft: 9195b0bacc666c6 [logterm: 1, index: 3] sent MsgVote request to 7d0d073235bdca52 at term 3
2017-04-18 11:11:34.805730 I | raft: 9195b0bacc666c6 is starting a new election at term 3
2017-04-18 11:11:34.805780 I | raft: 9195b0bacc666c6 became candidate at term 4
2017-04-18 11:11:34.805797 I | raft: 9195b0bacc666c6 received MsgVoteResp from 9195b0bacc666c6 at term 4
2017-04-18 11:11:34.805814 I | raft: 9195b0bacc666c6 [logterm: 1, index: 3] sent MsgVote request to 3eb1e71170c6eb84 at term 4
2017-04-18 11:11:34.805828 I | raft: 9195b0bacc666c6 [logterm: 1, index: 3] sent MsgVote request to 7d0d073235bdca52 at term 4
2017-04-18 11:11:34.908457 I | raft: 3eb1e71170c6eb84 is starting a new election at term 3
2017-04-18 11:11:34.908482 I | raft: 3eb1e71170c6eb84 became candidate at term 4
2017-04-18 11:11:34.908491 I | raft: 3eb1e71170c6eb84 received MsgVoteResp from 3eb1e71170c6eb84 at term 4
2017-04-18 11:11:34.908502 I | raft: 3eb1e71170c6eb84 [logterm: 1, index: 3] sent MsgVote request to 9195b0bacc666c6 at term 4
2017-04-18 11:11:34.908511 I | raft: 3eb1e71170c6eb84 [logterm: 1, index: 3] sent MsgVote request to 7d0d073235bdca52 at term 4
2017-04-18 11:11:35.104155 I | raft: 7d0d073235bdca52 is starting a new election at term 3
2017-04-18 11:11:35.104177 I | raft: 7d0d073235bdca52 became candidate at term 4
2017-04-18 11:11:35.104185 I | raft: 7d0d073235bdca52 received MsgVoteResp from 7d0d073235bdca52 at term 4
2017-04-18 11:11:35.104193 I | raft: 7d0d073235bdca52 [logterm: 1, index: 3] sent MsgVote request to 9195b0bacc666c6 at term 4
2017-04-18 11:11:35.104201 I | raft: 7d0d073235bdca52 [logterm: 1, index: 3] sent MsgVote request to 3eb1e71170c6eb84 at term 4
2017-04-18 11:11:36.111965 W | rafthttp: health check for peer 9195b0bacc666c6 could not connect: EOF
2017-04-18 11:11:36.116128 W | rafthttp: health check for peer 7d0d073235bdca52 could not connect: EOF
2017-04-18 11:11:36.116191 W | rafthttp: health check for peer 9195b0bacc666c6 could not connect: EOF
2017-04-18 11:11:36.125982 W | rafthttp: health check for peer 3eb1e71170c6eb84 could not connect: EOF
2017-04-18 11:11:36.126040 W | rafthttp: health check for peer 7d0d073235bdca52 could not connect: EOF
2017-04-18 11:11:36.126053 W | rafthttp: health check for peer 3eb1e71170c6eb84 could not connect: EOF
2017-04-18 11:11:36.203260 I | raft: 9195b0bacc666c6 is starting a new election at term 4
2017-04-18 11:11:36.203284 I | raft: 9195b0bacc666c6 became candidate at term 5
2017-04-18 11:11:36.203292 I | raft: 9195b0bacc666c6 received MsgVoteResp from 9195b0bacc666c6 at term 5
2017-04-18 11:11:36.203301 I | raft: 9195b0bacc666c6 [logterm: 1, index: 3] sent MsgVote request to 3eb1e71170c6eb84 at term 5
2017-04-18 11:11:36.203309 I | raft: 9195b0bacc666c6 [logterm: 1, index: 3] sent MsgVote request to 7d0d073235bdca52 at term 5
2017-04-18 11:11:36.216076 I | raft: 7d0d073235bdca52 is starting a new election at term 4
2017-04-18 11:11:36.216095 I | raft: 7d0d073235bdca52 became candidate at term 5
2017-04-18 11:11:36.216104 I | raft: 7d0d073235bdca52 received MsgVoteResp from 7d0d073235bdca52 at term 5
2017-04-18 11:11:36.216112 I | raft: 7d0d073235bdca52 [logterm: 1, index: 3] sent MsgVote request to 9195b0bacc666c6 at term 5
2017-04-18 11:11:36.216119 I | raft: 7d0d073235bdca52 [logterm: 1, index: 3] sent MsgVote request to 3eb1e71170c6eb84 at term 5
2017-04-18 11:11:36.217252 I | raft: 3eb1e71170c6eb84 is starting a new election at term 4
2017-04-18 11:11:36.217269 I | raft: 3eb1e71170c6eb84 became candidate at term 5
2017-04-18 11:11:36.217277 I | raft: 3eb1e71170c6eb84 received MsgVoteResp from 3eb1e71170c6eb84 at term 5
2017-04-18 11:11:36.217285 I | raft: 3eb1e71170c6eb84 [logterm: 1, index: 3] sent MsgVote request to 9195b0bacc666c6 at term 5
2017-04-18 11:11:36.217292 I | raft: 3eb1e71170c6eb84 [logterm: 1, index: 3] sent MsgVote request to 7d0d073235bdca52 at term 5
2017-04-18 11:11:37.704511 I | raft: 3eb1e71170c6eb84 is starting a new election at term 5
2017-04-18 11:11:37.704558 I | raft: 3eb1e71170c6eb84 became candidate at term 6
2017-04-18 11:11:37.704576 I | raft: 3eb1e71170c6eb84 received MsgVoteResp from 3eb1e71170c6eb84 at term 6
2017-04-18 11:11:37.704590 I | raft: 3eb1e71170c6eb84 [logterm: 1, index: 3] sent MsgVote request to 9195b0bacc666c6 at term 6
2017-04-18 11:11:37.704604 I | raft: 3eb1e71170c6eb84 [logterm: 1, index: 3] sent MsgVote request to 7d0d073235bdca52 at term 6
2017-04-18 11:11:37.903656 I | raft: 9195b0bacc666c6 is starting a new election at term 5
2017-04-18 11:11:37.903684 I | raft: 9195b0bacc666c6 became candidate at term 6
2017-04-18 11:11:37.903691 I | raft: 7d0d073235bdca52 is starting a new election at term 5
2017-04-18 11:11:37.903709 I | raft: 7d0d073235bdca52 became candidate at term 6
2017-04-18 11:11:37.903716 I | raft: 7d0d073235bdca52 received MsgVoteResp from 7d0d073235bdca52 at term 6
2017-04-18 11:11:37.903722 I | raft: 9195b0bacc666c6 received MsgVoteResp from 9195b0bacc666c6 at term 6
2017-04-18 11:11:37.903728 I | raft: 7d0d073235bdca52 [logterm: 1, index: 3] sent MsgVote request to 9195b0bacc666c6 at term 6
2017-04-18 11:11:37.903733 I | raft: 9195b0bacc666c6 [logterm: 1, index: 3] sent MsgVote request to 3eb1e71170c6eb84 at term 6
2017-04-18 11:11:37.903738 I | raft: 7d0d073235bdca52 [logterm: 1, index: 3] sent MsgVote request to 3eb1e71170c6eb84 at term 6
2017-04-18 11:11:37.903742 I | raft: 9195b0bacc666c6 [logterm: 1, index: 3] sent MsgVote request to 7d0d073235bdca52 at term 6
2017-04-18 11:11:38.116169 E | etcdserver: publish error: etcdserver: request timed out
2017-04-18 11:11:38.125666 E | etcdserver: publish error: etcdserver: request timed out
2017-04-18 11:11:38.125689 E | etcdserver: publish error: etcdserver: request timed out
2017-04-18 11:11:38.905155 I | raft: 7d0d073235bdca52 is starting a new election at term 6
2017-04-18 11:11:38.905202 I | raft: 7d0d073235bdca52 became candidate at term 7
2017-04-18 11:11:38.905217 I | raft: 7d0d073235bdca52 received MsgVoteResp from 7d0d073235bdca52 at term 7
2017-04-18 11:11:38.905233 I | raft: 7d0d073235bdca52 [logterm: 1, index: 3] sent MsgVote request to 9195b0bacc666c6 at term 7
2017-04-18 11:11:38.905246 I | raft: 7d0d073235bdca52 [logterm: 1, index: 3] sent MsgVote request to 3eb1e71170c6eb84 at term 7
2017-04-18 11:11:39.002181 I | raft: 3eb1e71170c6eb84 is starting a new election at term 6
2017-04-18 11:11:39.002209 I | raft: 3eb1e71170c6eb84 became candidate at term 7
2017-04-18 11:11:39.002219 I | raft: 3eb1e71170c6eb84 received MsgVoteResp from 3eb1e71170c6eb84 at term 7
2017-04-18 11:11:39.002228 I | raft: 3eb1e71170c6eb84 [logterm: 1, index: 3] sent MsgVote request to 7d0d073235bdca52 at term 7
2017-04-18 11:11:39.002235 I | raft: 3eb1e71170c6eb84 [logterm: 1, index: 3] sent MsgVote request to 9195b0bacc666c6 at term 7
2017-04-18 11:11:39.806776 I | raft: 9195b0bacc666c6 is starting a new election at term 6
2017-04-18 11:11:39.806813 I | raft: 9195b0bacc666c6 became candidate at term 7
2017-04-18 11:11:39.806820 I | raft: 9195b0bacc666c6 received MsgVoteResp from 9195b0bacc666c6 at term 7
2017-04-18 11:11:39.806826 I | raft: 9195b0bacc666c6 [logterm: 1, index: 3] sent MsgVote request to 3eb1e71170c6eb84 at term 7
2017-04-18 11:11:39.806831 I | raft: 9195b0bacc666c6 [logterm: 1, index: 3] sent MsgVote request to 7d0d073235bdca52 at term 7
2017-04-18 11:11:39.902141 I | raft: 7d0d073235bdca52 is starting a new election at term 7
2017-04-18 11:11:39.902164 I | raft: 7d0d073235bdca52 became candidate at term 8
2017-04-18 11:11:39.902173 I | raft: 7d0d073235bdca52 received MsgVoteResp from 7d0d073235bdca52 at term 8
2017-04-18 11:11:39.902181 I | raft: 7d0d073235bdca52 [logterm: 1, index: 3] sent MsgVote request to 9195b0bacc666c6 at term 8
2017-04-18 11:11:39.902189 I | raft: 7d0d073235bdca52 [logterm: 1, index: 3] sent MsgVote request to 3eb1e71170c6eb84 at term 8
2017-04-18 11:11:40.001900 I | raft: 3eb1e71170c6eb84 is starting a new election at term 7
2017-04-18 11:11:40.001932 I | raft: 3eb1e71170c6eb84 became candidate at term 8
2017-04-18 11:11:40.001945 I | raft: 3eb1e71170c6eb84 received MsgVoteResp from 3eb1e71170c6eb84 at term 8
2017-04-18 11:11:40.001956 I | raft: 3eb1e71170c6eb84 [logterm: 1, index: 3] sent MsgVote request to 9195b0bacc666c6 at term 8
2017-04-18 11:11:40.001970 I | raft: 3eb1e71170c6eb84 [logterm: 1, index: 3] sent MsgVote request to 7d0d073235bdca52 at term 8
2017-04-18 11:11:41.112074 W | rafthttp: health check for peer 9195b0bacc666c6 could not connect: EOF
2017-04-18 11:11:41.116266 W | rafthttp: health check for peer 9195b0bacc666c6 could not connect: EOF
2017-04-18 11:11:41.116285 W | rafthttp: health check for peer 7d0d073235bdca52 could not connect: EOF
2017-04-18 11:11:41.126283 W | rafthttp: health check for peer 3eb1e71170c6eb84 could not connect: EOF
2017-04-18 11:11:41.126308 W | rafthttp: health check for peer 7d0d073235bdca52 could not connect: EOF
2017-04-18 11:11:41.126327 W | rafthttp: health check for peer 3eb1e71170c6eb84 could not connect: EOF
2017-04-18 11:11:41.202463 I | raft: 7d0d073235bdca52 is starting a new election at term 8
2017-04-18 11:11:41.202488 I | raft: 7d0d073235bdca52 became candidate at term 9
2017-04-18 11:11:41.202497 I | raft: 7d0d073235bdca52 received MsgVoteResp from 7d0d073235bdca52 at term 9
2017-04-18 11:11:41.202506 I | raft: 7d0d073235bdca52 [logterm: 1, index: 3] sent MsgVote request to 9195b0bacc666c6 at term 9
2017-04-18 11:11:41.202514 I | raft: 7d0d073235bdca52 [logterm: 1, index: 3] sent MsgVote request to 3eb1e71170c6eb84 at term 9
2017-04-18 11:11:41.306952 I | raft: 3eb1e71170c6eb84 is starting a new election at term 8
2017-04-18 11:11:41.306999 I | raft: 3eb1e71170c6eb84 became candidate at term 9
2017-04-18 11:11:41.307016 I | raft: 3eb1e71170c6eb84 received MsgVoteResp from 3eb1e71170c6eb84 at term 9
2017-04-18 11:11:41.307031 I | raft: 3eb1e71170c6eb84 [logterm: 1, index: 3] sent MsgVote request to 9195b0bacc666c6 at term 9
2017-04-18 11:11:41.307045 I | raft: 3eb1e71170c6eb84 [logterm: 1, index: 3] sent MsgVote request to 7d0d073235bdca52 at term 9
2017-04-18 11:11:41.502362 I | raft: 9195b0bacc666c6 is starting a new election at term 7
2017-04-18 11:11:41.502386 I | raft: 9195b0bacc666c6 became candidate at term 8
2017-04-18 11:11:41.502395 I | raft: 9195b0bacc666c6 received MsgVoteResp from 9195b0bacc666c6 at term 8
2017-04-18 11:11:41.502403 I | raft: 9195b0bacc666c6 [logterm: 1, index: 3] sent MsgVote request to 7d0d073235bdca52 at term 8
2017-04-18 11:11:41.502411 I | raft: 9195b0bacc666c6 [logterm: 1, index: 3] sent MsgVote request to 3eb1e71170c6eb84 at term 8
2017-04-18 11:11:42.403855 I | raft: 7d0d073235bdca52 is starting a new election at term 9
2017-04-18 11:11:42.403887 I | raft: 7d0d073235bdca52 became candidate at term 10
2017-04-18 11:11:42.403898 I | raft: 7d0d073235bdca52 received MsgVoteResp from 7d0d073235bdca52 at term 10
2017-04-18 11:11:42.403906 I | raft: 7d0d073235bdca52 [logterm: 1, index: 3] sent MsgVote request to 9195b0bacc666c6 at term 10
2017-04-18 11:11:42.403912 I | raft: 7d0d073235bdca52 [logterm: 1, index: 3] sent MsgVote request to 3eb1e71170c6eb84 at term 10
2017-04-18 11:11:42.606993 I | raft: 3eb1e71170c6eb84 is starting a new election at term 9
2017-04-18 11:11:42.607039 I | raft: 3eb1e71170c6eb84 became candidate at term 10
2017-04-18 11:11:42.607055 I | raft: 3eb1e71170c6eb84 received MsgVoteResp from 3eb1e71170c6eb84 at term 10
2017-04-18 11:11:42.607071 I | raft: 3eb1e71170c6eb84 [logterm: 1, index: 3] sent MsgVote request to 7d0d073235bdca52 at term 10
2017-04-18 11:11:42.607087 I | raft: 3eb1e71170c6eb84 [logterm: 1, index: 3] sent MsgVote request to 9195b0bacc666c6 at term 10
2017-04-18 11:11:42.804550 I | raft: 9195b0bacc666c6 is starting a new election at term 8
2017-04-18 11:11:42.804599 I | raft: 9195b0bacc666c6 became candidate at term 9
2017-04-18 11:11:42.804618 I | raft: 9195b0bacc666c6 received MsgVoteResp from 9195b0bacc666c6 at term 9
2017-04-18 11:11:42.804634 I | raft: 9195b0bacc666c6 [logterm: 1, index: 3] sent MsgVote request to 3eb1e71170c6eb84 at term 9
2017-04-18 11:11:42.804652 I | raft: 9195b0bacc666c6 [logterm: 1, index: 3] sent MsgVote request to 7d0d073235bdca52 at term 9
2017-04-18 11:11:43.803536 I | raft: 7d0d073235bdca52 is starting a new election at term 10
2017-04-18 11:11:43.803584 I | raft: 7d0d073235bdca52 became candidate at term 11
2017-04-18 11:11:43.803602 I | raft: 7d0d073235bdca52 received MsgVoteResp from 7d0d073235bdca52 at term 11
2017-04-18 11:11:43.803618 I | raft: 7d0d073235bdca52 [logterm: 1, index: 3] sent MsgVote request to 9195b0bacc666c6 at term 11
2017-04-18 11:11:43.803630 I | raft: 7d0d073235bdca52 [logterm: 1, index: 3] sent MsgVote request to 3eb1e71170c6eb84 at term 11
2017-04-18 11:11:44.001951 I | raft: 3eb1e71170c6eb84 is starting a new election at term 10
2017-04-18 11:11:44.001992 I | raft: 3eb1e71170c6eb84 became candidate at term 11
2017-04-18 11:11:44.002005 I | raft: 3eb1e71170c6eb84 received MsgVoteResp from 3eb1e71170c6eb84 at term 11
2017-04-18 11:11:44.002017 I | raft: 3eb1e71170c6eb84 [logterm: 1, index: 3] sent MsgVote request to 7d0d073235bdca52 at term 11
2017-04-18 11:11:44.002030 I | raft: 3eb1e71170c6eb84 [logterm: 1, index: 3] sent MsgVote request to 9195b0bacc666c6 at term 11
2017-04-18 11:11:44.206964 I | raft: 9195b0bacc666c6 is starting a new election at term 9
2017-04-18 11:11:44.207013 I | raft: 9195b0bacc666c6 became candidate at term 10
2017-04-18 11:11:44.207036 I | raft: 9195b0bacc666c6 received MsgVoteResp from 9195b0bacc666c6 at term 10
2017-04-18 11:11:44.207052 I | raft: 9195b0bacc666c6 [logterm: 1, index: 3] sent MsgVote request to 3eb1e71170c6eb84 at term 10
2017-04-18 11:11:44.207067 I | raft: 9195b0bacc666c6 [logterm: 1, index: 3] sent MsgVote request to 7d0d073235bdca52 at term 10
2017-04-18 11:11:45.121586 E | etcdserver: publish error: etcdserver: request timed out
2017-04-18 11:11:45.129954 E | etcdserver: publish error: etcdserver: request timed out
2017-04-18 11:11:45.129979 E | etcdserver: publish error: etcdserver: request timed out
@gyuho gyuho self-assigned this Apr 18, 2017
@gyuho
Copy link
Contributor Author

gyuho commented Apr 18, 2017

Should be fixed via etcd-io/etcd#7767.

gyuho added a commit to gyuho/etcd that referenced this issue Apr 19, 2017
@gyuho
embed: signal 'grpcServerC' before server cmux
024f480 
CMux.Serve blocks, so grpcServerC was never closed.

Fix etcd-io/etcdlabs#216.

Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
@gyuho gyuho mentioned this issue Apr 19, 2017
Merged
gyuho added a commit to gyuho/etcd that referenced this issue Apr 19, 2017
@gyuho
embed: signal 'grpcServerC' before cmux serve
ba299bc 
CMux.Serve blocks, so grpcServerC was never closed.

Fix etcd-io/etcdlabs#216.

Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@gyuho gyuho

Labels
Backend Bug
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

embed: signal 'grpcServerC' before cmux serve gyuho/etcd
1 participant
@gyuho