Unable to Connect via etcdctl with auto-tls or self-signed Certificates

[bhujangr]

Server Version

etcd Version: 3.2.3
Git SHA: ae23b0e
Go Version: go1.8.3
Go OS/Arch: darwin/amd64

etcdctl version

VERSION: 3.2.3
API VERSION: 3.2

Server

$ ./etcd --name node1 --auto-tls=true --advertise-client-urls https://127.0.0.1:2379 --listen-client-urls https://127.0.0.1:2379 --initial-cluster-state new --initial-cluster-token test --client-cert-auth=false

etcdctl

$ ./etcdctl --endpoints=127.0.0.1:2379 endpoint status

On the above etcdctl call, eventually I get an error of the form:
Failed to get the status of endpoint 127.0.0.1:2379 (context deadline exceeded)

Turning on debug, with the etcdctl call, results in about a dozen lines of the form:
grpc: transport: http2Client.notifyError got notified that the client transport was broken unexpected EOF.

On the server (I don't think anything is abnormal):

2017-07-16 21:21:12.841852 I | etcdmain: etcd Version: 3.2.3
2017-07-16 21:21:12.841948 I | etcdmain: Git SHA: ae23b0e
2017-07-16 21:21:12.841951 I | etcdmain: Go Version: go1.8.3
2017-07-16 21:21:12.841954 I | etcdmain: Go OS/Arch: darwin/amd64
2017-07-16 21:21:12.841957 I | etcdmain: setting maximum number of CPUs to 8, total number of available CPUs is 8
2017-07-16 21:21:12.841963 N | etcdmain: failed to detect default host (default host not supported on darwin_amd64)
2017-07-16 21:21:12.841967 W | etcdmain: no data-dir provided, using default data-dir ./node1.etcd
2017-07-16 21:21:12.842247 I | embed: listening for peers on http://localhost:2380
2017-07-16 21:21:12.861826 I | embed: listening for client requests on 127.0.0.1:2379
2017-07-16 21:21:12.862449 I | etcdserver: name = node1
2017-07-16 21:21:12.862457 I | etcdserver: data dir = node1.etcd
2017-07-16 21:21:12.862461 I | etcdserver: member dir = node1.etcd/member
2017-07-16 21:21:12.862463 I | etcdserver: heartbeat = 100ms
2017-07-16 21:21:12.862465 I | etcdserver: election = 1000ms
2017-07-16 21:21:12.862468 I | etcdserver: snapshot count = 100000
2017-07-16 21:21:12.862473 I | etcdserver: advertise client URLs = https://127.0.0.1:2379
2017-07-16 21:21:12.862476 I | etcdserver: initial advertise peer URLs = http://localhost:2380
2017-07-16 21:21:12.862481 I | etcdserver: initial cluster = node1=http://localhost:2380
2017-07-16 21:21:12.962244 I | etcdserver: starting member 78292c913dfe7adb in cluster 3f3fa93bdae9fb7
2017-07-16 21:21:12.962305 I | raft: 78292c913dfe7adb became follower at term 0
2017-07-16 21:21:12.962325 I | raft: newRaft 78292c913dfe7adb [peers: [], term: 0, commit: 0, applied: 0, lastindex: 0, lastterm: 0]
2017-07-16 21:21:12.962332 I | raft: 78292c913dfe7adb became follower at term 1
2017-07-16 21:21:12.964722 W | auth: simple token is not cryptographically signed
2017-07-16 21:21:12.966114 I | etcdserver: starting server... [version: 3.2.3, cluster version: to_be_decided]
2017-07-16 21:21:12.966147 I | embed: ClientTLS: cert = node1.etcd/fixtures/client/cert.pem, key = node1.etcd/fixtures/client/key.pem, ca = , trusted-ca = , client-cert-auth = false
2017-07-16 21:21:12.966224 E | etcdserver: cannot monitor file descriptor usage (cannot get FDUsage on darwin)
2017-07-16 21:21:12.966487 I | etcdserver/membership: added member 78292c913dfe7adb [http://localhost:2380] to cluster 3f3fa93bdae9fb7
2017-07-16 21:21:13.264590 I | raft: 78292c913dfe7adb is starting a new election at term 1
2017-07-16 21:21:13.264633 I | raft: 78292c913dfe7adb became candidate at term 2
2017-07-16 21:21:13.264659 I | raft: 78292c913dfe7adb received MsgVoteResp from 78292c913dfe7adb at term 2
2017-07-16 21:21:13.264676 I | raft: 78292c913dfe7adb became leader at term 2
2017-07-16 21:21:13.264685 I | raft: raft.node: 78292c913dfe7adb elected leader 78292c913dfe7adb at term 2
2017-07-16 21:21:13.264841 I | etcdserver: setting up the initial cluster version to 3.2
2017-07-16 21:21:13.270229 N | etcdserver/membership: set the initial cluster version to 3.2
2017-07-16 21:21:13.270292 I | etcdserver: published {Name:node1 ClientURLs:[https://127.0.0.1:2379]} to cluster 3f3fa93bdae9fb7
2017-07-16 21:21:13.270311 I | etcdserver/api: enabled capabilities for version 3.2
2017-07-16 21:21:13.270326 I | embed: ready to serve client requests
2017-07-16 21:21:13.270613 I | embed: serving client requests on 127.0.0.1:2379

The binaries were downloaded from the official 3.2.3 release.

I did review #6951 as the error message seems to be the same. Tried generating certificates using both openssl as well as cfssl and see the exact same problem. The official docker image too result in the same issue.

The same thing works perfectly fine without TLS.

Not sure if am doing something wrong, any pointers would be highly appreciated.

[gyuho]

2017-07-16 21:21:12.966147 I | embed: ClientTLS: cert = node1.etcd/fixtures/client/cert.pem, key = node1.etcd/fixtures/client/key.pem, ca = , trusted-ca = , client-cert-auth = false

You still need specify those auto-generated certs in your etcdctl commands.

https://github.com/coreos/etcd/tree/master/etcdctl
https://github.com/coreos/etcd/blob/master/Documentation/op-guide/security.md