Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

embed: fix go 1.7 http issue #6463

Merged
merged 1 commit into from
Sep 18, 2016
Merged

embed: fix go 1.7 http issue #6463

merged 1 commit into from
Sep 18, 2016

Conversation

xiang90
Copy link
Contributor

@xiang90 xiang90 commented Sep 18, 2016

go 1.7 introduces HTTP2 compatibility issue. Now we
need to explicitly enable HTTP2 when TLS is set.

Fix #6455. @gyuho

@gyuho
Copy link
Contributor

gyuho commented Sep 18, 2016

@xiang90 Confirmed that this fixes the issue. LGTM. Thanks!

@gyuho
Copy link
Contributor

gyuho commented Sep 18, 2016

How about setting h2 NextProto here https://github.com/coreos/etcd/blob/master/pkg/transport/listener.go#L208-L209?

ServerConfig is also used in others tests.

go 1.7 introduces HTTP2 compability issue. Now we
need to explicitly enable HTTP2 when TLS is set.
@xiang90
Copy link
Contributor Author

xiang90 commented Sep 18, 2016

@gyuho Fixed.

Copy link
Contributor

@gyuho gyuho left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks!

@xiang90 xiang90 merged commit 33dbf5c into etcd-io:master Sep 18, 2016
@xiang90 xiang90 deleted the fix_http branch September 18, 2016 13:44
@smarterclayton
Copy link
Contributor

@timothysc / @liggitt this is relevant for kube as well if we turn on H2 with client certs

@timothysc
Copy link

ack.

@smarterclayton
Copy link
Contributor

smarterclayton commented Sep 28, 2016

This was fixed for me on v3.1.0-alpha.0 plus this patch (and this commit) but seems to be broken again in v3.1.0-alpha.1. I'm not 100% sure I'm not making a mistake, but the commit is still there but clients fail to connect.

$ etcd --listen-peer-urls=https://0.0.0.0:7001 --listen-client-urls=https://0.0.0.0:4001  --advertise-client-urls=https://10.1.2.1:4001 --cert-file openshift.local.config/master/etcd.server.crt --key-file openshift.local.config/master/etcd.server.key --peer-cert-file openshift.local.config/master/etcd.server.crt  --peer-key-file openshift.local.config/master/etcd.server.key --initial-advertise-peer-urls https://10.1.2.1:7001 --initial-cluster=default=https://10.1.2.1:7001 &
...
$ etcdctl -v
etcdctl version: 3.1.0-alpha.1
API version: 2
$ ETCDCTL_API=3 etcdctl --endpoints https://10.1.2.1:4001 --cert openshift.local.config/master/master.etcd-client.crt --key openshift.local.config/master/master.etcd-client.key --cacert openshift.local.config/master/ca.crt member list
2016/09/28 19:30:58 transport: http2Client.notifyError got notified that the client transport was broken unexpected EOF.
Error:  context deadline exceeded

@gyuho
Copy link
Contributor

gyuho commented Sep 28, 2016

@smarterclayton Works fine from my side?

rm -rf /tmp/certs && mkdir -p /tmp/certs

rm -f /tmp/etcd-${ETCD_VER}-linux-amd64.tar.gz && rm -rf /tmp/test-etcd && mkdir -p /tmp/test-etcd

curl -L ${DOWNLOAD_URL}/${ETCD_VER}/etcd-${ETCD_VER}-linux-amd64.tar.gz -o /tmp/etcd-${ETCD_VER}-linux-amd64.tar.gz
tar xzvf /tmp/etcd-${ETCD_VER}-linux-amd64.tar.gz -C /tmp/test-etcd --strip-components=1

/tmp/test-etcd/etcd --version

etcd Version: 3.1.0-alpha.1
Git SHA: 2469a95
Go Version: go1.7.1
Go OS/Arch: linux/amd64


echo '{
  "key": {
    "algo": "rsa",
    "size": 4096
  },
  "names": [
    {
      "O": "etcd",
      "OU": "etcd, security team",
      "L": "San Francisco",
      "ST": "California",
      "C": "USA"
    }
  ],
  "CN": "etcd"
}
' > /tmp/certs/trusted-ca-csr.json

cfssl gencert --initca=true /tmp/certs/trusted-ca-csr.json | cfssljson --bare /tmp/certs/trusted-ca


echo '{
  "signing": {
    "default": {
        "usages": [
          "signing",
          "key encipherment",
          "server auth",
          "client auth"
        ],
        "expiry": "87600h"
    }
  }
}
' > /tmp/certs/gencert-config.json


echo '{
  "key": {
    "algo": "rsa",
    "size": 4096
  },
  "names": [
    {
      "O": "etcd",
      "OU": "etcd, security team",
      "L": "San Francisco",
      "ST": "California",
      "C": "USA"
    }
  ],
  "CN": "etcd",
  "hosts": [
    "localhost"
  ]
}
' > /tmp/certs/request-ca-csr-test-name-1.json

cfssl gencert \
    --ca /tmp/certs/trusted-ca.pem \
    --ca-key /tmp/certs/trusted-ca-key.pem \
    --config /tmp/certs/gencert-config.json \
    /tmp/certs/request-ca-csr-test-name-1.json | cfssljson --bare /tmp/certs/test-name-1



/tmp/test-etcd/etcd --name test-name-1 --data-dir /tmp/test-name-1.data \
    --listen-client-urls https://localhost:12379 --advertise-client-urls https://localhost:12379 \
    --listen-peer-urls https://localhost:12380 --initial-advertise-peer-urls https://localhost:12380 \
    --initial-cluster test-name-1=https://localhost:12380 \
    --initial-cluster-token test-token --initial-cluster-state new \
    --client-cert-auth \
    --cert-file /tmp/certs/test-name-1.pem \
    --key-file /tmp/certs/test-name-1-key.pem \
    --trusted-ca-file /tmp/certs/trusted-ca.pem \
    --peer-client-cert-auth \
    --peer-cert-file /tmp/certs/test-name-1.pem \
    --peer-key-file /tmp/certs/test-name-1-key.pem \
    --peer-trusted-ca-file /tmp/certs/trusted-ca.pem


ETCDCTL_API=3 /tmp/test-etcd/etcdctl \
  --endpoints=localhost:12379 \
  --cert=/tmp/certs/test-name-1.pem \
  --key=/tmp/certs/test-name-1-key.pem \
  --cacert=./trusted-ca.pem \
  put foo bar

@smarterclayton
Copy link
Contributor

smarterclayton commented Sep 29, 2016

Couple of differences - I'm using SAN certificates where the listed IP is one of the secondaries, I'm not using pems. But I agree, everything looks like it should work.

EDIT: we also use both IP and DNS to identify 192.168.1.103, because of various annoyances with Safari (or other bad browsers).

@smarterclayton
Copy link
Contributor

Using downloaded etcd v3.1.0-alpha.1 darwin I get:

$ /tmp/etcd3/etcd-v3.1.0-alpha.1-darwin-amd64/etcd --listen-peer-urls=https://0.0.0.0:7001 --listen-client-urls=https://0.0.0.0:4001  --advertise-client-urls=https://192.168.1.103:4001 --cert-file openshift.local.config/master/etcd.server.crt --key-file openshift.local.config/master/etcd.server.key --peer-cert-file openshift.local.config/master/etcd.server.crt  --peer-key-file openshift.local.config/master/etcd.server.key --initial-advertise-peer-urls https://192.168.1.103:7001 --initial-cluster=default=https://192.168.1.103:7001 --peer-client-cert-auth --client-cert-auth
2016-09-28 22:20:48.468378 I | etcdmain: etcd Version: 3.1.0-alpha.1
2016-09-28 22:20:48.468463 I | etcdmain: Git SHA: 2469a95
2016-09-28 22:20:48.468466 I | etcdmain: Go Version: go1.7.1
2016-09-28 22:20:48.468469 I | etcdmain: Go OS/Arch: darwin/amd64
2016-09-28 22:20:48.468472 I | etcdmain: setting maximum number of CPUs to 8, total number of available CPUs is 8
2016-09-28 22:20:48.468479 W | etcdmain: no data-dir provided, using default data-dir ./default.etcd
2016-09-28 22:20:48.468506 I | embed: peerTLS: cert = openshift.local.config/master/etcd.server.crt, key = openshift.local.config/master/etcd.server.key, ca = , trusted-ca = , client-cert-auth = true
2016-09-28 22:20:48.469081 I | embed: listening for peers on https://0.0.0.0:7001
2016-09-28 22:20:48.469119 I | embed: listening for client requests on 0.0.0.0:4001
2016-09-28 22:20:48.470877 I | etcdserver: name = default
2016-09-28 22:20:48.470886 I | etcdserver: data dir = default.etcd
2016-09-28 22:20:48.470892 I | etcdserver: member dir = default.etcd/member
2016-09-28 22:20:48.470895 I | etcdserver: heartbeat = 100ms
2016-09-28 22:20:48.470898 I | etcdserver: election = 1000ms
2016-09-28 22:20:48.470900 I | etcdserver: snapshot count = 10000
2016-09-28 22:20:48.470905 I | etcdserver: advertise client URLs = https://192.168.1.103:4001
2016-09-28 22:20:48.470910 I | etcdserver: initial advertise peer URLs = https://192.168.1.103:7001
2016-09-28 22:20:48.470915 I | etcdserver: initial cluster = default=https://192.168.1.103:7001
2016-09-28 22:20:48.620091 I | etcdserver: starting member 3092679e8c56a1a5 in cluster e989df3141e943e1
2016-09-28 22:20:48.620127 I | raft: 3092679e8c56a1a5 became follower at term 0
2016-09-28 22:20:48.620142 I | raft: newRaft 3092679e8c56a1a5 [peers: [], term: 0, commit: 0, applied: 0, lastindex: 0, lastterm: 0]
2016-09-28 22:20:48.620146 I | raft: 3092679e8c56a1a5 became follower at term 1
2016-09-28 22:20:48.631194 I | etcdserver: starting server... [version: 3.1.0-alpha.1, cluster version: to_be_decided]
2016-09-28 22:20:48.631227 I | embed: ClientTLS: cert = openshift.local.config/master/etcd.server.crt, key = openshift.local.config/master/etcd.server.key, ca = , trusted-ca = , client-cert-auth = true
2016-09-28 22:20:48.631311 E | etcdserver: cannot monitor file descriptor usage (cannot get FDUsage on darwin)
2016-09-28 22:20:48.631680 I | membership: added member 3092679e8c56a1a5 [https://192.168.1.103:7001] to cluster e989df3141e943e1
2016-09-28 22:20:49.629200 I | raft: 3092679e8c56a1a5 is starting a new election at term 1
2016-09-28 22:20:49.629255 I | raft: 3092679e8c56a1a5 became candidate at term 2
2016-09-28 22:20:49.629261 I | raft: 3092679e8c56a1a5 received vote from 3092679e8c56a1a5 at term 2
2016-09-28 22:20:49.629271 I | raft: 3092679e8c56a1a5 became leader at term 2
2016-09-28 22:20:49.629276 I | raft: raft.node: 3092679e8c56a1a5 elected leader 3092679e8c56a1a5 at term 2
2016-09-28 22:20:49.629397 I | etcdserver: setting up the initial cluster version to 3.1
2016-09-28 22:20:49.629497 I | etcdserver: published {Name:default ClientURLs:[https://192.168.1.103:4001]} to cluster e989df3141e943e1
2016-09-28 22:20:49.629513 I | embed: ready to serve client requests
2016-09-28 22:20:49.629817 I | embed: serving client requests on [::]:4001
2016-09-28 22:20:49.632464 N | membership: set the initial cluster version to 3.1
2016-09-28 22:20:49.632495 I | api: enabled capabilities for version 3.1
2016/09/28 22:20:49 Failed to dial [::]:4001: connection error: desc = "transport: remote error: tls: bad certificate"; please retry.
2016/09/28 22:20:49 Failed to dial [::]:4001: connection error: desc = "transport: remote error: tls: bad certificate"; please retry.
2016/09/28 22:20:49 Failed to dial [::]:4001: connection error: desc = "transport: remote error: tls: bad certificate"; please retry.
2016/09/28 22:20:49 Failed to dial [::]:4001: connection error: desc = "transport: remote error: tls: bad certificate"; please retry.
2016/09/28 22:20:49 Failed to dial [::]:4001: connection error: desc = "transport: remote error: tls: bad certificate"; please retry.
2016/09/28 22:20:49 Failed to dial [::]:4001: connection error: desc = "transport: remote error: tls: bad certificate"; please retry.
$ ETCDCTL_API=3 /tmp/etcd3/etcd-v3.1.0-alpha.1-darwin-amd64/etcdctl --endpoints https://192.168.1.103:4001 --cert openshift.local.config/master/master.etcd-client.crt --key openshift.local.config/master/master.etcd-client.key --cacert openshift.local.config/master/ca.crt member list
2016/09/28 22:22:30 transport: http2Client.notifyError got notified that the client transport was broken unexpected EOF.

Here's the cert we have generated:

openssl x509 -in openshift.local.config/master/etcd.server.crt -text
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 6 (0x6)
        Signature Algorithm: sha256WithRSAEncryption
        Issuer: CN=openshift-signer@1475114271
        Validity
            Not Before: Sep 29 01:57:51 2016 GMT
            Not After : Sep 29 01:57:52 2018 GMT
        Subject: CN=10.1.2.1
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
            RSA Public Key: (2048 bit)
                Modulus (2048 bit):
                    00:db:8c:50:91:a0:9c:d2:f9:48:6e:4f:75:bb:01:
                    4b:17:a4:1e:06:8d:ab:d0:34:09:53:b0:78:26:1b:
                    cc:b3:93:1e:4b:ae:72:e7:c9:4f:3d:42:39:63:20:
                    0b:7f:86:48:65:68:a1:55:0d:de:2c:80:70:c7:82:
                    47:4d:95:43:01:7f:df:83:1a:66:2b:a1:23:06:f9:
                    94:0b:2b:18:56:ac:ad:9a:e6:50:44:f7:0d:bb:40:
                    1b:26:98:4f:82:0c:41:19:c4:46:96:b6:86:d6:a5:
                    3e:f7:81:0a:7e:e0:c1:e0:7f:ba:3a:fb:54:89:32:
                    d5:4b:be:a6:88:97:45:d4:5f:b9:ed:7d:0a:fd:e9:
                    cf:04:fc:7d:fc:8b:30:98:0a:97:05:68:7a:77:8c:
                    0c:04:f6:e7:3b:52:6d:57:32:e3:7b:f4:21:9c:9f:
                    14:70:7f:51:a8:31:cf:d1:57:81:ca:7a:d8:8b:98:
                    a0:79:53:a0:af:d2:8d:6d:15:a3:1f:d6:d5:7d:03:
                    b3:07:0f:ce:52:cf:31:e4:c5:bf:9c:21:5c:c9:1a:
                    0f:27:70:8e:1d:5c:8e:11:8e:92:00:15:d4:c7:f2:
                    f5:35:1e:1d:a4:de:b9:23:45:4f:90:68:18:48:e4:
                    32:6d:8b:57:fe:11:bb:91:1b:dd:1b:39:fc:fa:6a:
                    1f:b5
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            X509v3 Key Usage: critical
                Digital Signature, Key Encipherment
            X509v3 Extended Key Usage:
                TLS Web Server Authentication
            X509v3 Basic Constraints: critical
                CA:FALSE
            X509v3 Subject Alternative Name:
                DNS:kubernetes, DNS:kubernetes.default, DNS:kubernetes.default.svc, DNS:kubernetes.default.svc.cluster.local, DNS:localhost, DNS:openshift, DNS:openshift.default, DNS:openshift.default.svc, DNS:openshift.default.svc.cluster.local, DNS:10.1.2.1, DNS:127.0.0.1, DNS:172.30.0.1, DNS:192.168.1.103, IP Address:10.1.2.1, IP Address:127.0.0.1, IP Address:172.30.0.1, IP Address:192.168.1.103

This is a self-signed generated cert (for simplifying setup).

@gyuho
Copy link
Contributor

gyuho commented Sep 29, 2016

@smarterclayton Could you create a new issue for that? Thanks!

@smarterclayton
Copy link
Contributor

Opened #6565, thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

4 participants