-
Notifications
You must be signed in to change notification settings - Fork 9.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Etcd unexpected request time consumption with HTTPs v3 API #14077
Comments
same as #14065 |
This issue should can be fixed by pull/13985. The |
Thanks a lot for your tips. We will reply you after we test your PR. ^_^ |
Is this issue fixed? can we close it? |
The related PR isn't merged yet. It seems that the contributor is still struggling to work on the PR. I may spend some time to get it resolved sometime this or next week. |
What happened?
We use a API gateway APISIX to watch etcd cluster(
auth
andTLS
enabled) resource with HTTPs request, the APISIX cluster will raise about 260 connection to etcd cluster.In this scenario, the etcd operation with
HTTPs
request will speed too much time (more than 20s). the operation withgRPC
protocol wouldn't spend that much time.Simple conclusion
Etcd operation with
RESTful
request with TLS will spend too much time (more than 10s).Etcd operation with
gRPC
request performing normal (within 0.5s).Etcd use
grpc-gateway
to convertHTTP
request togRPC
request, Once theauth
is enabled, the perf of etcd cluster becomes lower, and then when TLS is enabled, the perf of the etcd cluster becomes even worse.We had a try about etcd cluster
without TLS
, then it perf as normal one again.What did you expect to happen?
With
TLS enabled
scenarios, etcd operation within1s
.How can we reproduce it (as minimally and precisely as possible)?
auth
andTLS
enabled etcd cluster env can launch with apisix-etcd-mtlsAnything else we need to know?
No response
Etcd version (please run commands below)
Etcd configuration (command line flags or environment variables)
ETCD_CLIENT_KEY_FILE=/opt/etcd/ssl/client-key.pem
HOSTNAME=bc18fac70256
ETCD_LISTEN_CLIENT_URLS=https://0.0.0.0:2379
PWD=/opt/bitnami/etcd
OS_FLAVOUR=debian-10
ETCD_ENABLE_V2=true
ETCD_LISTEN_PEER_URLS=https://0.0.0.0:2380
ETCD_CLIENT_CERT_FILE=/opt/etcd/ssl/client.pem
HOME=/
ETCD_PEER_TRUSTED_CA_FILE=/opt/etcd/ssl/ca.pem
ETCD_INITIAL_CLUSTER_TOKEN=etcd-cluster
ETCD_ENABLE_GRPC_GATEWAY=true
ETCD_NAME=etcd1
ETCD_PEER_CLIENT_CERT_AUTH=true
ETCD_ROOT_PASSWORD=123456
TERM=xterm
ETCD_KEY_FILE=/opt/etcd/ssl/server-key.pem
ETCD_PEER_KEY_FILE=/opt/etcd/ssl/server-key.pem
SHLVL=1
BITNAMI_APP_NAME=etcd
APP_VERSION=3.5.4
ETCD_PEER_CERT_FILE=/opt/etcd/ssl/server.pem
ETCD_INITIAL_CLUSTER_STATE=new
ETCD_INITIAL_CLUSTER=etcd1=https://etcd1:2380,etcd2=https://etcd2:2380,etcd3=https://etcd3:2380
OS_NAME=linux
PATH=/opt/bitnami/common/bin:/opt/bitnami/etcd/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
ETCD_ADVERTISE_CLIENT_URLS=https://0.0.0.0:2379
ETCD_CERT_FILE=/opt/etcd/ssl/server.pem
ETCD_INITIAL_ADVERTISE_PEER_URLS=https://etcd1:2380
OS_ARCH=amd64
_=/usr/bin/env
Etcd debug information (please run commands blow, feel free to obfuscate the IP address or FQDN in the output)
Most elapse spend in
send
phaseRelevant log output
No response
The text was updated successfully, but these errors were encountered: