etcd cpu high in my test machine

[werbenhu]

Issue description

I just finished setting up the test environment, and then established two interfaces, and saw that the CPU of etcd is a bit high

here is my env:

docker-compose.yml

version: "3"

services:
  apisix:
    image: "apache/apisix:2.12.1-alpine"
    restart: always
    container_name: apisix
    volumes:
      - ./log:/usr/local/apisix/logs
      - ./conf/apisix/conf.yaml:/usr/local/apisix/conf/config.yaml:ro
    depends_on:
      - etcd
    ports:
      - "80:80/tcp"
      - "9091:9091/tcp"
      - "443:9443/tcp"
    networks:
      - default

  dashboard:
    image: "apache/apisix-dashboard:2.9.0"
    restart: always
    container_name: apisix-dashboard
    volumes:
      - ./conf/dashboard/conf.yaml:/usr/local/apisix-dashboard/conf/conf.yaml:ro
    depends_on:
      - etcd
    ports:
      - "9000:9000/tcp"
    networks:
      - default

  etcd:
    image: bitnami/etcd:3.4.9
    user: root
    container_name: etcd
    restart: always
    volumes:
      - ./data/etcd:/etcd_data
    environment:
      ETCD_DATA_DIR: /etcd_data
      ETCD_ENABLE_V2: "true"
      ALLOW_NONE_AUTHENTICATION: "yes"
      ETCD_ADVERTISE_CLIENT_URLS: "http://0.0.0.0:2379"
      ETCD_LISTEN_CLIENT_URLS: "http://0.0.0.0:2379"
    ports:
      - "2379:2379/tcp"
    networks:
      - xxx

networks:
  default:
    external:
      name: xxx

conf/apisix/conf.yaml

apisix:
  node_listen: 80              # APISIX listening port
  enable_ipv6: false

  allow_admin:                  # http://nginx.org/en/docs/http/ngx_http_access_module.html#allow
    - 0.0.0.0/0              # We need to restrict ip access rules for security. 0.0.0.0/0 is for test.

  admin_key:
    - name: "admin"
      key: edd1c9f034335f136f87ad84b625c8f1
      role: admin                 # admin: manage all configuration data
                                  # viewer: only can view configuration data
    - name: "viewer"
      key: 4054f7cf07e344346cd3f287985e76a2
      role: viewer
  
  enable_control: true
  control:
    ip: "0.0.0.0"
    port: 9092

etcd:
  host:                           # it's possible to define multiple etcd hosts addresses of the same etcd cluster.
    - "http://etcd:2379"     # multiple etcd address
  prefix: "/apisix"               # apisix configurations prefix
  timeout: 30                     # 30 seconds

plugin_attr:
  prometheus:
    export_addr:
      ip: "0.0.0.0"
      port: 9091

conf/dashboard/conf.yaml

conf:
  listen:
    host: 0.0.0.0     # `manager api` listening ip or host name
    port: 9000          # `manager api` listening port
  allow_list:           # If we don't set any IP list, then any IP access is allowed by default.
    - 0.0.0.0/0
  etcd:
    endpoints:          # supports defining multiple etcd host addresses for an etcd cluster
      - "http://etcd:2379"
                          # yamllint disable rule:comments-indentation
                          # etcd basic auth info
    # username: "root"    # ignore etcd username if not enable etcd auth
    # password: "123456"  # ignore etcd password if not enable etcd auth
    mtls:
      key_file: ""          # Path of your self-signed client side key
      cert_file: ""         # Path of your self-signed client side cert
      ca_file: ""           # Path of your self-signed ca cert, the CA is used to sign callers' certificates
    # prefix: /apisix     # apisix config's prefix in etcd, /apisix by default
  log:
    error_log:
      level: warn       # supports levels, lower to higher: debug, info, warn, error, panic, fatal
      file_path:
        logs/error.log  # supports relative path, absolute path, standard output
                        # such as: logs/error.log, /tmp/logs/error.log, /dev/stdout, /dev/stderr
    access_log:
      file_path:
        logs/access.log  # supports relative path, absolute path, standard output
                         # such as: logs/access.log, /tmp/logs/access.log, /dev/stdout, /dev/stderr
                         # log example: 2020-12-09T16:38:09.039+0800	INFO	filter/logging.go:46	/apisix/admin/routes/r1	{"status": 401, "host": "127.0.0.1:9000", "query": "asdfsafd=adf&a=a", "requestId": "3d50ecb8-758c-46d1-af5b-cd9d1c820156", "latency": 0, "remoteIP": "127.0.0.1", "method": "PUT", "errs": []}
authentication:
  secret:
    secret              # secret for jwt token generation.
                        # NOTE: Highly recommended to modify this value to protect `manager api`.
                        # if it's default value, when `manager api` start, it will generate a random string to replace it.
  expire_time: 3600     # jwt token expire time, in second
  users:                # yamllint enable rule:comments-indentation
    - username: admin   # username and password for login `manager api`
      password: admin
    - username: user
      password: user

plugins:                          # plugin list (sorted in alphabetical order)
  - api-breaker
  - authz-keycloak
  - basic-auth
  - batch-requests
  - consumer-restriction
  - cors
  # - dubbo-proxy
  - echo
  # - error-log-logger
  # - example-plugin
  - fault-injection
  - grpc-transcode
  - hmac-auth
  - http-logger
  - ip-restriction
  - jwt-auth
  - kafka-logger
  - key-auth
  - limit-conn
  - limit-count
  - limit-req
  # - log-rotate
  # - node-status
  - openid-connect
  - prometheus
  - proxy-cache
  - proxy-mirror
  - proxy-rewrite
  - redirect
  - referer-restriction
  - request-id
  - request-validation
  - response-rewrite
  - serverless-post-function
  - serverless-pre-function
  # - skywalking
  - sls-logger
  - syslog
  - tcp-logger
  - udp-logger
  - uri-blocker
  - wolf-rbac
  - zipkin
  - server-info
  - traffic-split

Environment

• apisix version (cmd: apisix version):apache/apisix:2.12.1-alpine
• OS (cmd: uname -a): centos7.6

other Environment you can find in the issue description.

Steps to reproduce

1. start server by docker-compose up -d
2. add two test interface api
3. i used limit-count plugin in one of the api router

Actual result

CPU often reaches 25%

Error log

none

Expected result

No response

[leslie-tsang]

Hello there, Please check PR 5723 and PR 6124 out, Do let me know if you need any help. :)

[werbenhu]

i turned off the server-info plugin in dashboard conf and restart server , CPU stays low most of the time now, but it can still reach 8-10% for a few moments.

[werbenhu]

When the CPU is high, there are many processes with the name “openresty”, There are no requests to my machine now. I'm not sure if this is the normal situation.

[leslie-tsang]

Hello there, can you try #6202, this PR fix the high CPU usage issue.

[tokers]

• server-info

These processes are the APISIX worker processes, it's normal.

[leslie-tsang]

Considered resolved due to lack of feedback. Feel free to reopen it if needed.

[werbenhu]

hi, i figure out it in #6804