Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Evict leader does't work as expected #599

Closed
weekface opened this issue Jun 21, 2019 · 1 comment · Fixed by #626
Closed

Evict leader does't work as expected #599

weekface opened this issue Jun 21, 2019 · 1 comment · Fixed by #626
Assignees
Labels
test/stability stability tests type/bug Something isn't working

Comments

@weekface
Copy link
Contributor

    {
      "store": {
        "id": 1,
        "address": "e2e-cluster1-tikv-0.e2e-cluster1-tikv-peer.e2e-cluster1.svc:20160",
        "labels": [
          {
            "key": "host",
            "value": "kube-node-1"
          },
          {
            "key": "rack",
            "value": "rack1"
          }
        ],
        "version": "3.0.0-rc.1",
        "state_name": "Up"
      },
      "status": {
        "capacity": "252 GiB",
        "available": "119 GiB",
        "leader_weight": 1,
        "region_count": 19,
        "region_weight": 1,
        "region_score": 19,
        "region_size": 19,
        "start_ts": "2019-06-21T08:45:53Z",
        "last_heartbeat_ts": "2019-06-21T10:29:44.570183932Z",
        "uptime": "1h43m51.570183932s"
      }
    }
  ]
}
[jenkins@k8s-e2e-test ~]$ curl 127.0.0.1:2379/pd/api/v1/schedulers
Handling connection for 2379
[
  "evict-leader-scheduler-4",
  "evict-leader-scheduler-1",
  "balance-region-scheduler",
  "balance-leader-scheduler",
  "balance-hot-region-scheduler",
  "label-scheduler",
  "evict-leader-scheduler-5"
]

There three evict-leader-scheduler-* all exist in the schedulers api. This is a bug.

@weekface weekface added test/stability stability tests type/bug Something isn't working labels Jun 21, 2019
@weekface
Copy link
Contributor Author

failed again:

I0621 19:13:39.247192       1 pods.go:152] savely delete pod namespace ns1 name cluster1-pd-2 leader name cluster1-pd-0
I0621 19:13:40.847572       1 actions.go:669] checking tidb cluster [ns1/cluster1] status
I0621 19:13:56.573789       1 pods.go:152] savely delete pod namespace ns1 name cluster1-pd-1 leader name cluster1-pd-0
I0621 19:14:22.732340       1 pods.go:152] savely delete pod namespace ns1 name cluster1-pd-0 leader name cluster1-pd-2
E0621 19:14:45.961808       1 actions.go:1475] check [ns1/cluster1] TiDB instance [0] configuration updated failed: desired [1024], actual [1000] not equal
E0621 19:15:45.952541       1 actions.go:1475] check [ns1/cluster1] TiDB instance [0] configuration updated failed: desired [1024], actual [1000] not equal
E0621 19:16:40.704683       1 pods.go:188] failed to evict leader from ns1/cluster1-tikv-2, before: 1, now: 29
E0621 19:16:45.978560       1 actions.go:1475] check [ns1/cluster1] TiDB instance [0] configuration updated failed: desired [1024], actual [1000] not equal
    {
      "store": {
        "id": 5,
        "address": "cluster1-tikv-2.cluster1-tikv-peer.ns1.svc:20160",
        "labels": [
          {
            "key": "host",
            "value": "172.16.4.149"
          },
          {
            "key": "rack",
            "value": "rack0"
          }
        ],
        "version": "3.0.0-rc.2",
        "state_name": "Up"
      },
      "status": {
        "capacity": "1.4 TiB",
        "available": "1.4 TiB",
        "leader_count": 9,
        "leader_weight": 1,
        "leader_score": 292,
        "leader_size": 292,
        "region_count": 29,
        "region_weight": 1,
        "region_score": 346,
        "region_size": 346,
        "start_ts": "2019-06-21T19:17:46Z",
        "last_heartbeat_ts": "2019-06-22T00:41:28.398805448Z",
        "uptime": "5h23m42.398805448s"
      }
    }
  ]
}
[root@172.16.4.149 ~]# curl 10.233.29.214:2379/pd/api/v1/schedulers
[
  "label-scheduler",
  "balance-region-scheduler",
  "balance-leader-scheduler",
  "balance-hot-region-scheduler"
]

@xiaojingchen @shuijing198799 PTAL

yahonda pushed a commit that referenced this issue Dec 27, 2021
* en: replace Mydumper with Dumpling

Signed-off-by: Ran <huangran@pingcap.com>

* address comments

Signed-off-by: Ran <huangran@pingcap.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
test/stability stability tests type/bug Something isn't working
Projects
None yet
2 participants