diff --git a/Makefile b/Makefile index c43770ce8d4c..afa834416263 100644 --- a/Makefile +++ b/Makefile @@ -39,14 +39,14 @@ build: pd-server pd-ctl pd-tso-bench pd-recover pd-server: export GO111MODULE=on pd-server: ifeq ("$(WITH_RACE)", "1") - CGO_ENABLED=1 go build -race -ldflags '$(LDFLAGS)' -o bin/pd-server cmd/pd-server/main.go + CGO_ENABLED=1 go build -race -gcflags '$(GCFLAGS)' -ldflags '$(LDFLAGS)' -o bin/pd-server cmd/pd-server/main.go else - CGO_ENABLED=0 go build -ldflags '$(LDFLAGS)' -o bin/pd-server cmd/pd-server/main.go + CGO_ENABLED=0 go build -gcflags '$(GCFLAGS)' -ldflags '$(LDFLAGS)' -o bin/pd-server cmd/pd-server/main.go endif pd-ctl: export GO111MODULE=on pd-ctl: - CGO_ENABLED=0 go build -ldflags '$(LDFLAGS)' -o bin/pd-ctl tools/pd-ctl/main.go + CGO_ENABLED=0 go build -gcflags '$(GCFLAGS)' -ldflags '$(LDFLAGS)' -o bin/pd-ctl tools/pd-ctl/main.go pd-tso-bench: export GO111MODULE=on pd-tso-bench: CGO_ENABLED=0 go build -o bin/pd-tso-bench tools/pd-tso-bench/main.go diff --git a/conf/config.toml b/conf/config.toml index 99ac0065da17..3b3289eb61cd 100644 --- a/conf/config.toml +++ b/conf/config.toml @@ -67,8 +67,7 @@ leader-schedule-limit = 4 region-schedule-limit = 64 replica-schedule-limit = 64 merge-schedule-limit = 8 -#tolerant-size-ratio = 5.0 -# Enable two-way merge, set it to true may help improving merge speed. +#tolerant-size-ratio = 0.0 #enable-one-way-merge = false # customized schedulers, the format is as below diff --git a/docs/api.html b/docs/api.html index 46f0332c653e..8c3df3a9d11d 100644 --- a/docs/api.html +++ b/docs/api.html @@ -166,4 +166,4 @@ .resource-modal li > ul { margin-bottom: 1em; } -

Placement Driver API API documentation version v1

http://{pdAddr}/pd/api/{version}

  • pdAddr: required (string)

    The PD server address, formatted as 'host:port'.

  • version: required (v1)

/cluster/status

Cluster status.

get

get

Get cluster status.

/version

The version of PD server.

get

get

Get the version of PD server.

/status

The build info of PD server.

get

get

Get the build info of PD server.

/diagnose

Diagnostic information of the cluster.

get

get

/members

The PD servers in the cluster.

get

get

List all PD servers in the cluster.

delete post

A specific PD server.

delete

Remove a PD server from the cluster.

post

Set leader priority of a PD member.

Remove a PD server from the cluster.

URI Parameters

  • name: required (string)

HTTP status code 200

The PD server is successfully removed.

HTTP status code 400

The input is invalid.

HTTP status code 404

The member does not exist.

HTTP status code 500

PD server failed to proceed the request.

Set leader priority of a PD member.

URI Parameters

  • name: required (string)

Body

Media type: application/json

Type: object

Properties
  • leader-priority: required (integer)

HTTP status code 200

The leader priority is updated.

HTTP status code 400

The input is invalid.

HTTP status code 404

The member does not exist.

HTTP status code 500

PD server failed to proceed the request.

delete

A specific PD server.

delete

Remove a PD server from the cluster.

Remove a PD server from the cluster.

URI Parameters

  • id: required (integer)

HTTP status code 200

The PD server is successfully removed.

HTTP status code 400

The input is invalid.

HTTP status code 500

PD server failed to proceed the request.

/leader

The leader PD server of the cluster.

get

get

Get the leader PD server of the cluster.

post

post

Transfer leadership to another PD server.

Transfer leadership to another PD server.

HTTP status code 200

The transfer command is submitted.

HTTP status code 500

PD server failed to proceed the request.

post

post

Transfer leadership to the specific PD server.

Transfer leadership to the specific PD server.

URI Parameters

  • nextLeader: required (string)

HTTP status code 200

The transfer command is submitted.

HTTP status code 500

PD server failed to proceed the request.

/health

Health status of PD servers.

get

get

/config

PD cluster configuration.

get post

get

Get full config.

post

Update a config item.

Update a config item.

Body

Media type: application/json

Type: object

HTTP status code 200

The config is updated.

HTTP status code 500

PD server failed to proceed the request.

get post

Schedule configuration.

get

Get schedule config.

post

Update a schedule config item.

Update a schedule config item.

Body

Media type: application/json

Type: object

HTTP status code 200

The config is updated.

HTTP status code 400

The input is invalid.

HTTP status code 500

PD server failed to proceed the request.

get post

Replication configuration.

get

Get replication config.

post

Update a replication config item.

Update a replication config item.

Body

Media type: application/json

Type: object

HTTP status code 200

The config is updated.

HTTP status code 400

The input is invalid.

HTTP status code 500

PD server failed to proceed the request.

get post delete

The config of a namespace.

get

Get configuration of a namespace.

post

Update a namespace config item.

delete

Delete a namespace config.

Get configuration of a namespace.

URI Parameters

  • namespaceName: required (string)

    The name of the namespace.

HTTP status code 200

Body

Media type: application/json

Type: object

Properties
  • leader-schedule-limit: required (integer)
  • region-schedule-limit: required (integer)
  • replica-schedule-limit: required (integer)
  • merge-schedule-limit: required (integer)
  • max-replicas: required (integer)

HTTP status code 404

The namespace does not exist.

Update a namespace config item.

URI Parameters

  • namespaceName: required (string)

    The name of the namespace.

Body

Media type: application/json

Type: object

HTTP status code 200

The config is updated.

HTTP status code 400

The input is invalid.

HTTP status code 404

The namespace does not exist.

Delete a namespace config.

URI Parameters

  • namespaceName: required (string)

    The name of the namespace.

HTTP status code 200

The config is removed.

HTTP status code 404

The namespace does not exist.

get post

The label property configuration.

get

Get label property config.

post

Update label property config item.

Update label property config item.

Body

Media type: application/json

Type: object

Properties
  • action: required (one of set, delete)
  • type: required (reject-leader)
  • label-key: required (string)
  • label-value: required (string)

HTTP status code 200

The config is updated.

HTTP status code 500

PD server failed to proceed the request.

/stores

The stores in the cluster.

get

get

Get stores in the cluster.

/store/{storeId}

A specific store.

get delete

get

Get a store's information.

delete

Take down a store from the cluster.

Get a store's information.

URI Parameters

  • storeId: required (integer)

HTTP status code 200

Body

Media type: application/json

Type: object

Properties
  • store: required (object)
    • id: required (integer)
    • address: required (string)
    • state: required (one of 0, 1, 2)
    • state_name: required (one of Up, Disconnected, Down, Offline, Tombstone)
    • labels: (array of StoreLabel)

      Items: StoreLabel

      • key: required (string)
      • value: required (string)
    • version: (string)
  • status: required (object)
    • capacity: required (string)
    • available: required (string)
    • leader_count: (integer)
    • leader_weight: (number)
    • leader_score: (number)
    • leader_size: (integer)
    • region_count: (integer)
    • region_weight: (number)
    • region_score: (number)
    • region_size: (integer)
    • sending_snap_count: (integer)
    • receiving_snap_count: (integer)
    • applying_snap_count: (integer)
    • is_busy: (boolean)
    • start_ts: (string)
    • last_heartbeat_ts: (string)
    • uptime: (string)

HTTP status code 400

The input is invalid.

HTTP status code 500

PD server failed to proceed the request.

Take down a store from the cluster.

URI Parameters

  • storeId: required (integer)

Query Parameters

  • force: (string)

    Set status to Tombstone directly.

HTTP status code 200

The store is set as Offline or Tombstone.

HTTP status code 400

The input is invalid.

HTTP status code 404

The store does not exist.

HTTP status code 410

The store has already been removed.

HTTP status code 500

PD server failed to proceed the request.

post

The specific store's state.

post

Set the store's state.

Set the store's state.

URI Parameters

  • storeId: required (integer)

Query Parameters

  • state: required (one of Up, Offline, Tombstone)

HTTP status code 200

The store's state is updated.

HTTP status code 400

The input is invalid.

HTTP status code 404

The store does not exist.

HTTP status code 500

PD server failed to proceed the request.

post

The specific store's label.

post

Set the store's label.

Set the store's label.

URI Parameters

  • storeId: required (integer)

Body

Media type: application/json

Type: object

HTTP status code 200

The store's label is updated.

HTTP status code 400

The input is invalid.

HTTP status code 500

PD server failed to proceed the request.

post

The specific store's weight.

post

Set the store's leader/region weight.

Set the store's leader/region weight.

URI Parameters

  • storeId: required (integer)

Body

Media type: application/json

Type: object

HTTP status code 200

The store's weight is updated.

HTTP status code 400

The input is invalid.

HTTP status code 500

PD server failed to proceed the request.

/labels

The store label values in the cluster.

get

get

List all label values.

get

get

List stores that have specific label values.

/region

A specific region in the cluster.

get

get

Search for a region by region ID.

Search for a region by region ID.

URI Parameters

  • id: required (integer)

HTTP status code 200

Body

Media type: application/json

Type: object

Properties
  • id: required (integer)
  • start_key: required (string)
  • end_key: required (string)
  • epoch: (object)
    • conf_ver: (integer)
    • version: (integer)
  • peers: (array of Peer)

    Items: Peer

    • id: required (integer)
    • store_id: required (integer)
    • is_learner: (boolean)
  • leader: (object)
    • id: required (integer)
    • store_id: required (integer)
    • is_learner: (boolean)
  • down_peers: (array of PeerStats)

    Items: PeerStats

    • peer: (object)
      • id: required (integer)
      • store_id: required (integer)
      • is_learner: (boolean)
    • down_seconds: required (integer)
  • pending_peers: (array of Peer)

    Items: Peer

    • id: required (integer)
    • store_id: required (integer)
    • is_learner: (boolean)
  • written_bytes: (integer)
  • read_bytes: (integer)
  • approximate_size: (integer)
  • approximate_keys: (integer)

HTTP status code 400

The input is invalid.

HTTP status code 500

PD server failed to proceed the request.

get

get

Search for a region by a key.

Search for a region by a key.

URI Parameters

  • key: required (string)

HTTP status code 200

Body

Media type: application/json

Type: object

Properties
  • id: required (integer)
  • start_key: required (string)
  • end_key: required (string)
  • epoch: (object)
    • conf_ver: (integer)
    • version: (integer)
  • peers: (array of Peer)

    Items: Peer

    • id: required (integer)
    • store_id: required (integer)
    • is_learner: (boolean)
  • leader: (object)
    • id: required (integer)
    • store_id: required (integer)
    • is_learner: (boolean)
  • down_peers: (array of PeerStats)

    Items: PeerStats

    • peer: (object)
      • id: required (integer)
      • store_id: required (integer)
      • is_learner: (boolean)
    • down_seconds: required (integer)
  • pending_peers: (array of Peer)

    Items: Peer

    • id: required (integer)
    • store_id: required (integer)
    • is_learner: (boolean)
  • written_bytes: (integer)
  • read_bytes: (integer)
  • approximate_size: (integer)
  • approximate_keys: (integer)

HTTP status code 500

PD server failed to proceed the request.

/regions

The regions in the cluster.

get

get

List all regions in the cluster.

get

get

List regions with the highest write flow.

get

get

List regions with the highest read flow.

get

get

List regions with the largest conf version.

get

get

List regions with the largest version.

get

get

List regions with the largest size.

get

get

List regions start from a key.

get

get

List regions with unhealthy status.

List regions with unhealthy status.

URI Parameters

  • filter: required (one of miss-peer, extra-peer, pending-peer, down-peer, incorrect-ns)

HTTP status code 200

Body

Media type: application/json

Type: object

Properties
  • count: required (integer)
  • regions: required (array of Region)

    Items: Region

    • id: required (integer)
    • start_key: required (string)
    • end_key: required (string)
    • epoch: (object)
      • conf_ver: (integer)
      • version: (integer)
    • peers: (array of Peer)

      Items: Peer

      • id: required (integer)
      • store_id: required (integer)
      • is_learner: (boolean)
    • leader: (object)
      • id: required (integer)
      • store_id: required (integer)
      • is_learner: (boolean)
    • down_peers: (array of PeerStats)

      Items: PeerStats

      • peer: (object)
        • id: required (integer)
        • store_id: required (integer)
        • is_learner: (boolean)
      • down_seconds: required (integer)
    • pending_peers: (array of Peer)

      Items: Peer

      • id: required (integer)
      • store_id: required (integer)
      • is_learner: (boolean)
    • written_bytes: (integer)
    • read_bytes: (integer)
    • approximate_size: (integer)
    • approximate_keys: (integer)

HTTP status code 500

PD server failed to proceed the request.

get

get

List sibling regions of a specific region.

List sibling regions of a specific region.

URI Parameters

  • id: required (integer)

HTTP status code 200

Body

Media type: application/json

Type: object

Properties
  • count: required (integer)
  • regions: required (array of Region)

    Items: Region

    • id: required (integer)
    • start_key: required (string)
    • end_key: required (string)
    • epoch: (object)
      • conf_ver: (integer)
      • version: (integer)
    • peers: (array of Peer)

      Items: Peer

      • id: required (integer)
      • store_id: required (integer)
      • is_learner: (boolean)
    • leader: (object)
      • id: required (integer)
      • store_id: required (integer)
      • is_learner: (boolean)
    • down_peers: (array of PeerStats)

      Items: PeerStats

      • peer: (object)
        • id: required (integer)
        • store_id: required (integer)
        • is_learner: (boolean)
      • down_seconds: required (integer)
    • pending_peers: (array of Peer)

      Items: Peer

      • id: required (integer)
      • store_id: required (integer)
      • is_learner: (boolean)
    • written_bytes: (integer)
    • read_bytes: (integer)
    • approximate_size: (integer)
    • approximate_keys: (integer)

HTTP status code 400

The input is invalid.

HTTP status code 404

The region does not exist.

HTTP status code 500

PD server failed to proceed the request.

get

get

List all regions of a specific store.

List all regions of a specific store.

URI Parameters

  • id: required (integer)

HTTP status code 200

Body

Media type: application/json

Type: object

Properties
  • count: required (integer)
  • regions: required (array of Region)

    Items: Region

    • id: required (integer)
    • start_key: required (string)
    • end_key: required (string)
    • epoch: (object)
      • conf_ver: (integer)
      • version: (integer)
    • peers: (array of Peer)

      Items: Peer

      • id: required (integer)
      • store_id: required (integer)
      • is_learner: (boolean)
    • leader: (object)
      • id: required (integer)
      • store_id: required (integer)
      • is_learner: (boolean)
    • down_peers: (array of PeerStats)

      Items: PeerStats

      • peer: (object)
        • id: required (integer)
        • store_id: required (integer)
        • is_learner: (boolean)
      • down_seconds: required (integer)
    • pending_peers: (array of Peer)

      Items: Peer

      • id: required (integer)
      • store_id: required (integer)
      • is_learner: (boolean)
    • written_bytes: (integer)
    • read_bytes: (integer)
    • approximate_size: (integer)
    • approximate_keys: (integer)

HTTP status code 400

The input is invalid.

HTTP status code 500

PD server failed to proceed the request.

/schedulers

Running schedulers.

get post

get

List running schedulers.

post

Create a scheduler.

Create a scheduler.

Body

Media type: application/json

Type: object

Properties
  • name: required (string)

HTTP status code 200

The scheduler is created.

HTTP status code 400

Bad format request.

HTTP status code 500

PD server failed to proceed the request.

delete

A specific scheduler.

delete

Delete a scheduler.

Delete a scheduler.

URI Parameters

  • name: required (string)

    The name of the scheduler.

HTTP status code 200

The scheduler is removed.

HTTP status code 500

PD server failed to proceed the request.

/operators

Pending operators.

get post

get

List pending operators.

post

Create an operator.

Create an operator.

Body

Media type: application/json

Type: object

Properties
  • name: required (string)

HTTP status code 200

The operator is created.

HTTP status code 400

The input is invalid.

HTTP status code 500

PD server failed to proceed the request.

get delete

A specific Region's pending operator.

get

Get a Region's pending operator.

delete

Cancel a Region's pending operator.

Get a Region's pending operator.

URI Parameters

  • regionId: required (integer)

    A Region's Id.

HTTP status code 200

Body

Media type: application/json

Type: string

HTTP status code 400

The input is invalid.

HTTP status code 500

PD server failed to proceed the request.

Cancel a Region's pending operator.

URI Parameters

  • regionId: required (integer)

    A Region's Id.

HTTP status code 200

The pending operator is cancelled.

HTTP status code 400

The input is invalid.

HTTP status code 500

PD server failed to proceed the request.

/hotspot

The hot spots status in the cluster.

get

get

List the hot write regions.

get

get

List the hot read regions.

get

get

List the hot stores.

/stats

Statistics of the cluster.

get

get

Get region statistics of a specified range.

/trend

Trend of data growth and movements.

get

get

Get the growth and changes of data in the most recent period of time.

/admin

delete

delete

Drop a specific region from cache.

Drop a specific region from cache.

URI Parameters

  • id: required (integer)

HTTP status code 200

The region is removed from server cache.

HTTP status code 400

The input is invalid.

HTTP status code 500

PD server failed to proceed the request.

post

The log level of PD server.

post

Set log level.

Set log level.

Body

Media type: application/json

Type: string

HTTP status code 200

The log level is updated.

HTTP status code 400

The input is invalid.

HTTP status code 500

PD server failed to proceed the request.

/classifier

The namespace classifier. Methods depend on current classifier.

\ No newline at end of file +

Placement Driver API API documentation version v1

http://{pdAddr}/pd/api/{version}

  • pdAddr: required (string)

    The PD server address, formatted as 'host:port'.

  • version: required (v1)

/cluster/status

Cluster status.

get

get

Get cluster status.

/version

The version of PD server.

get

get

Get the version of PD server.

/status

The build info of PD server.

get

get

Get the build info of PD server.

/diagnose

Diagnostic information of the cluster.

get

get

/members

The PD servers in the cluster.

get

get

List all PD servers in the cluster.

delete post

A specific PD server.

delete

Remove a PD server from the cluster.

post

Set leader priority of a PD member.

Remove a PD server from the cluster.

URI Parameters

  • name: required (string)

HTTP status code 200

The PD server is successfully removed.

HTTP status code 400

The input is invalid.

HTTP status code 404

The member does not exist.

HTTP status code 500

PD server failed to proceed the request.

Set leader priority of a PD member.

URI Parameters

  • name: required (string)

Body

Media type: application/json

Type: object

Properties
  • leader-priority: required (integer)

HTTP status code 200

The leader priority is updated.

HTTP status code 400

The input is invalid.

HTTP status code 404

The member does not exist.

HTTP status code 500

PD server failed to proceed the request.

delete

A specific PD server.

delete

Remove a PD server from the cluster.

Remove a PD server from the cluster.

URI Parameters

  • id: required (integer)

HTTP status code 200

The PD server is successfully removed.

HTTP status code 400

The input is invalid.

HTTP status code 500

PD server failed to proceed the request.

/leader

The leader PD server of the cluster.

get

get

Get the leader PD server of the cluster.

post

post

Transfer leadership to another PD server.

Transfer leadership to another PD server.

HTTP status code 200

The transfer command is submitted.

HTTP status code 500

PD server failed to proceed the request.

post

post

Transfer leadership to the specific PD server.

Transfer leadership to the specific PD server.

URI Parameters

  • nextLeader: required (string)

HTTP status code 200

The transfer command is submitted.

HTTP status code 500

PD server failed to proceed the request.

/health

Health status of PD servers.

get

get

/config

PD cluster configuration.

get post

get

Get full config.

post

Update a config item.

Update a config item.

Body

Media type: application/json

Type: object

HTTP status code 200

The config is updated.

HTTP status code 500

PD server failed to proceed the request.

get post

Schedule configuration.

get

Get schedule config.

post

Update a schedule config item.

Update a schedule config item.

Body

Media type: application/json

Type: object

HTTP status code 200

The config is updated.

HTTP status code 400

The input is invalid.

HTTP status code 500

PD server failed to proceed the request.

get post

Replication configuration.

get

Get replication config.

post

Update a replication config item.

Update a replication config item.

Body

Media type: application/json

Type: object

HTTP status code 200

The config is updated.

HTTP status code 400

The input is invalid.

HTTP status code 500

PD server failed to proceed the request.

get post delete

The config of a namespace.

get

Get configuration of a namespace.

post

Update a namespace config item.

delete

Delete a namespace config.

Get configuration of a namespace.

URI Parameters

  • namespaceName: required (string)

    The name of the namespace.

HTTP status code 200

Body

Media type: application/json

Type: object

Properties
  • leader-schedule-limit: required (integer)
  • region-schedule-limit: required (integer)
  • replica-schedule-limit: required (integer)
  • merge-schedule-limit: required (integer)
  • max-replicas: required (integer)

HTTP status code 404

The namespace does not exist.

Update a namespace config item.

URI Parameters

  • namespaceName: required (string)

    The name of the namespace.

Body

Media type: application/json

Type: object

HTTP status code 200

The config is updated.

HTTP status code 400

The input is invalid.

HTTP status code 404

The namespace does not exist.

Delete a namespace config.

URI Parameters

  • namespaceName: required (string)

    The name of the namespace.

HTTP status code 200

The config is removed.

HTTP status code 404

The namespace does not exist.

get post

The label property configuration.

get

Get label property config.

post

Update label property config item.

Update label property config item.

Body

Media type: application/json

Type: object

Properties
  • action: required (one of set, delete)
  • type: required (reject-leader)
  • label-key: required (string)
  • label-value: required (string)

HTTP status code 200

The config is updated.

HTTP status code 500

PD server failed to proceed the request.

/stores

The stores in the cluster.

get

get

Get stores in the cluster.

get post

The balance rate limit for all stores.

get

Get all stores' balance rate limit.

post

Set all stores' balance rate limit.

Set all stores' balance rate limit.

Body

Media type: application/json

Type: object

HTTP status code 200

All stores' balance rate limits are updated.

HTTP status code 400

The input is invalid.

HTTP status code 500

PD server failed to proceed the request.

delete

Remove all tombstone stores.

delete

Remove all tombstone stores.

/store/{storeId}

A specific store.

get delete

get

Get a store's information.

delete

Take down a store from the cluster.

Get a store's information.

URI Parameters

  • storeId: required (integer)

HTTP status code 200

Body

Media type: application/json

Type: object

Properties
  • store: required (object)
    • id: required (integer)
    • address: required (string)
    • state: required (one of 0, 1, 2)
    • state_name: required (one of Up, Disconnected, Down, Offline, Tombstone)
    • labels: (array of StoreLabel)

      Items: StoreLabel

      • key: required (string)
      • value: required (string)
    • version: (string)
  • status: required (object)
    • capacity: required (string)
    • available: required (string)
    • leader_count: (integer)
    • leader_weight: (number)
    • leader_score: (number)
    • leader_size: (integer)
    • region_count: (integer)
    • region_weight: (number)
    • region_score: (number)
    • region_size: (integer)
    • sending_snap_count: (integer)
    • receiving_snap_count: (integer)
    • applying_snap_count: (integer)
    • is_busy: (boolean)
    • start_ts: (string)
    • last_heartbeat_ts: (string)
    • uptime: (string)

HTTP status code 400

The input is invalid.

HTTP status code 500

PD server failed to proceed the request.

Take down a store from the cluster.

URI Parameters

  • storeId: required (integer)

Query Parameters

  • force: (string)

    Set status to Tombstone directly.

HTTP status code 200

The store is set as Offline or Tombstone.

HTTP status code 400

The input is invalid.

HTTP status code 404

The store does not exist.

HTTP status code 410

The store has already been removed.

HTTP status code 500

PD server failed to proceed the request.

post

The state for the specific store.

post

Set the store's state.

Set the store's state.

URI Parameters

  • storeId: required (integer)

Query Parameters

  • state: required (one of Up, Offline, Tombstone)

HTTP status code 200

The store's state is updated.

HTTP status code 400

The input is invalid.

HTTP status code 404

The store does not exist.

HTTP status code 500

PD server failed to proceed the request.

post

The label for the specific store.

post

Set the store's label.

Set the store's label.

URI Parameters

  • storeId: required (integer)

Body

Media type: application/json

Type: object

HTTP status code 200

The store's label is updated.

HTTP status code 400

The input is invalid.

HTTP status code 500

PD server failed to proceed the request.

post

The weight for the specific store.

post

Set the store's leader/region weight.

Set the store's leader/region weight.

URI Parameters

  • storeId: required (integer)

Body

Media type: application/json

Type: object

HTTP status code 200

The store's weight is updated.

HTTP status code 400

The input is invalid.

HTTP status code 500

PD server failed to proceed the request.

post

The balance rate limit for the specific store.

post

Set the store's balance rate limit.

Set the store's balance rate limit.

URI Parameters

  • storeId: required (integer)

Body

Media type: application/json

Type: object

HTTP status code 200

The store's balance rate limit is updated.

HTTP status code 400

The input is invalid.

HTTP status code 500

PD server failed to proceed the request.

/labels

The store label values in the cluster.

get

get

List all label values.

get

get

List stores that have specific label values.

/region

A specific region in the cluster.

get

get

Search for a region by region ID.

Search for a region by region ID.

URI Parameters

  • id: required (integer)

HTTP status code 200

Body

Media type: application/json

Type: object

Properties
  • id: required (integer)
  • start_key: required (string)
  • end_key: required (string)
  • epoch: (object)
    • conf_ver: (integer)
    • version: (integer)
  • peers: (array of Peer)

    Items: Peer

    • id: required (integer)
    • store_id: required (integer)
    • is_learner: (boolean)
  • leader: (object)
    • id: required (integer)
    • store_id: required (integer)
    • is_learner: (boolean)
  • down_peers: (array of PeerStats)

    Items: PeerStats

    • peer: (object)
      • id: required (integer)
      • store_id: required (integer)
      • is_learner: (boolean)
    • down_seconds: required (integer)
  • pending_peers: (array of Peer)

    Items: Peer

    • id: required (integer)
    • store_id: required (integer)
    • is_learner: (boolean)
  • written_bytes: (integer)
  • read_bytes: (integer)
  • approximate_size: (integer)
  • approximate_keys: (integer)

HTTP status code 400

The input is invalid.

HTTP status code 500

PD server failed to proceed the request.

get

get

Search for a region by a key.

Search for a region by a key.

URI Parameters

  • key: required (string)

HTTP status code 200

Body

Media type: application/json

Type: object

Properties
  • id: required (integer)
  • start_key: required (string)
  • end_key: required (string)
  • epoch: (object)
    • conf_ver: (integer)
    • version: (integer)
  • peers: (array of Peer)

    Items: Peer

    • id: required (integer)
    • store_id: required (integer)
    • is_learner: (boolean)
  • leader: (object)
    • id: required (integer)
    • store_id: required (integer)
    • is_learner: (boolean)
  • down_peers: (array of PeerStats)

    Items: PeerStats

    • peer: (object)
      • id: required (integer)
      • store_id: required (integer)
      • is_learner: (boolean)
    • down_seconds: required (integer)
  • pending_peers: (array of Peer)

    Items: Peer

    • id: required (integer)
    • store_id: required (integer)
    • is_learner: (boolean)
  • written_bytes: (integer)
  • read_bytes: (integer)
  • approximate_size: (integer)
  • approximate_keys: (integer)

HTTP status code 500

PD server failed to proceed the request.

/regions

The regions in the cluster.

get

get

List all regions in the cluster.

get

get

List regions with the highest write flow.

get

get

List regions with the highest read flow.

get

get

List regions with the largest conf version.

get

get

List regions with the largest version.

get

get

List regions with the largest size.

get

get

List regions start from a key.

get

get

List regions with unhealthy status.

List regions with unhealthy status.

URI Parameters

  • filter: required (one of miss-peer, extra-peer, pending-peer, down-peer, incorrect-ns)

HTTP status code 200

Body

Media type: application/json

Type: object

Properties
  • count: required (integer)
  • regions: required (array of Region)

    Items: Region

    • id: required (integer)
    • start_key: required (string)
    • end_key: required (string)
    • epoch: (object)
      • conf_ver: (integer)
      • version: (integer)
    • peers: (array of Peer)

      Items: Peer

      • id: required (integer)
      • store_id: required (integer)
      • is_learner: (boolean)
    • leader: (object)
      • id: required (integer)
      • store_id: required (integer)
      • is_learner: (boolean)
    • down_peers: (array of PeerStats)

      Items: PeerStats

      • peer: (object)
        • id: required (integer)
        • store_id: required (integer)
        • is_learner: (boolean)
      • down_seconds: required (integer)
    • pending_peers: (array of Peer)

      Items: Peer

      • id: required (integer)
      • store_id: required (integer)
      • is_learner: (boolean)
    • written_bytes: (integer)
    • read_bytes: (integer)
    • approximate_size: (integer)
    • approximate_keys: (integer)

HTTP status code 500

PD server failed to proceed the request.

get

get

List sibling regions of a specific region.

List sibling regions of a specific region.

URI Parameters

  • id: required (integer)

HTTP status code 200

Body

Media type: application/json

Type: object

Properties
  • count: required (integer)
  • regions: required (array of Region)

    Items: Region

    • id: required (integer)
    • start_key: required (string)
    • end_key: required (string)
    • epoch: (object)
      • conf_ver: (integer)
      • version: (integer)
    • peers: (array of Peer)

      Items: Peer

      • id: required (integer)
      • store_id: required (integer)
      • is_learner: (boolean)
    • leader: (object)
      • id: required (integer)
      • store_id: required (integer)
      • is_learner: (boolean)
    • down_peers: (array of PeerStats)

      Items: PeerStats

      • peer: (object)
        • id: required (integer)
        • store_id: required (integer)
        • is_learner: (boolean)
      • down_seconds: required (integer)
    • pending_peers: (array of Peer)

      Items: Peer

      • id: required (integer)
      • store_id: required (integer)
      • is_learner: (boolean)
    • written_bytes: (integer)
    • read_bytes: (integer)
    • approximate_size: (integer)
    • approximate_keys: (integer)

HTTP status code 400

The input is invalid.

HTTP status code 404

The region does not exist.

HTTP status code 500

PD server failed to proceed the request.

get

get

List all regions of a specific store.

List all regions of a specific store.

URI Parameters

  • id: required (integer)

HTTP status code 200

Body

Media type: application/json

Type: object

Properties
  • count: required (integer)
  • regions: required (array of Region)

    Items: Region

    • id: required (integer)
    • start_key: required (string)
    • end_key: required (string)
    • epoch: (object)
      • conf_ver: (integer)
      • version: (integer)
    • peers: (array of Peer)

      Items: Peer

      • id: required (integer)
      • store_id: required (integer)
      • is_learner: (boolean)
    • leader: (object)
      • id: required (integer)
      • store_id: required (integer)
      • is_learner: (boolean)
    • down_peers: (array of PeerStats)

      Items: PeerStats

      • peer: (object)
        • id: required (integer)
        • store_id: required (integer)
        • is_learner: (boolean)
      • down_seconds: required (integer)
    • pending_peers: (array of Peer)

      Items: Peer

      • id: required (integer)
      • store_id: required (integer)
      • is_learner: (boolean)
    • written_bytes: (integer)
    • read_bytes: (integer)
    • approximate_size: (integer)
    • approximate_keys: (integer)

HTTP status code 400

The input is invalid.

HTTP status code 500

PD server failed to proceed the request.

/schedulers

Running schedulers.

get post

get

List running schedulers.

post

Create a scheduler.

Create a scheduler.

Body

Media type: application/json

Type: object

Properties
  • name: required (string)

HTTP status code 200

The scheduler is created.

HTTP status code 400

Bad format request.

HTTP status code 500

PD server failed to proceed the request.

delete

A specific scheduler.

delete

Delete a scheduler.

Delete a scheduler.

URI Parameters

  • name: required (string)

    The name of the scheduler.

HTTP status code 200

The scheduler is removed.

HTTP status code 500

PD server failed to proceed the request.

/operators

Pending operators.

get post

get

List pending operators.

post

Create an operator.

Create an operator.

Body

Media type: application/json

Type: object

Properties
  • name: required (string)

HTTP status code 200

The operator is created.

HTTP status code 400

The input is invalid.

HTTP status code 500

PD server failed to proceed the request.

get delete

A specific Region's pending operator.

get

Get a Region's pending operator.

delete

Cancel a Region's pending operator.

Get a Region's pending operator.

URI Parameters

  • regionId: required (integer)

    A Region's Id.

HTTP status code 200

Body

Media type: application/json

Type: string

HTTP status code 400

The input is invalid.

HTTP status code 500

PD server failed to proceed the request.

Cancel a Region's pending operator.

URI Parameters

  • regionId: required (integer)

    A Region's Id.

HTTP status code 200

The pending operator is cancelled.

HTTP status code 400

The input is invalid.

HTTP status code 500

PD server failed to proceed the request.

/hotspot

The hot spots status in the cluster.

get

get

List the hot write regions.

get

get

List the hot read regions.

get

get

List the hot stores.

/stats

Statistics of the cluster.

get

get

Get region statistics of a specified range.

/trend

Trend of data growth and movements.

get

get

Get the growth and changes of data in the most recent period of time.

/admin

delete

delete

Drop a specific region from cache.

Drop a specific region from cache.

URI Parameters

  • id: required (integer)

HTTP status code 200

The region is removed from server cache.

HTTP status code 400

The input is invalid.

HTTP status code 500

PD server failed to proceed the request.

post

The log level of PD server.

post

Set log level.

Set log level.

Body

Media type: application/json

Type: string

HTTP status code 200

The log level is updated.

HTTP status code 400

The input is invalid.

HTTP status code 500

PD server failed to proceed the request.

/classifier

The namespace classifier. Methods depend on current classifier.

\ No newline at end of file diff --git a/server/api/api.raml b/server/api/api.raml index 6612ce9f68dd..e7ac091b6743 100644 --- a/server/api/api.raml +++ b/server/api/api.raml @@ -71,9 +71,13 @@ types: region-schedule-limit?: integer replica-schedule-limit?: integer merge-schedule-limit?: integer + hot-region-schedule-limit?: integer + hot-region-cache-hits-threshold?: integer + store-balance-rate?: number tolerant-size-ratio?: number low-space-ratio?: number high-space-ratio?: number + scheduler-max-waiting-operator?: integer disable-raft-learner?: boolean disable-remove-down-replica?: boolean disable-replace-offline-replica?: boolean @@ -663,6 +667,41 @@ types: 500: description: PD server failed to proceed the request. + /limit: + description: The balance rate limit for all stores. + get: + description: Get all stores' balance rate limit. + responses: + 200: + body: + application/json: + type: string + 500: + description: PD server failed to proceed the request. + post: + description: Set all stores' balance rate limit. + body: + application/json: + description: key-value pair. + type: object + responses: + 200: + description: All stores' balance rate limits are updated. + 400: + description: The input is invalid. + 500: + description: PD server failed to proceed the request. + + /remove-tombstone: + description: Remove all tombstone stores. + delete: + description: Remove all tombstone stores. + responses: + 200: + description: All tombstone stores are removed. + 500: + description: PD server failed to proceed the request. + /store/{storeId}: description: A specific store. uriParameters: @@ -696,7 +735,7 @@ types: description: PD server failed to proceed the request. /state: - description: The specific store's state. + description: The state for the specific store. post: description: Set the store's state. queryParameters: @@ -714,7 +753,7 @@ types: description: PD server failed to proceed the request. /label: - description: The specific store's label. + description: The label for the specific store. post: description: Set the store's label. body: @@ -730,7 +769,7 @@ types: description: PD server failed to proceed the request. /weight: - description: The specific store's weight. + description: The weight for the specific store. post: description: Set the store's leader/region weight. body: @@ -746,6 +785,22 @@ types: 500: description: PD server failed to proceed the request. + /limit: + description: The balance rate limit for the specific store. + post: + description: Set the store's balance rate limit. + body: + application/json: + description: key-value pair. + type: object + responses: + 200: + description: The store's balance rate limit is updated. + 400: + description: The input is invalid. + 500: + description: PD server failed to proceed the request. + /labels: description: The store label values in the cluster. get: diff --git a/server/schedule/selector.go b/server/schedule/selector.go index 721ec425f222..0530f106c13a 100644 --- a/server/schedule/selector.go +++ b/server/schedule/selector.go @@ -34,7 +34,7 @@ func NewBalanceSelector(kind core.ResourceKind, filters []Filter) *BalanceSelect } } -// SelectSource selects the store that can pass all filters and has the minimal +// SelectSource selects the store that can pass all filters and has the maximal // resource score. func (s *BalanceSelector) SelectSource(opt Options, stores []*core.StoreInfo) *core.StoreInfo { var result *core.StoreInfo @@ -51,7 +51,7 @@ func (s *BalanceSelector) SelectSource(opt Options, stores []*core.StoreInfo) *c return result } -// SelectTarget selects the store that can pass all filters and has the maximal +// SelectTarget selects the store that can pass all filters and has the minimal // resource score. func (s *BalanceSelector) SelectTarget(opt Options, stores []*core.StoreInfo, filters ...Filter) *core.StoreInfo { filters = append(filters, s.filters...) diff --git a/server/schedulers/balance_region.go b/server/schedulers/balance_region.go index aa53feb37e95..cc9057911ad2 100644 --- a/server/schedulers/balance_region.go +++ b/server/schedulers/balance_region.go @@ -207,12 +207,29 @@ func (s *balanceRegionScheduler) hasPotentialTarget(cluster schedule.Cluster, re for _, store := range cluster.GetStores() { if schedule.FilterTarget(cluster, store, filters) { + log.Debug("skip target store by filters", + zap.String("scheduler", s.GetName()), + zap.Uint64("region", region.GetID()), + zap.Uint64("source", source.GetID()), + zap.Uint64("target", store.GetID())) continue } if !store.IsUp() || store.DownTime() > cluster.GetMaxStoreDownTime() { + log.Debug("skip target store by status", + zap.String("scheduler", s.GetName()), + zap.Uint64("region", region.GetID()), + zap.Uint64("source", source.GetID()), + zap.Uint64("target", store.GetID()), + zap.Bool("isup", store.IsUp()), + zap.Duration("downtime", store.DownTime())) continue } if !shouldBalance(cluster, source, store, region, core.RegionKind, opInfluence) { + log.Debug("skip target store for it should not balance", + zap.String("scheduler", s.GetName()), + zap.Uint64("region", region.GetID()), + zap.Uint64("source", source.GetID()), + zap.Uint64("target", store.GetID())) continue } return true diff --git a/tools/pd-ctl/README.md b/tools/pd-ctl/README.md index 255e608109bd..8612dddbfa08 100644 --- a/tools/pd-ctl/README.md +++ b/tools/pd-ctl/README.md @@ -120,8 +120,9 @@ Usage: "max-store-down-time": "30m0s", "merge-schedule-limit": 8, "patrol-region-interval": "100ms", - "region-schedule-limit": 4, - "replica-schedule-limit": 8, + "region-schedule-limit": 64, + "replica-schedule-limit": 64, + "scheduler-max-waiting-operator": 3, "schedulers-v2": [ { "args": null, @@ -146,7 +147,7 @@ Usage: ], "split-merge-interval": "1h0m0s", "store-balance-rate": 1, - "tolerant-size-ratio": 5 + "tolerant-size-ratio": 0 } } >> config show all // Display all config information @@ -169,7 +170,7 @@ Usage: - `max-snapshot-count` controls the maximum number of snapshots that a single store receives or sends out at the same time. The scheduler is restricted by this configuration to avoid taking up normal application resources. When you need to improve the speed of adding replicas or balancing, increase this value. ```bash - >> config set max-snapshort-count 16 // Set the maximum number of snapshots to 16 + >> config set max-snapshot-count 16 // Set the maximum number of snapshots to 16 ``` - `max-pending-peer-count` controls the maximum number of pending peers in a single store. The scheduler is restricted by this configuration to avoid producing a large number of Regions without the latest log in some nodes. When you need to improve the speed of adding replicas or balancing, increase this value. Setting it to 0 indicates no limit.