Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: RFC for keyspace #39685

Merged
merged 40 commits into from
Oct 30, 2024
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
19cfda1
add keyspace rfc
ystaticy Dec 6, 2022
862e07d
docs:RFC for keyspace
ystaticy Dec 7, 2022
634a9fc
add client-go and cop section
iosmanthus Dec 9, 2022
16d9467
Merge pull request #1 from iosmanthus/iosmanthus/keyspace-rfc
ystaticy Dec 12, 2022
37cc3d3
Merge remote-tracking branch 'origin/master' into keyspace_rfc
ystaticy May 10, 2024
3748198
add BR desc
ystaticy May 10, 2024
cd19be5
add lightning desc
ystaticy May 10, 2024
d56d923
add pd desc
ystaticy May 11, 2024
2ede41a
add BR desc
ystaticy May 11, 2024
4bbc4b6
add gC desc
ystaticy May 11, 2024
32d1583
add tiflash desc
ystaticy May 11, 2024
6294f55
update tidb desc
ystaticy May 11, 2024
093ccaa
fix PD desc
ystaticy May 15, 2024
1a1cbcc
fix BR desc
ystaticy May 15, 2024
b4543b5
fix GC desc
ystaticy May 15, 2024
45c7c87
fix GC desc
ystaticy May 15, 2024
7b29bba
fix GC desc
ystaticy May 15, 2024
3b339d6
rfc: add TiFlash keyspace sections
iosmanthus May 15, 2024
0121e99
Merge pull request #3 from iosmanthus/iosmanthus/keyspace_rfc
ystaticy May 15, 2024
3b2fa78
fix link
ystaticy May 17, 2024
0bddd8f
fix link
ystaticy May 17, 2024
88f75fe
add ks state machine
ystaticy May 28, 2024
d55948a
add ks name
ystaticy May 28, 2024
c3288bf
support TiCDC
zeminzhou May 28, 2024
2d0f60c
add only
zeminzhou May 28, 2024
9fb1c9e
optimize with chatgpt
zeminzhou May 28, 2024
bb2efa7
Merge pull request #4 from zeminzhou/zeminzhou/keyspace_rfc
ystaticy Jun 5, 2024
dd33412
correct grammatical and spelling errors
ystaticy Sep 19, 2024
37765f2
small fix
ystaticy Sep 23, 2024
25879e2
small fix: Grammar and Expression
ystaticy Sep 23, 2024
39b745c
small fix: Grammar and Expression
ystaticy Sep 23, 2024
c3785c7
update words
ystaticy Oct 14, 2024
410fffc
update words
ystaticy Oct 14, 2024
347556b
update words
ystaticy Oct 14, 2024
f9f0e9f
update words
ystaticy Oct 14, 2024
f3b521d
add link
ystaticy Oct 29, 2024
98d26fd
add pd interface link
ystaticy Oct 29, 2024
92b81c5
add Observability part
ystaticy Oct 29, 2024
657e401
add Observability part
ystaticy Oct 29, 2024
9d7af9e
add Observability part
ystaticy Oct 29, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
69 changes: 69 additions & 0 deletions docs/design/2022-12-06-Keyspace.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
# Proposal: Keyspace
* Authors: [ystaticy](https://github.com/ystaticy), [iosmanthus](https://github.com/iosmanthus), [AmoebaProtozoa](https://github.com/AmoebaProtozoa)


## Motivation
We hope to store data of multiple applications in a TiKV cluster.
For example in multi-tenant scenario.A common solution is to add a prefix in front of the key to distinguish the data of multiple applications.
So we introduce a new concept 'Keyspace' to describe the logical of the isolation with different business scenarios in a TiKV cluster.


## Keyspace key prefix
Keyspace can only be used when TiKV enabled api v2.
ystaticy marked this conversation as resolved.
Show resolved Hide resolved
### Key encoding:
We use different key prefix to distinguish different Keyspace.
[Keyspace key encoding](https://github.com/tikv/rfcs/blob/master/text/0069-api-v2.md#key-encoding)

'x', 'r' are key mode prefixes that indicates which mode the key is belonging to.
After mode prefix is 3 bytes for keyspace.
1. `x`: It represent TxnKV key.
2. `r`: It represent RawKV key.

So when TiDB enabled Keyspace, there are 4 bytes in key more than the scenario which does not use Keyspace.
It will start with 'x' + keyspace Id (3 bytes) ,because TiDB uses TxnKV client.

### Keyspace ID
Behind key mode, it take 3 bytes to store `keyspaceID`, the default keyspaceID is `[0, 0, 0]`.
The max keyspace id is 16777216; Keyspace ID is allocated by PD createKeyspace interface.

## Architecture
![group and expression](./imgs/keyspace-arch.png)
### PD
PD is responsible for Keyspace Meta managment.
It provide http interface and RPC interface to create, load, change state or configuration Keyspace Meta.
Keyspace Meta store in etcd data in PD.

### Client-go
Compare with API V1. API V2 add 1 byte Keymode and 3 bytes keyspace Id in front of userkey.

### TiDB
A tenant can have multiple TiDB nodes. But a TiDB node can only serve one tenant.
It can enabled by tidb configuration by `keyspace-name`.

### TiKV
TiKV should enable API V2, and different keyspace will distinguish by different key prefix.

## Implementation

### PD
To be supplemented in another PR

### TiDB
#### Using API V2 and etcd namespace to distinguish TiKV data and etcd path by Keyspace.
1. Configure Keyspace name by setting `keyspace-name` in the configuration file.
* When TiDB server start, it will open a TiKV driver. The driver have a TxnKV client object `tikv.KVStore`. If `keyspace-name` has been set, it will start to use `tikv.KVStore` by API V2 to access TiKV.
2. The etcd path should has different prefix by etcd namespace.
* There is a etcd client be created when domain init. If `keyspace-name` has been set, it will add a etcd namespace when create etcd client.
* The format of etcd namespace is `/keyspaces/tidb/$keyspaceId`.

#### TiDB BR support Keyspace
To be supplemented in another PR

### TiKV
#### client-go
To be supplemented in another PR
#### Coprocessor
To be supplemented in another PR



Binary file added docs/design/imgs/keyspace-arch.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.