-
Notifications
You must be signed in to change notification settings - Fork 3.8k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
multitenant: prevent tenant from overriding next tenant's span config
Fixes #95882 We want to ensure we have a split at the non-inclusive end of the tenant's keyspace, which also happens to be the inclusive start of the next tenant's (with ID=ours+1). If we didn't do anything here, and we were the tenant with the highest ID thus far, our last range would extend to /Max. If a tenant with a higher ID was created, when installing its initial span config record, it would carve itself off (and our last range would only extend to that next tenant's boundary), but this is a bit awkward for code that wants to rely on the invariant that ranges for a tenant only extend to the tenant's well-defined end key. So how do we ensure this split at this tenant's non-inclusive end key? Hard splits are controlled by the start keys on span config records[^1], so we'll try insert one accordingly. But we cannot do this blindly. We cannot assume that tenants are created in ID order, so it's possible that the tenant with the next ID is already present + running. If so, it may already have span config records that start at the key at which we want to write a span config record for. Over-writing it blindly would be a mistake -- there's no telling what config that next tenant has associated for that span. So we'll do something simple -- we'll check transactionally whether there's anything written already, and if so, do nothing. We already have the split we need. [^1]: See ComputeSplitKey in spanconfig.StoreReader. Release note: None
- Loading branch information
Showing
4 changed files
with
187 additions
and
9 deletions.
There are no files selected for viewing
115 changes: 115 additions & 0 deletions
115
pkg/ccl/spanconfigccl/spanconfigreconcilerccl/testdata/multitenant/tenant_end_key_split
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,115 @@ | ||
# Test that tenant creation does not overwrite span config state for another | ||
# tenant. | ||
|
||
reconcile | ||
---- | ||
|
||
mutations discard | ||
---- | ||
|
||
# Initialize a new tenant, tenant=11, that DOES NOT have a pre-existing tenant, | ||
# tenant=12, next to it. | ||
initialize tenant=11 | ||
---- | ||
|
||
# A record IS written for a key that logically belongs to the next tenant, | ||
# tenant=12, because tenant=12 DOES NOT exist. | ||
state offset=57 limit=3 | ||
---- | ||
... | ||
/Tenant/11{-\x00} database system (tenant) | ||
/Tenant/12{-\x00} database system (tenant) | ||
|
||
# Start the reconciliation loop for the tenant=11. It'll first clear out its own | ||
# first-key record and install span configs for its SQL state. It won't touch | ||
# the record we created at in the tenant=12 keyspace because it's not taught to | ||
# look there, and neither should it. The keyspace it's responsible for is its | ||
# own. | ||
reconcile tenant=11 | ||
---- | ||
|
||
# Peek near the start of the span_configurations table where tenant=11's records | ||
# are stored. The first one is from the start of its keyspace to start of | ||
# table with ID=4: /Tenant/11{-/Table/4}. | ||
state offset=56 limit=3 | ||
---- | ||
... | ||
/Table/5{7-8} ttl_seconds=3600 ignore_strict_gc=true num_replicas=5 rangefeed_enabled=true | ||
/Tenant/11{-/Table/4} database system (tenant) | ||
/Tenant/11/Table/{4-5} database system (tenant) | ||
... | ||
|
||
# Peek near the end of the span_configurations table where tenant=11's records | ||
# are stored. The last one is for its last system table: | ||
# /Tenant/11/Table/5{7-8}. Right now the split is at /Tenant/12. Which is fine. | ||
state offset=99 limit=3 | ||
---- | ||
... | ||
/Tenant/11/Table/5{7-8} database system (tenant) | ||
/Tenant/12{-\x00} database system (tenant) | ||
|
||
# Just another view of what the tenant's reconciler actually did. It got rid of | ||
# the original, single-key /Tenant/11{-\x00} record, and replaced it with | ||
# /Tenant/11{-/Table/4}, just like the comment above said. The remaining upserts | ||
# are for its SQL state. | ||
mutations tenant=11 limit=2 | ||
---- | ||
delete /Tenant/11{-\x00} | ||
upsert /Tenant/11{-/Table/4} database system (tenant) | ||
upsert /Tenant/11/Table/{4-5} database system (tenant) | ||
upsert /Tenant/11/Table/{5-6} database system (tenant) | ||
upsert /Tenant/11/Table/{6-7} database system (tenant) | ||
upsert /Tenant/11/Table/{7-8} database system (tenant) | ||
upsert /Tenant/11/Table/1{1-2} database system (tenant) | ||
upsert /Tenant/11/Table/1{2-3} database system (tenant) | ||
upsert /Tenant/11/Table/1{3-4} database system (tenant) | ||
upsert /Tenant/11/Table/1{4-5} database system (tenant) | ||
upsert /Tenant/11/Table/1{5-6} database system (tenant) | ||
upsert /Tenant/11/Table/{19-20} database system (tenant) | ||
upsert /Tenant/11/Table/2{0-1} database system (tenant) | ||
upsert /Tenant/11/Table/2{1-2} database system (tenant) | ||
upsert /Tenant/11/Table/2{3-4} database system (tenant) | ||
upsert /Tenant/11/Table/2{4-5} database system (tenant) | ||
upsert /Tenant/11/Table/2{5-6} database system (tenant) | ||
upsert /Tenant/11/Table/2{6-7} database system (tenant) | ||
upsert /Tenant/11/Table/2{7-8} database system (tenant) | ||
upsert /Tenant/11/Table/2{8-9} database system (tenant) | ||
upsert /Tenant/11/NamespaceTable/{30-Max} database system (tenant) | ||
upsert /Tenant/11/{NamespaceTable/Max-Table/32} database system (tenant) | ||
upsert /Tenant/11/Table/3{2-3} database system (tenant) | ||
upsert /Tenant/11/Table/3{3-4} database system (tenant) | ||
upsert /Tenant/11/Table/3{4-5} database system (tenant) | ||
upsert /Tenant/11/Table/3{5-6} database system (tenant) | ||
upsert /Tenant/11/Table/3{6-7} database system (tenant) | ||
upsert /Tenant/11/Table/3{7-8} database system (tenant) | ||
upsert /Tenant/11/Table/{39-40} database system (tenant) | ||
upsert /Tenant/11/Table/4{0-1} database system (tenant) | ||
upsert /Tenant/11/Table/4{1-2} database system (tenant) | ||
upsert /Tenant/11/Table/4{2-3} database system (tenant) | ||
upsert /Tenant/11/Table/4{3-4} database system (tenant) | ||
upsert /Tenant/11/Table/4{4-5} database system (tenant) | ||
upsert /Tenant/11/Table/4{6-7} database system (tenant) | ||
upsert /Tenant/11/Table/4{8-9} database system (tenant) | ||
upsert /Tenant/11/Table/5{0-1} database system (tenant) | ||
upsert /Tenant/11/Table/5{1-2} database system (tenant) | ||
upsert /Tenant/11/Table/5{2-3} database system (tenant) | ||
upsert /Tenant/11/Table/5{3-4} database system (tenant) | ||
upsert /Tenant/11/Table/5{4-5} database system (tenant) | ||
upsert /Tenant/11/Table/5{5-6} database system (tenant) | ||
upsert /Tenant/11/Table/5{6-7} database system (tenant) | ||
upsert /Tenant/11/Table/5{7-8} database system (tenant) | ||
|
||
# Initialize a new tenant, tenant=10, that DOES have a pre-existing tenant, | ||
# tenant=11, next to it. | ||
initialize tenant=10 | ||
---- | ||
|
||
# A record IS NOT written for a key that logically belongs to the next tenant, | ||
# tenant=11, because tenant=11 DOES exist. | ||
state offset=57 limit=3 | ||
---- | ||
... | ||
/Tenant/10{-\x00} database system (tenant) | ||
/Tenant/11{-/Table/4} database system (tenant) | ||
/Tenant/11/Table/{4-5} database system (tenant) | ||
... |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters