Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

kv,sql: change {system.zones,span_configurations} to honor fine grained data domiciliation rules pertaining to indexed values #70913

Closed
knz opened this issue Sep 30, 2021 · 2 comments
Labels
C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) no-issue-activity T-kv KV Team X-stale

Comments

@knz
Copy link
Contributor

knz commented Sep 30, 2021

Describe the problem

When configuring zones for table partitions, the zone configs themselves are stored in a single range in system.zones. The zone configs contain partition prefixes from the tables, and thus region-sensitive data, and so that data escapes the region that it should remain in.

This makes it impossible to do strict data sovereignty partitioning using multi-region CockroachDB when domicilied data is indexed. (The issue does not exist when domicilied data is not indexed.)

Note: we already document this limitation in https://www.cockroachlabs.com/docs/stable/data-domiciling.html#limitations

Epic: CRDB-10287

To Reproduce

  1. create a table and split it into multiple region via zone configs
  2. dump the binary content of system.zones: observe that the region-specific prefixes are stored in the common range

Expected behavior

Any zone config binary data that includes a copy from region-specific table data should be stored in the same region as that table's data.

Environment:

crdb v21.2

Jira issue: CRDB-10284

@knz knz added the C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. label Sep 30, 2021
@knz
Copy link
Contributor Author

knz commented Sep 30, 2021

cc @awoods187 @mwang1026 you may want to track this for the GDPR roadmap.

cc @irfansharif since you're working on something similar. The span configs are subject to the same problem currently.

@blathers-crl blathers-crl bot added the T-kv KV Team label Sep 30, 2021
@knz knz added C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) and removed C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. labels Sep 30, 2021
@knz knz changed the title kv,sql: data in zone configurations violate data domiciliation rules kv,sql: data in zone configurations violate data domiciliation rules pertaining to indexed values Sep 30, 2021
@knz knz changed the title kv,sql: data in zone configurations violate data domiciliation rules pertaining to indexed values kv,sql: data in zone configurations does not honor data domiciliation rules pertaining to indexed values Sep 30, 2021
@knz knz changed the title kv,sql: data in zone configurations does not honor data domiciliation rules pertaining to indexed values kv,sql: change zone configuration storage to honor data domiciliation rules pertaining to indexed values Sep 30, 2021
@knz knz changed the title kv,sql: change zone configuration storage to honor data domiciliation rules pertaining to indexed values kv,sql: change zone configuration storage to honor fine grained data domiciliation rules pertaining to indexed values Sep 30, 2021
@irfansharif irfansharif changed the title kv,sql: change zone configuration storage to honor fine grained data domiciliation rules pertaining to indexed values kv,sql: change {system.zones,span_configurations} to honor fine grained data domiciliation rules pertaining to indexed values Sep 30, 2021
@irfansharif irfansharif changed the title kv,sql: change {system.zones,span_configurations} to honor fine grained data domiciliation rules pertaining to indexed values kv,sql: change {system.zones,span_configurations} to honor fine grained data domiciliation rules pertaining to indexed values Sep 30, 2021
@github-actions
Copy link

We have marked this issue as stale because it has been inactive for
18 months. If this issue is still relevant, removing the stale label
or adding a comment will keep it active. Otherwise, we'll close it in
10 days to keep the issue queue tidy. Thank you for your contribution
to CockroachDB!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) no-issue-activity T-kv KV Team X-stale
Projects
No open projects
Archived in project
Development

No branches or pull requests

1 participant