Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature Request: support changing primary Vindex in MoveTables #13147

Closed
maxenglander opened this issue May 24, 2023 · 0 comments · Fixed by #13782
Closed

Feature Request: support changing primary Vindex in MoveTables #13147

maxenglander opened this issue May 24, 2023 · 0 comments · Fixed by #13782

Comments

@maxenglander
Copy link
Collaborator

Feature Description

When using MoveTables to move data from one sharded keyspace to another, it would be great if you could have the source and target keyspaces have different primary Vindexes.

Currently, Vitess lets you do this, but the results end up being wrong because of the way the target workflow tablet selection works.

For example, given two sharded keyspaces with the same shard spec -80,80- but different primary Vindexes, MoveTables will create VReplication tasks in the target shards which stream only the corresponding source shards. This means that some data may live in source shard -80 which needs to go to target 80-, but won't.

Use Case(s)

When migrating to Vitess, users may have a custom manual sharding setup. Ideally, they would be able to migrate to Vitess by setting up a source keyspace with shards corresponding to the shards of their manual setup, and then use MoveTables to move that data into a target keyspace. (I am hand-waving the challenge of getting Vitess to route queries to the right source shard.) They might want the target keyspace to use a Vitess-native Vindex like hash, rather than whatever custom sharding strategy they employ in the source keyspace.

Another use case is if a user is already in Vitess, and finds out that their primary Vindex is suboptimal for some reason. Perhaps it results in hot shards. They would like to change the primary Vindex to get better performance, data distribution, etc.

@maxenglander maxenglander added Type: Feature Needs Triage This issue needs to be correctly labelled and triaged labels May 24, 2023
@deepthi deepthi added Component: VReplication and removed Needs Triage This issue needs to be correctly labelled and triaged labels Jun 2, 2023
@maxenglander maxenglander self-assigned this Aug 14, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants