You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When using MoveTables to move data from one sharded keyspace to another, it would be great if you could have the source and target keyspaces have different primary Vindexes.
Currently, Vitess lets you do this, but the results end up being wrong because of the way the target workflow tablet selection works.
For example, given two sharded keyspaces with the same shard spec -80,80- but different primary Vindexes, MoveTables will create VReplication tasks in the target shards which stream only the corresponding source shards. This means that some data may live in source shard -80 which needs to go to target 80-, but won't.
Use Case(s)
When migrating to Vitess, users may have a custom manual sharding setup. Ideally, they would be able to migrate to Vitess by setting up a source keyspace with shards corresponding to the shards of their manual setup, and then use MoveTables to move that data into a target keyspace. (I am hand-waving the challenge of getting Vitess to route queries to the right source shard.) They might want the target keyspace to use a Vitess-native Vindex like hash, rather than whatever custom sharding strategy they employ in the source keyspace.
Another use case is if a user is already in Vitess, and finds out that their primary Vindex is suboptimal for some reason. Perhaps it results in hot shards. They would like to change the primary Vindex to get better performance, data distribution, etc.
The text was updated successfully, but these errors were encountered:
Feature Description
When using
MoveTables
to move data from one sharded keyspace to another, it would be great if you could have the source and target keyspaces have different primary Vindexes.Currently, Vitess lets you do this, but the results end up being wrong because of the way the target workflow tablet selection works.
For example, given two sharded keyspaces with the same shard spec
-80,80-
but different primary Vindexes,MoveTables
will create VReplication tasks in the target shards which stream only the corresponding source shards. This means that some data may live in source shard-80
which needs to go to target80-
, but won't.Use Case(s)
When migrating to Vitess, users may have a custom manual sharding setup. Ideally, they would be able to migrate to Vitess by setting up a source keyspace with shards corresponding to the shards of their manual setup, and then use
MoveTables
to move that data into a target keyspace. (I am hand-waving the challenge of getting Vitess to route queries to the right source shard.) They might want the target keyspace to use a Vitess-native Vindex likehash
, rather than whatever custom sharding strategy they employ in the source keyspace.Another use case is if a user is already in Vitess, and finds out that their primary Vindex is suboptimal for some reason. Perhaps it results in hot shards. They would like to change the primary Vindex to get better performance, data distribution, etc.
The text was updated successfully, but these errors were encountered: