[docdb] Splitting: Disable automatic splitting for 2DC enabled tables

[bmatican]

After we have #8234, it should be very easy to expand the set of constraints on what to enable/disable splitting on.

Until we have #5373, we should at least disable automatic splitting on such tables.

[nspiegelberg]

Thoughts:

1. We should add this as a step to 'setup_universe_replication'.
2. This should ensure that both the Producer and Consumer have splitting disabled (if we have each cluster responsible for disabling, we could have mixed version issues).
3. I think we can make this stateless (e.g. don't worry about re-enable logic if we delete_universe_replication), just have the user manually enable it.

[bmatican]

@nspiegelberg Can you elaborate a bit more on 1) do you mean disable splitting as part of replication setup, via an extra flag in the command, or just automatically set some property which disables splitting?

For 2) maybe I'm missing something, but when you say mixed versions, are you worried about

• Producer is on version X different from consumer on version Y
• OR that during a software upgrade on either side, we'd have different versions?

What I was thinking, for version compatibility, is that whatever fix we generate here, will be in the version that enables tablet splitting by default. So there would only be 2 states of the world

• Old: Cluster without tablet splitting -- nothing to worry about
• New: Cluster with tablet splitting -- should include this fix, thus this particular cluster won't do splitting for xCluster tables

I was kind of hoping for something stateless AND not requiring new explicit user intervention, eg: since the master (CatalogManager) is responsible for whether or not to trigger a split for a tablet, if it knew that this tablet belongs to a table which is either a consumer or a producer in xCluster, then that would be a very cheap and self-contained fix.

[nspiegelberg]

My original solution is to just have the CLI command also handle a splitting disable on the tables involved in the command. You're suggesting a solution that checks the in-memory table state at split request time. For that:

1. Consumer : See ConsumerRegistryPB. It is forwarded to the Consumer Tablets on heartbeats and tells which tablets need CDCPollers setup to scrape data from the Producers.
2. Producer : See SysCDCStreamEntryPB. This is less straightforward. It is a mapping of stream_id:table_id. We could probably add some logic to scrape this on startup. Note that this applies to all CDC streams, not just 2DC. There is a CDCStreamOptionsPB that records the consumer source type.