[docdb][2dc] Integration of 2DC with tablet splitting

[bmatican]

Notes from @rahuldesirazu on potential cheap path forward

• The consumer CM has to subscribe to tablet splits on the producer side (either through a HB mechanism or through some push based mechanic from the producer).
• The consumer LB needs to have a path both for adding recently split tablets to the cdc map and moving around tablets in the map to keep polling work balanced.

[abigalekim]

Implementation design decision notes:
Where in the codebase are we updating the tablet mapping?

• The tablet mapping should be updated within the consumer TServer nodes.
• The node that updates the mapping should then write a SPLIT_OP to the consumer master’s WAL.
• When the consumer master reads this SPLIT_OP, then we update the mapping via the CDCPoller
• This could potentially be a source of latency, but because the main operation with splitting at the TServer level is that it creates hard links at the RockDB level, then copies the Raft Log index file (this is a fast operation, taking less than a millisecond), it should be fine in terms of latency.
• An additional design decision we need to consider is how to tell whether we are reading from a split tablet that hasn’t been updated in master yet
• We can solve this by keeping a metric to keep maximum staleness, take note if we’re ever reading from a split tablet given split timestamp, then we take this timestamp and subtract it from previous timestamp

We can create an UpdateTabletMapping function that is called in the CatalogManager, similar to CreateTabletMapping.
It’s currently called UpdateCDCConsumer and is in ent/catalog_manager module.

[nspiegelberg]

Duplicate of #10175

[hulien22]

Closing in favour of #10175