flow based tablet load balancer (#16351)

Signed-off-by: Michael Demmer <mdemmer@slack-corp.com>
Signed-off-by: Venkatraju V <venkatraju@slack-corp.com>
Signed-off-by: Venkatraju <venkatraju@gmail.com>
Co-authored-by: Venkatraju V <venkatraju@slack-corp.com>
Co-authored-by: Venkatraju <venkatraju@gmail.com>

changelog/21.0/21.0.0/summary.md

@@ -13,6 +13,7 @@
   - **[Tablet Throttler: Multi-Metric support](#tablet-throttler)**
   - **[Allow Cross Cell Promotion in PRS](#allow-cross-cell)**
   - **[Support for recursive CTEs](#recursive-cte)**
+  - **[VTGate Tablet Balancer](#tablet-balancer)**
 
 ## <a id="major-changes"/>Major Changes
 
@@ -105,4 +106,11 @@ Up until now if the users wanted to promote a replica in a different cell than t
 We have now added a new flag `--allow-cross-cell-promotion` that lets `PlannedReparentShard` choose a primary in a different cell even if no new primary is provided explicitly.
 
 ### <a id="recursive-cte"/>Experimental support for recursive CTEs
-We have added experimental support for recursive CTEs in Vitess. We are marking it as experimental because it is not yet fully tested and may have some limitations. We are looking for feedback from the community to improve this feature.
+We have added experimental support for recursive CTEs in Vitess. We are marking it as experimental because it is not yet fully tested and may have some limitations. We are looking for feedback from the community to improve this feature.
+
+### <a id="tablet-balancer"/>VTGate Tablet Balancer
+When a VTGate routes a query and has multiple available tablets for a given shard / tablet type (e.g. REPLICA), the current default behavior routes the query with local cell affinity and round robin policy. The VTGate Tablet Balancer provides an alternate mechanism that routes queries to maintain an even distribution of query load to each tablet, while preferentially routing to tablets in the same cell as the VTGate.
+
+The tablet balancer is enabled by a new flag `--enable-balancer` and configured by `--balancer-vtgate-cells` and `--balancer-keyspaces`.
+
+See [RFC for details](https://github.com/vitessio/vitess/issues/12241).

go/flags/endtoend/vtgate.txt

@@ -28,6 +28,8 @@ Flags:
       --allow-kill-statement                                             Allows the execution of kill statement
       --allowed_tablet_types strings                                     Specifies the tablet types this vtgate is allowed to route queries to. Should be provided as a comma-separated set of tablet types.
       --alsologtostderr                                                  log to standard error as well as files
+      --balancer-keyspaces strings                                       When in balanced mode, a comma-separated list of keyspaces for which to use the balancer (optional)
+      --balancer-vtgate-cells strings                                    When in balanced mode, a comma-separated list of cells that contain vtgates (required)
       --bind-address string                                              Bind address for the server. If empty, the server will listen on all available unicast and anycast IP addresses of the local system.
       --buffer_drain_concurrency int                                     Maximum number of requests retried simultaneously. More concurrency will increase the load on the PRIMARY vttablet when draining the buffer. (default 1)
       --buffer_keyspace_shards string                                    If not empty, limit buffering to these entries (comma separated). Entry format: keyspace or keyspace/shard. Requires --enable_buffer=true.
@@ -53,6 +55,7 @@ Flags:
       --discovery_high_replication_lag_minimum_serving duration          Threshold above which replication lag is considered too high when applying the min_number_serving_vttablets flag. (default 2h0m0s)
       --discovery_low_replication_lag duration                           Threshold below which replication lag is considered low enough to be healthy. (default 30s)
       --emit_stats                                                       If set, emit stats to push-based monitoring and stats backends
+      --enable-balancer                                                  Enable the tablet balancer to evenly spread query load for a given tablet type
       --enable-partial-keyspace-migration                                (Experimental) Follow shard routing rules: enable only while migrating a keyspace shard by shard. See documentation on Partial MoveTables for more. (default false)
       --enable-views                                                     Enable views support in vtgate.
       --enable_buffer                                                    Enable buffering (stalling) of primary traffic during failovers.