spanconfig,kv: merge adjacent ranges with identical configs

[irfansharif]

spanconfig,kv: merge adjacent ranges with identical configs

Fixes #72389.
Fixes #66063 (gated behind a cluster setting).

This should drastically reduce the total number of ranges in the system,
especially when running with a large number of tables and/or tenants. To
understand what the new set of split points are, consider the following
test snippet:

exec-sql tenant=11
CREATE DATABASE db;
CREATE TABLE db.t0();
CREATE TABLE db.t1();
CREATE TABLE db.t2();
CREATE TABLE db.t3();
CREATE TABLE db.t4();
CREATE TABLE db.t5();
CREATE TABLE db.t6();
CREATE TABLE db.t7();
CREATE TABLE db.t8();
CREATE TABLE db.t9();
ALTER TABLE db.t5 CONFIGURE ZONE using num_replicas = 42;
----

# If installing a unique zone config for a table in the middle, we
# should observe three splits (before, the table itself, and after).

diff offset=48
----
--- gossiped system config span (legacy)
+++ span config infrastructure (current)
...
 /Tenant/10                                 database system (tenant)
 /Tenant/11                                 database system (tenant)
+/Tenant/11/Table/106                       range default
+/Tenant/11/Table/111                       num_replicas=42
+/Tenant/11/Table/112                       range default

This PR introduces two cluster settings to selectively opt into this
optimization: spanconfig.{tenant,host}_coalesce_adjacent.enabled
(defaulting to true and false respectively). We also don't coalesce
system table ranges on the host tenant. We had a few implementation
choices here:

(a) Down in KV, augment the spanconfig.Store to coalesce the in-memory
state when updating entries. On every update, we'd check if the span
we're writing to has the same config as the preceding and/or
succeeding one, and if so, write out a larger span instead. We
previously prototyped a form of this in #68491.

Pros:
- reduced memory footprint of spanconfig.Store
- read path (computing splits) stays fast -- just have to look up
  the right span from the backing interval tree
Cons:
- uses more persisted storage than necessary
- difficult to disable the optimization dynamically (though still
  possible -- we'd effectively have to restart the KVSubscriber and
  populate in-memory span config state using a store that
  does/doesn't coalesce configs)
- difficult to implement; our original prototype did not have #73150
  which is important for reducing reconciliation round trips

(b) Same as (a) but coalesce configs up in the spanconfig.Store
maintained in reconciler itself.

Pros:
- reduced storage use (both persisted and in-memory)
- read path (computing splits) stays fast -- just have to look up
  the right span from the backing interval tree
Cons:
- very difficult to disable the optimization dynamically (each
  tenant process would need to be involved)
- most difficult to implement

(c) Same as (a) but through another API on the spanconfig.Store
interface that accepts only a single update at a time and does not
generate deltas (not something we need down in KV). Removes the
implementation complexity.

(d) Keep the contents of system.span_configurations and the in-memory
state of spanconfig.Stores as it is today, uncoalesced. When
determining split points, iterate through adjacent configs within
the provided key bounds and see whether we could ignore certain
split keys.

Pros:
- easiest to implement
- easy to disable the optimization dynamically, for ex. through a
  cluster setting
Cons:
- uses more storage (persisted and in-memory) than necessary
- read path (computing splits) is more expensive if iterating
  through adjacent configs

This PR implements option (d). For a benchmark on how slow (d) is going
to be in practice with varying numbers of entries to be scanning over
(10k, 100k, 1m):

    $ dev bench pkg/spanconfig/spanconfigstore -f=BenchmarkStoreComputeSplitKey -v
    BenchmarkStoreComputeSplitKey
    BenchmarkStoreComputeSplitKey/num-entries=10000
    BenchmarkStoreComputeSplitKey/num-entries=10000-10               1166842 ns/op
    BenchmarkStoreComputeSplitKey/num-entries=100000
    BenchmarkStoreComputeSplitKey/num-entries=100000-10             12273375 ns/op
    BenchmarkStoreComputeSplitKey/num-entries=1000000
    BenchmarkStoreComputeSplitKey/num-entries=1000000-10           140591766 ns/op
    PASS

It's feasible that in the future we re-work this in favor of (c).

---

spanconfig/store: use templatized btree impl instead

   $ dev bench pkg/spanconfig/spanconfigstore -f=BenchmarkStoreComputeSplitKey -v
    BenchmarkStoreComputeSplitKey
    BenchmarkStoreComputeSplitKey/num-entries=10000
    BenchmarkStoreComputeSplitKey/num-entries=10000-10                431093 ns/op
    BenchmarkStoreComputeSplitKey/num-entries=100000
    BenchmarkStoreComputeSplitKey/num-entries=100000-10              4308200 ns/op
    BenchmarkStoreComputeSplitKey/num-entries=1000000
    BenchmarkStoreComputeSplitKey/num-entries=1000000-10            43827373 ns/op
    PASS

    $ benchstat old.txt new.txt # from previous commit
    name                                         old time/op  new time/op  delta
    StoreComputeSplitKey/num-entries=10000-10    1.17ms ± 0%  0.43ms ± 0%   ~     (p=1.000 n=1+1)
    StoreComputeSplitKey/num-entries=100000-10   12.4ms ± 0%   4.3ms ± 0%   ~     (p=1.000 n=1+1)
    StoreComputeSplitKey/num-entries=1000000-10   136ms ± 0%    44ms ± 0%   ~     (p=1.000 n=1+1)

---

spanconfig/store: intern span configs

• Avoid the expensive proto.Equal() when computing split keys;
• Reduce memory overhead of the data structure

    $ dev bench pkg/spanconfig/spanconfigstore -f=BenchmarkStoreComputeSplitKey -v
    BenchmarkStoreComputeSplitKey
    BenchmarkStoreComputeSplitKey/num-entries=10000
    BenchmarkStoreComputeSplitKey/num-entries=10000-10                 90323 ns/op
    BenchmarkStoreComputeSplitKey/num-entries=100000
    BenchmarkStoreComputeSplitKey/num-entries=100000-10               915936 ns/op
    BenchmarkStoreComputeSplitKey/num-entries=1000000
    BenchmarkStoreComputeSplitKey/num-entries=1000000-10             9575781 ns/op

    $ benchstat old.txt new.txt # from previous commit
    name                                         old time/op  new time/op  delta
    StoreComputeSplitKey/num-entries=10000-10     431µs ± 0%    90µs ± 0%   ~     (p=1.000 n=1+1)
    StoreComputeSplitKey/num-entries=100000-10   4.31ms ± 0%  0.92ms ± 0%   ~     (p=1.000 n=1+1)
    StoreComputeSplitKey/num-entries=1000000-10  43.8ms ± 0%   9.6ms ± 0%   ~     (p=1.000 n=1+1)

Release note: None
Release justification: high benefit change to existing functionality
(affecting only multi-tenant clusters).

[cockroach-teamcity]

This change is 

[irfansharif]

This should be ready for review. There are some random out-of-package tests that need to be updated since they expect splits post-table creation (they were added after we introduced tenant range splits), and I also want to write a small benchmark for the read path. Like discussed offline, plan for this PR is to let it bake on master for a few days before backporting to 22.1, aiming for the .1 release. I also think this PR makes #78299 redundant (LMK if you disagree) -- so I'm just going to close that as well.

[irfansharif]

There are some random out-of-package tests that need to be updated since they expect splits post-table creation (they were added after we introduced tenant range splits), and I also want to write a small benchmark for the read path.

Done and done.

[BramGruneir]

Would it be possible to backport this to 22.1?

[irfansharif]

Yup.

plan for this PR is to let it bake on master for a few days before backporting to 22.1, aiming for the .1 release.

For single-tenant clusters it won't be enabled by default; there'll be an opt-in cluster setting.

[BramGruneir]

Awesome! Thanks.

[irfansharif]

@ajwerner, @nvanbenschoten and @arulajmani: Bump for reviews. Like discussed elsewhere, this PR should get baking time to build confidence. We're also aiming for this to be part of 22.1.1 when rolling out to multi-tenant clusters.

[craig]

Build succeeded:

• GitHub CI (Cockroach)

[theodore-hyman]

@irfansharif Thank you for your time and attention on this matter. I am chiming in just to confirm the timeline here as a quick sanity check.

Again, I know you just said that this would be backported to v22.1.1 and you just said:

I'll open the backport now but not merge to the release until some point mid next week

However just re-confirming, apologies for the redundancy.

My understanding is that this fix, in the form of a new hidden config option, will be released as part of v22.1.1 that is scheduled for public release June 6, 2022. Please confirm or let me know if I am misunderstanding. Thank you

[irfansharif]

The backport (#80811) was merged, so it'll make it to 22.1.1 whenever that goes out. That said, I don't know which customer you're referring to (DM?) and whether this would help. This PR wasn't intended as a fix for anything, so let's discuss elsewhere what we're expecting this to help with.