kvserver: add storage time-series metrics for level size and score

[nicktrav]

Currently, the only way to infer the compaction score and heuristics is to use the LSM printout from the logs (emitted once every ten minutes), or to call the /debug/lsm endpoint manually, and track values over time. This makes it difficult to debug issues retroactively.

Add two new sets of per-LSM-level time-series metrics for level size and level score. These new metrics have names of the form storage.$LEVEL-level-{size,score}.

Closes #88415.

Release note (ops change): Adds two new sets of per-LSM-level time-series metrics, one for level size and another for level score. These metrics are of the form storage.$LEVEL-level-{size,score}.

[cockroach-teamcity]

This change is 

[nicktrav]

Example of how these can be used:

[sumeerbhola]

Reviewed 2 of 2 files at r1, all commit messages.
Reviewable status: complete! 1 of 0 LGTMs obtained (waiting on @jbowens and @nicktrav)

---

pkg/util/metric/metric.proto line 54 at r1 (raw file):

  // UNITLESS expresses that the metric's measurement does not have units (e.g.
  // a score).
  UNITLESS = 9;

Is adding an enum value here sufficient, or does it need some plumbing elsewhere too?
btw, COUNT does not have units either, and we seem to use it both for gauge and cumulative.
Is the assumption that COUNT is for integers while this UNITLESS value should be used for floats? I think it's worth clarifying via code comment how one should decide between COUNT and UNITLESS.

[nicktrav]

Reviewable status: complete! 0 of 0 LGTMs obtained (and 1 stale) (waiting on @jbowens and @sumeerbhola)

---

pkg/util/metric/metric.proto line 54 at r1 (raw file):

COUNT does not have units either, and we seem to use it both for gauge and cumulative.

Reverted to just use COUNT.

[jbowens]

Nice!

Reviewed 3 of 3 files at r2, all commit messages.
Reviewable status: complete! 0 of 0 LGTMs obtained (and 1 stale) (waiting on @sumeerbhola)

[nicktrav]

TFTRs!

bors r=sumeerbhola,jbowens

[craig]

Build succeeded:

• Bazel Essential CI (Cockroach)

[blathers-crl]

Encountered an error creating backports. Some common things that can go wrong:

1. The backport branch might have already existed.
2. There was a merge conflict.
3. The backport branch contained merge commits.

You might need to create your backport manually using the backport tool.

---

error creating merge commit from d41cce0 to blathers/backport-release-22.1-88504: POST https://api.github.com/repos/cockroachdb/cockroach/merges: 409 Merge conflict []

you may need to manually resolve merge conflicts with the backport tool.

Backport to branch 22.1.x failed. See errors above.

---

_{🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is otan.}