storage: suggest compaction when replicas are deleted

[spencerkimball]

In aggregate, if enough contiguous replicas are migrated away from a
range, we would like to compact the underlying RocksDB storage engine.
To that end, on clear a replica's data, we suggest a compaction.

Release note: None

[cockroach-teamcity]

This change is 

[tbg]

---

Reviewed 5 of 5 files at r1.
Review status: all files reviewed at latest revision, 2 unresolved discussions, all commit checks successful.

---

pkg/storage/client_raft_test.go, line 3898 at r1 (raw file):

	testutils.SucceedsSoon(t, func() error {
		// Check for a compaction metric indicating non-zero bytes are queued.
		if c := mtc.stores[2].Compactor().Metrics.BytesQueued.Value(); c == 0 {

The compaction could also be processed (maybe not in today's code, but in principle), so check there's something queued or processed or skipped.

---

pkg/storage/client_raft_test.go, line 3904 at r1 (raw file):

		// other stores.
		for i := 0; i < 2; i++ {
			if c := mtc.stores[i].Compactor().Metrics.BytesQueued.Value(); c != 0 {

Ditto.

---

Comments from Reviewable

[spencerkimball]

Review status: 4 of 5 files reviewed at latest revision, 2 unresolved discussions.

---

pkg/storage/client_raft_test.go, line 3898 at r1 (raw file):

Previously, tschottdorf (Tobias Schottdorf) wrote…

The compaction could also be processed (maybe not in today's code, but in principle), so check there's something queued or processed or skipped.

Done.

---

pkg/storage/client_raft_test.go, line 3904 at r1 (raw file):

Previously, tschottdorf (Tobias Schottdorf) wrote…

Ditto.

Done.

---

Comments from Reviewable

[tbg]

Reviewed 1 of 1 files at r2.
Review status: all files reviewed at latest revision, all discussions resolved.

---

Comments from Reviewable

[spencerkimball]

@tschottdorf note that in the process of testing this with an actual migration, I noticed some other problems, which I've updated this commit to fix. PTAL.

[bdarnell]

---

Review status: 3 of 5 files reviewed at latest revision, all discussions resolved, some commit checks failed.

---

Comments from Reviewable

[tbg]

---

Reviewed 2 of 2 files at r3.
Review status: all files reviewed at latest revision, all discussions resolved, some commit checks failed.

---

Comments from Reviewable