kvserver: request option to prevent splitting rows in responses

[erikgrinaker]

SQL rows can be split across multiple KV pairs. During range splits, we take this into account at the MVCC level to prevent splitting a row across multiple ranges by decoding the KV pairs:

cockroach/pkg/storage/mvcc.go

Lines 3881 to 3892 in 2e038a1

	if _, tenID, err := keys.DecodeTenantPrefix(it.UnsafeKey().Key); err == nil {
	if _, _, err := keys.MakeSQLCodec(tenID).DecodeTablePrefix(it.UnsafeKey().Key); err == nil {
	// The first key in this range represents a row in a SQL table. Advance the
	// minSplitKey past this row to avoid the problems described above.
	firstRowKey, err := keys.EnsureSafeSplitKey(it.Key().Key)
	if err != nil {
	return nil, err
	}
	// Allow a split key before other rows in the same table.
	minSplitKey = firstRowKey.PrefixEnd()
	}
	}

However, when doing limited scans (with MaxSpanRequestKeys or TargetBytes limits), the response can end up returning a partial row, causing the client to either have to discard it or run another request to complete the row. This is particularly relevant for the streaming client API in #68430.

We should add a Header parameter that prevents these limits from splitting rows. Since these limits are now strict limits (i.e. we will discard the last value that exceeds TargetBytes), this implies that we'll discard these partial rows to keep below the limits unless it is the first row in the result. This needs to be plumbed down into MVCC where the limits are applied:

cockroach/pkg/storage/pebble_mvcc_scanner.go

Lines 714 to 719 in 0ee3bbd

	// Check if we actually exceeded the limit.
	if size >= p.targetBytes || (p.targetBytesAvoidExcess && size+nextSize > p.targetBytes) {
	p.curExcluded = true
	p.resumeReason = roachpb.RESUME_BYTE_LIMIT
	return false
	}

Note that even though this option will currently only apply to Scan and ReverseScan requests, we add it to Header since that is where the related limits are configured.