mvcc: chunk reads for restoring

[heyitsanthony]

Loading all keys at once would cause etcd to use twice as much
memory than it would need to serve the keys, causing RSS to spike on
boot. Instead, load the keys into the mvcc by chunk. Uses pipelining
for some concurrency.

Fixes #7822

tmpfs benchmarks:

Before:

go test -v -run=BenchmarkStoreRestore -bench=BenchmarkStoreRestore
BenchmarkStoreRestoreRevs1-8     1000000      4446 ns/op     814 B/op      7 allocs/op
BenchmarkStoreRestoreRevs10-8     200000     11053 ns/op    5397 B/op     38 allocs/op
BenchmarkStoreRestoreRevs20-8     100000     17458 ns/op   10518 B/op     69 allocs/op

After:

go test -bench=BenchmarkStoreRestore -run=BenchmarkStoreRestore
BenchmarkStoreRestoreRevs1-8    	 1000000	      2464 ns/op	     641 B/op	       6 allocs/op
BenchmarkStoreRestoreRevs10-8   	  200000	      7396 ns/op	    4924 B/op	      37 allocs/op
BenchmarkStoreRestoreRevs20-8   	  100000	     13664 ns/op	    9715 B/op	      68 allocs/op

[gyuho]

lgtm. /cc @xiang90

[xiang90]

LGTM. Thanks.

[codecov-io]

Codecov Report

❗ No coverage uploaded for pull request base (master@47f5b7c). Click here to learn what that means.
The diff coverage is 89.79%.

@@            Coverage Diff            @@
##             master    #7900   +/-   ##
=========================================
  Coverage          ?   75.82%           
=========================================
  Files             ?      332           
  Lines             ?    26332           
  Branches          ?        0           
=========================================
  Hits              ?    19966           
  Misses            ?     4938           
  Partials          ?     1428

Impacted Files	Coverage Δ
mvcc/kvstore.go	87.37% <89.79%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 47f5b7c...1aca63e. Read the comment docs.

[davissp14]

Hey guys, any way we could get this released with 3.2?

[heyitsanthony]

@davissp14 can you confirm this fixes the RSS spike?

[davissp14]

I'll give this a try this afternoon.

[davissp14]

Just confirming this did fix the RSS spike. Thanks guys.

Also any thoughts on bringing this into 3.2 release?

[heyitsanthony]

OK, I'm fine with moving this up to 3.2.

@gyuho @xiang90 any objections to slipping this into 3.2?

[gyuho]

No objection from me. @xiang90 ?

[xiang90]

No objection.