Commits on Jan 20, 2021

1. kvserver: avoid engine bypass in sideload storage …

   While investigating cockroachdb#59126, I realized we were still going through the
   engine for listing the files. I rectified this and as a result,
   completely removed the in-mem implementation of the sideloaded storage
   that we had in place: instead of it, we simply use an in-mem engine,
   which now works like a charm thanks to not bypassing the engine for
   listing the file contents (`filepath.Glob` --> `eng.List`).
   
   While I was here, I added some defense in depth: it was unreasonable
   that `TruncateTo` was returning an error as a result of failing to
   perform best-effort cleanup on its directory.
   
   I am no longer seeing the spurious errors observed in cockroachdb#59126 with
   this change. This makes sense, as in this test the sideloaded storage
   is set up with an in-memory engine. The old code, which read from
   actual disk, would try to glob `auxiliary/...` (i.e. some path relative
   to the cwd - oops), which would return no results. As a result, the
   sideloaded storage would try to invoke `eng.Remove("auxiliary/...")`
   which the engine would refuse, since there would actually be (in-mem)
   files stored in the engine. This all resolves now that everyone is
   looking at the same files.
   
   It's possible that this fixes the issue with BenchmarkUserFile, as
   previously errors from TruncateTo could block the raft log queue,
   which in turn could at least slow down the test, but it's hard to
   say for sure.
   
   Touches cockroachdb#59126.
   Touches cockroachdb#31913.
   
   Release note: None

   

   tbg committed Jan 20, 2021
   Configuration menu

   • View commit details
   • Copy full SHA for 16638bc
   • Browse repository at this point

   Copy the full SHA

   16638bc View commit details

   Browse the repository at this point in the history