gvfs clone may fail due to prefetch packfile "gaps"

[derrickstolee]

The prefetch packfiles use timestamps to signal "this is what the cache server knew about at this time". When a client prefetches, they contact a cache server with their latest timestamp and the server responds with a list of packfiles. As we call different cache servers, we can get into a state where there are "gaps": some commits and trees never exist in a prefetch pack.

In a normal enlistment, these gaps can be refilled on-demand. However, at clone time we may be directed to an existing shared object cache, see that the cache has the commit and root tree that we expect, but be missing a tree that is reachable from that root tree. As we attempt to run a git checkout, we will fail to find that object and we don't have the read-object hook available.

This is a very odd data shape that has always been possible, but it is more likely to be hit now that we have background prefetch (we are more likely to satisfy CommitAndRootTreeExists() for the latest ref).

One fix is to run git rev-list --objects <root-tree-hash> instead of just checking if the objects exist. If that command succeeds, then we can continue without downloading the initial packfile. This will take less time than the git checkout step, so may be reasonable.

Another fix is to always download that initial packfile. This is likely not the best solution because we rarely need that data a second time, and we would be slowing down clones that are usually "trivial".

[sanoursa]

@derrickstolee I don't think prefetch is the cause here. Clone already tries to download the tip commit (plus full frees) for the branch it's going to checkout, because there's no guarantee that the prefetch packs will have the commit we want, so this is a data shape we already tried to account for. See https://github.com/Microsoft/VFSForGit/blob/master/GVFS/GVFS/CommandLine/CloneVerb.cs#L520

However, the issue is the implementation of that method makes a big assumption - if the commit object and its root tree object exist locally, then we probably also have all the rest of the trees, and skip downloading anything. See https://github.com/Microsoft/VFSForGit/blob/master/GVFS/GVFS/CommandLine/GVFSVerb.cs#L406

So we could fix this in a couple of ways - 1) always download that tip commit+trees or 2) detect the missing tree errors from our initial git checkout, and if we see that, fall back to downloading and retry.

[sanoursa]

My point is, the issue is not with prefetch. Prefetch contains full trees, so background prefetch is not at fault here. The root cause is that an existing repo could touch just the root tree object, not download the full set of trees, and then a subsequent clone attempt will fail in this manner.