Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CloneVerb: Force download of commit and trees on failed initial checkout #313

Merged
merged 2 commits into from
Oct 3, 2018

Conversation

derrickstolee
Copy link
Contributor

On Clone, we need to create the .git/index file during a checkout, which requires having all trees reachable from the root tree of the tip commit. To ensure we have all of the objects, we call TryDownloadCommit(). However, this method doesn't actually try downloading a pack of trees if we already have the commit and root tree.

Due to the shared object cache, we may have these objects for a few reasons:

  1. A recent fetch included a ref update that points to that commit, and we downloaded the commit and tree as loose objects.
  2. A background prefetch downloaded a prefetch pack containing that commit and tree, but some of the reachable trees are missing due to clock skew between cache servers.

This fixes the issue by forcibly downloading the packfile if the checkout fails and trying again.

Resolves #306.

return true;
}

protected bool ForceTryDownloadCommit(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see anything being "forced" here. This is really referring to the other method, which first checks the cache. I would suggest just adding a skipLocalCacheCheck param to the existing method instead.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

GVFS/GVFS/CommandLine/CloneVerb.cs Outdated Show resolved Hide resolved
GVFS/GVFS/CommandLine/CloneVerb.cs Outdated Show resolved Hide resolved
GVFSGitObjects gitObjects,
out string error)
{
if (!gitObjects.TryDownloadCommit(commitId))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does the objects endpoint always respond with all of the trees reachable by commitId?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All trees reachable from its root tree, yes.

@wilbaker
Copy link
Member

Due to the shared object cache, we may have these objects for a few reasons:

  1. A recent fetch included a ref update that points to that commit, and we downloaded the commit and tree as loose objects.

I'm trying to make sure I understand this scenario. If this were to happen, is the problem that not all reachable trees were downloaded? Or is the issue that they were downloaded as loose objects?

@derrickstolee
Copy link
Contributor Author

Due to the shared object cache, we may have these objects for a few reasons:

  1. A recent fetch included a ref update that points to that commit, and we downloaded the commit and tree as loose objects.

I'm trying to make sure I understand this scenario. If this were to happen, is the problem that not all reachable trees were downloaded? Or is the issue that they were downloaded as loose objects?

The issue is that we don't have all reachable trees. When we ask for objects one-at-a-time, we don't close by reachability. But on clone time, we ask for and expect a reachable closure of trees. Our condition of checking just the commit and root tree is insufficient for this case.

@derrickstolee derrickstolee force-pushed the checkout-gaps branch 3 times, most recently from c7e4cdb to c9e3c2b Compare October 2, 2018 13:39
string[] packDirFiles = Directory.EnumerateFiles(packDir, "*", SearchOption.AllDirectories).ToArray();
for (int i = 0; i < packDirFiles.Length; i++)
{
this.fileSystem.DeleteFile(Path.Combine(packDir, packDirFiles[i]));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your other option would be to use a new cache location for this test case only, and clone with --no-prefetch

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll use a new location.

// It is possible to have the above TryDownloadCommit() fail because we
// already have the commit and root tree we intend to check out, but
// don't have a tree further down the working directory. If we fail
// checkout here, it may be due to not having these trees and the
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's no longer "may be" since you're now checking the specific error

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@@ -401,9 +401,10 @@ protected void ValidatePathParameter(string path)
GitObjectsHttpRequestor objectRequestor,
GVFSGitObjects gitObjects,
GitRepo repo,
out string error)
out string error,
bool ignoreIfRootExists = true)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This name is confusing, and results in a double-negative check down below with !ignore. Rename it to checkLocalObjectCache?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

gvfs clone may fail due to prefetch packfile "gaps"
3 participants