Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gvfs-helper: fix race condition when creating loose object dirs #205

Conversation

derrickstolee
Copy link
Collaborator

When two gvfs-helper processes are the first to create a loose object
directory, the processes (A and B in the timeline below) could have
the following race:

  1. A sees that the directory does not exist.
  2. B sees that the directory does not exist.
  3. A creates the directory with success.
  4. B fails to create the directory and fails.

Instead of having B fail here, just check for the directory's
existence before reporting an error. That solves the race and
allows tests to pass.

When two gvfs-helper processes are the first to create a loose object
directory, the processes (A and B in the timeline below) could have
the following race:

1. A sees that the directory does not exist.
2. B sees that the directory does not exist.
3. A creates the directory with success.
4. B fails to create the directory and fails.

Instead of having B fail here, just check for the directory's
existence before reporting an error. That solves the race and
allows tests to pass.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
@@ -1049,7 +1049,9 @@ static void create_tempfile_for_loose(
strbuf_complete(&buf_path, '/');
strbuf_add(&buf_path, hex, 2);

if (!file_exists(buf_path.buf) && mkdir(buf_path.buf, 0777) == -1) {
if (!file_exists(buf_path.buf) &&
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A bit unrelated, do you know if there is a function dir_exists that checks if the path exists as a directory (or should we add one to use here)?

I couldn't find one in dir.c, but I imagine all it would need is:

int dir_exists(const char *f)
{
	struct stat sb;
	if (lstat(f, &sb) != 0)
		return false;

	return S_ISDIR(sb.st_mode);
}

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The following exists in builtin/clone.c:

static int dir_exists(const char *path)
{
	struct stat sb;
	return !stat(path, &sb);
}

It doesn't check the S_ISDIR().

I do see your interest in checking the type to make sure there isn't a file in the way. I'm not sure if there is enough value.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if there is enough value.

Yeah I was on the fence as well, we've not seen this be a problem thus far.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. Thanks!
I'll fix it Monday. I think I can just try the mkdir() and catch the EEXIST error code.
I avoided that because of the platform nature of the error codes.
(or just try the lstat() again inside the error case.)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please note that there is not only a dir_exists(), but also an is_directory() function (the latter of which is public). There is even a ticket to reconcile them: gitgitgadget#230

Copy link
Member

@wilbaker wilbaker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved with suggestion

@derrickstolee derrickstolee merged commit db6fa76 into microsoft:features/sparse-checkout-2.23.0 Oct 4, 2019
derrickstolee added a commit to microsoft/scalar that referenced this pull request Oct 5, 2019
…gvfs-helper

Resolves #156.

See microsoft/git#205 for the Git code change. One race condition still existed: who creates the loose object _directory_ first? The simple change is to check if the directory exists after the mkdir fails.
@jeffhostetler
Copy link

@derrickstolee Thanks for taking care of this!!

derrickstolee pushed a commit that referenced this pull request Jun 1, 2020
Includes commits from these pull requests:

	#191
	#205
	#206
	#207
	#208
	#215
	#220
	#221

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
derrickstolee pushed a commit that referenced this pull request Jul 20, 2020
Includes commits from these pull requests:

	#191
	#205
	#206
	#207
	#208
	#215
	#220
	#221

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
derrickstolee pushed a commit that referenced this pull request Jul 27, 2020
Includes commits from these pull requests:

	#191
	#205
	#206
	#207
	#208
	#215
	#220
	#221

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
derrickstolee pushed a commit that referenced this pull request Jul 28, 2020
Includes commits from these pull requests:

	#191
	#205
	#206
	#207
	#208
	#215
	#220
	#221

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
derrickstolee pushed a commit that referenced this pull request Oct 6, 2020
Includes commits from these pull requests:

	#191
	#205
	#206
	#207
	#208
	#215
	#220
	#221

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
derrickstolee pushed a commit that referenced this pull request Oct 6, 2020
Includes commits from these pull requests:

	#191
	#205
	#206
	#207
	#208
	#215
	#220
	#221

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
dscho pushed a commit that referenced this pull request Oct 6, 2020
Includes commits from these pull requests:

	#191
	#205
	#206
	#207
	#208
	#215
	#220
	#221

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
derrickstolee pushed a commit that referenced this pull request Oct 9, 2020
Includes commits from these pull requests:

	#191
	#205
	#206
	#207
	#208
	#215
	#220
	#221

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
derrickstolee pushed a commit that referenced this pull request Oct 9, 2020
Includes commits from these pull requests:

	#191
	#205
	#206
	#207
	#208
	#215
	#220
	#221

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
derrickstolee pushed a commit that referenced this pull request Oct 16, 2020
Includes commits from these pull requests:

	#191
	#205
	#206
	#207
	#208
	#215
	#220
	#221

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
derrickstolee pushed a commit that referenced this pull request Oct 19, 2020
Includes commits from these pull requests:

	#191
	#205
	#206
	#207
	#208
	#215
	#220
	#221

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
mjcheetham pushed a commit that referenced this pull request Dec 15, 2020
Includes commits from these pull requests:

	#191
	#205
	#206
	#207
	#208
	#215
	#220
	#221

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
mjcheetham pushed a commit that referenced this pull request Dec 15, 2020
Includes commits from these pull requests:

	#191
	#205
	#206
	#207
	#208
	#215
	#220
	#221

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
dscho pushed a commit that referenced this pull request Dec 18, 2020
Includes commits from these pull requests:

	#191
	#205
	#206
	#207
	#208
	#215
	#220
	#221

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
dscho pushed a commit that referenced this pull request Dec 24, 2020
Includes commits from these pull requests:

	#191
	#205
	#206
	#207
	#208
	#215
	#220
	#221

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
dscho pushed a commit that referenced this pull request Dec 28, 2020
Includes commits from these pull requests:

	#191
	#205
	#206
	#207
	#208
	#215
	#220
	#221

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
dscho pushed a commit that referenced this pull request Mar 4, 2021
Includes commits from these pull requests:

	#191
	#205
	#206
	#207
	#208
	#215
	#220
	#221

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
vdye pushed a commit that referenced this pull request Jul 19, 2023
Includes commits from these pull requests:

	#191
	#205
	#206
	#207
	#208
	#215
	#220
	#221

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
dscho pushed a commit that referenced this pull request Aug 8, 2023
Includes commits from these pull requests:

	#191
	#205
	#206
	#207
	#208
	#215
	#220
	#221

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
dscho pushed a commit that referenced this pull request Aug 8, 2023
Includes commits from these pull requests:

	#191
	#205
	#206
	#207
	#208
	#215
	#220
	#221

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
dscho pushed a commit that referenced this pull request Aug 11, 2023
Includes commits from these pull requests:

	#191
	#205
	#206
	#207
	#208
	#215
	#220
	#221

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
jeffhostetler added a commit that referenced this pull request Aug 23, 2023
Includes commits from these pull requests:

	#191
	#205
	#206
	#207
	#208
	#215
	#220
	#221

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
dscho pushed a commit that referenced this pull request Nov 3, 2023
Includes commits from these pull requests:

	#191
	#205
	#206
	#207
	#208
	#215
	#220
	#221

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
dscho pushed a commit that referenced this pull request Nov 3, 2023
Includes commits from these pull requests:

	#191
	#205
	#206
	#207
	#208
	#215
	#220
	#221

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
dscho pushed a commit that referenced this pull request Nov 3, 2023
Includes commits from these pull requests:

	#191
	#205
	#206
	#207
	#208
	#215
	#220
	#221

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
dscho pushed a commit that referenced this pull request Nov 8, 2023
Includes commits from these pull requests:

	#191
	#205
	#206
	#207
	#208
	#215
	#220
	#221

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
dscho pushed a commit that referenced this pull request Nov 14, 2023
Includes commits from these pull requests:

	#191
	#205
	#206
	#207
	#208
	#215
	#220
	#221

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
dscho pushed a commit that referenced this pull request Nov 20, 2023
Includes commits from these pull requests:

	#191
	#205
	#206
	#207
	#208
	#215
	#220
	#221

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
vdye pushed a commit that referenced this pull request Feb 27, 2024
Includes commits from these pull requests:

	#191
	#205
	#206
	#207
	#208
	#215
	#220
	#221

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
dscho pushed a commit that referenced this pull request Apr 23, 2024
Includes commits from these pull requests:

	#191
	#205
	#206
	#207
	#208
	#215
	#220
	#221

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
dscho pushed a commit that referenced this pull request Apr 23, 2024
Includes commits from these pull requests:

	#191
	#205
	#206
	#207
	#208
	#215
	#220
	#221

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
dscho pushed a commit that referenced this pull request Apr 23, 2024
Includes commits from these pull requests:

	#191
	#205
	#206
	#207
	#208
	#215
	#220
	#221

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
dscho pushed a commit that referenced this pull request Apr 24, 2024
Includes commits from these pull requests:

	#191
	#205
	#206
	#207
	#208
	#215
	#220
	#221

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
dscho pushed a commit that referenced this pull request Apr 29, 2024
Includes commits from these pull requests:

	#191
	#205
	#206
	#207
	#208
	#215
	#220
	#221

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
dscho pushed a commit that referenced this pull request May 14, 2024
Includes commits from these pull requests:

	#191
	#205
	#206
	#207
	#208
	#215
	#220
	#221

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
dscho pushed a commit that referenced this pull request May 14, 2024
Includes commits from these pull requests:

	#191
	#205
	#206
	#207
	#208
	#215
	#220
	#221

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
dscho pushed a commit that referenced this pull request Jun 3, 2024
Includes commits from these pull requests:

	#191
	#205
	#206
	#207
	#208
	#215
	#220
	#221

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
dscho pushed a commit that referenced this pull request Jul 17, 2024
Includes commits from these pull requests:

	#191
	#205
	#206
	#207
	#208
	#215
	#220
	#221

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
dscho pushed a commit that referenced this pull request Jul 17, 2024
Includes commits from these pull requests:

	#191
	#205
	#206
	#207
	#208
	#215
	#220
	#221

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
dscho pushed a commit that referenced this pull request Jul 17, 2024
Includes commits from these pull requests:

	#191
	#205
	#206
	#207
	#208
	#215
	#220
	#221

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
dscho pushed a commit that referenced this pull request Jul 18, 2024
Includes commits from these pull requests:

	#191
	#205
	#206
	#207
	#208
	#215
	#220
	#221

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
mjcheetham pushed a commit that referenced this pull request Jul 23, 2024
Includes commits from these pull requests:

	#191
	#205
	#206
	#207
	#208
	#215
	#220
	#221

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
dscho pushed a commit that referenced this pull request Jul 25, 2024
Includes commits from these pull requests:

	#191
	#205
	#206
	#207
	#208
	#215
	#220
	#221

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
mjcheetham pushed a commit that referenced this pull request Jul 29, 2024
Includes commits from these pull requests:

	#191
	#205
	#206
	#207
	#208
	#215
	#220
	#221

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
dscho pushed a commit that referenced this pull request Sep 18, 2024
Includes commits from these pull requests:

	#191
	#205
	#206
	#207
	#208
	#215
	#220
	#221

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
dscho pushed a commit that referenced this pull request Sep 24, 2024
Includes commits from these pull requests:

	#191
	#205
	#206
	#207
	#208
	#215
	#220
	#221

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
dscho pushed a commit that referenced this pull request Oct 8, 2024
Includes commits from these pull requests:

	#191
	#205
	#206
	#207
	#208
	#215
	#220
	#221

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants