Permission denied on config.lock when cloning in Docker under MacOS

[mmajchrzycki]

Hi,
I found a bug related to the FileLock system. What I'm doing:

1. In Ubuntu 20.04 running under Docker, I'm cloning the repository using clone_from.
2. The directory that I'm cloning to is mounted under MacOS as follows:
   docker run --rm --interactive -v $PWD:/work --user $(id -u):$(id -g) /bin/bash
3. Docker container is running with UID and GID of the MacOS

I'm getting this error:

Exception in thread my-repository:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/git/util.py", line 941, in _obtain_lock_or_raise
    fd = os.open(lock_file, flags, 0)
PermissionError: [Errno 13] Permission denied: '/work/repo/my-repository.git/.git/config.lock'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/app/clone-tool.py", line 124, in run
    git.Repo.clone_from(self.git_url, repo_dir)
  File "/usr/local/lib/python3.8/dist-packages/git/repo/base.py", line 1325, in clone_from
    return cls._clone(
  File "/usr/local/lib/python3.8/dist-packages/git/repo/base.py", line 1252, in _clone
    with repo.remotes[0].config_writer as writer:
  File "/usr/local/lib/python3.8/dist-packages/git/remote.py", line 1146, in config_writer
    writer = self.repo.config_writer()
  File "/usr/local/lib/python3.8/dist-packages/git/repo/base.py", line 604, in config_writer
    return GitConfigParser(self._get_config_path(config_level), read_only=False, repo=self)
  File "/usr/local/lib/python3.8/dist-packages/git/config.py", line 366, in __init__
    self._acquire_lock()
  File "/usr/local/lib/python3.8/dist-packages/git/config.py", line 385, in _acquire_lock
    self._lock._obtain_lock()
  File "/usr/local/lib/python3.8/dist-packages/git/util.py", line 951, in _obtain_lock
    return self._obtain_lock_or_raise()
  File "/usr/local/lib/python3.8/dist-packages/git/util.py", line 944, in _obtain_lock_or_raise
    raise IOError(str(e)) from e
OSError: [Errno 13] Permission denied: '/work/repo/my-repository.git/.git/config.lock'

The config.lock file as listed under Ubuntu:

$ ls -la
ls: cannot access 'config.lock': No such file or directory
total 144
drwxr-xr-x 14 501 dialout    448 Mar 24 13:42 .
drwxr-xr-x 18 501 dialout    576 Mar 24 13:42 ..
-rw-r--r--  1 501 dialout     23 Mar 24 13:42 HEAD
drwxr-xr-x  2 501 dialout     64 Mar 24 13:42 branches
-rw-r--r--  1 501 dialout    298 Mar 24 13:42 config
-?????????  ? ?   ?            ?            ? config.lock
-rw-r--r--  1 501 dialout     73 Mar 24 13:42 description
drwxr-xr-x 14 501 dialout    448 Mar 24 13:42 hooks
-rw-r--r--  1 501 dialout 122075 Mar 24 13:42 index
drwxr-xr-x  3 501 dialout     96 Mar 24 13:42 info
drwxr-xr-x  4 501 dialout    128 Mar 24 13:42 logs
drwxr-xr-x  4 501 dialout    128 Mar 24 13:42 objects
-rw-r--r--  1 501 dialout    114 Mar 24 13:42 packed-refs
drwxr-xr-x  5 501 dialout    160 Mar 24 13:42 refs

Also listed under MacOS:

$ ls -la
total 288
drwxr-xr-x  14 mmajchrzycki  staff     448 Mar 24 14:42 .
drwxr-xr-x  18 mmajchrzycki  staff     576 Mar 24 14:42 ..
-rw-r--r--   1 mmajchrzycki  staff      23 Mar 24 14:42 HEAD
drwxr-xr-x   2 mmajchrzycki  staff      64 Mar 24 14:42 branches
-rw-r--r--@  1 mmajchrzycki  staff     298 Mar 24 14:42 config
----------   1 mmajchrzycki  staff       0 Mar 24 14:42 config.lock
-rw-r--r--   1 mmajchrzycki  staff      73 Mar 24 14:42 description
drwxr-xr-x  14 mmajchrzycki  staff     448 Mar 24 14:42 hooks
-rw-r--r--   1 mmajchrzycki  staff  122075 Mar 24 14:42 index
drwxr-xr-x   3 mmajchrzycki  staff      96 Mar 24 14:42 info
drwxr-xr-x   4 mmajchrzycki  staff     128 Mar 24 14:42 logs
drwxr-xr-x   4 mmajchrzycki  staff     128 Mar 24 14:42 objects
-rw-r--r--   1 mmajchrzycki  staff     114 Mar 24 14:42 packed-refs

I will provide a minimum set-up (docker file and python script) to reproduce it later.

[Byron]

I am not sure this issue is related to GitPython, as having it here implies GitPython should work in an environment where file permissions are not working as expected.

Thus I am closing the issue as it's not actionable.

However, maybe also thanks to the scripts to help reproduce it, one can experiment and see if the situation is common enough to try implement (and contribute) workarounds. Please feel free to keep commenting here and once the context changed the issue can be reopened.

[mmajchrzycki]

I've got a simple example to reproduce the issue on MacOS

1. Get Dockerfile
2. Get Python script
3. Build the container:

docker build -t clone-test .

4. Run the container:

docker run -v $PWD:/work clone-test

Tested under:
MacOS 13.2.1 (22D68)
Docker version 20.10.23, build 7155243

[mmajchrzycki]

I reduced the example to the simplest one, and it seems the issue is os.open method. When built-in Python's open function is used, there is no issue with file permissions.
According to Python docs the built-in open function is preferred (unless we need to do very low-level tasks, which is not the case here).

[Byron]

Thanks for your help! From the description above it seems clear that os.open could be replaced with open without side-effects to fix the issue right away.

Could I interest in contributing this fix? Thanks for your consideration.

[mmajchrzycki]

Yup, definitely! I will make a PR soon.

[muthurajr]

Definitely we can switch from os.open() to open() as the fix. But underlying root cause is VirtioFS in Docker Desktop, and we can switch to gRPC Fuse as a temporary fix. Just posting it as it may help someone till PR is merged.

[HageMaster3108]

@muthurajr @mmajchrzycki Just created a PR. Found that removing the "0" in the os.open call would also reach the same goal shortly after having opened the PR though, what do you think? Why is the 0 even there?

fd = os.open(lock_file, flags, 0)

[HageMaster3108]

Just realised that you are the maintainer @Byron - do you know what the rationale is behind setting mode=0 in os.open()? Shouldn't it be mode=0o700 🤔

[Byron]

I think a relevant part of the answer would be my reply in your PR.

Regarding mode=0, I don't know and it probably doesn't matter as this 'lock-file' implementation is racy and won't actually protect concurrent access, from what I can tell now.