Fix git submodule usage in the testsuite #11541
Conversation
Git patched a security issue (CVE-2022-39253) which prevents the use of local file pathed submodules by default. The reason for this is that untrusted submodules could be abused to expose file system contents. The pip test suite tests handling of git submodules and to do so creates a git repo which it then uses as a submodule. This was failing due to git's stricter rules on acceptable submodules. Thankfully, pip creates and controls the content of this git repo which means pip can trust it. To express that to git we pass `-c protocol.file.allow=always` to the git submodule add command which overrides the default behavior. This fixes pypa#11540
| @@ -0,0 +1,2 @@ | |||
| Fix pip's submodule usage in the test suite to accomodate Git updates | |||
There was a problem hiding this comment.
This should be a trivial news entry.
|
My local git hasn't updated yet and while this PR does fix the first level of issue previously exposed it doesn't fix the new problem this exposes. The problem here is https://github.com/pypa/pip/blob/main/src/pip/_internal/vcs/git.py#L490-L493 is running submodule update during packages installs and that could potentially run against untrusted submodules. I'm not sure what the correct way to address this is within pip. |
|
When building from VCS, the user is implicitly trusting the source tree (since the build script is free to do anything when the tree is built anyway), so IMO it’s acceptable to simply trust the file protocol there. |
I'm unwilling to make that assertion myself simply because I am not familiar enough with the use case here. For example, I'm not sure it is true that the submodule has any code executed in it during the pip install process. The top level repo may have its setup.py script run, and in that case you definitely have to trust the top level repo. But is that true of the submodule? Eventually, if you run the software that has been installed then you are theoretically trusting the submodule if it is part of the final package contents. What about the case where Whatever decision is made should probably be made by someone other than myself. Feel free to update this PR if you like or we can close this one in favor of a replacement if that is easier. |
|
As someone who hasn't even looked at the vulnerability and is not a git submodule user, is there any reason to not just set the git config globally in our CI, where we know its OK for us to do so safely? |
|
#11547 does what I suggested above. |
|
I just tried to globally set |
|
I've no idea what's going on with the git CVE, but I'm a bit concerned that we're simply patching over the testsuite issue here. Is there a risk that this change in git might be a problem for pip users? Do we have any idea how people use the submodule functionality in pip? As I said somewhere else, I've no understanding of why pip has specialised code for submodules, but do we know what the use case is for that code? Is it more or less likely to be used with local submodules than the average git usage of submodules? My concern here is that we end up no longer supporting file URLs in submodules, so we've in effect desupported a previously-working usage, with no warning and no mitigation (short of "downgrade git"). That's not our fault, except in the sense that if there is a way to say "this usage is OK" to git, we can't find it and hence can't advise our users. I don't have any good answers here, other than to suggest that when we fix the testsuite, we don't mark the fix as "skip news", but rather include a news entry that explains what git did, and what we had to do as a result. That way, our users at least have a point of reference they can work from. |
|
Instead of trying to make The PR is available in #11554. This feels like a better fix to me. As a bonus, the test should now run on Windows since |
Git patched a security issue (CVE-2022-39253) which prevents the use of local file pathed submodules by default. The reason for this is that untrusted submodules could be abused to expose file system contents.
The pip test suite tests handling of git submodules and to do so creates a git repo which it then uses as a submodule. This was failing due to git's stricter rules on acceptable submodules. Thankfully, pip creates and controls the content of this git repo which means pip can trust it. To express that to git we pass
-c protocol.file.allow=alwaysto the git submodule add command which overrides the default behavior.This fixes #11540