-
Notifications
You must be signed in to change notification settings - Fork 3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[24.1] self signed certificates are no longer loaded #12779
Comments
I can confirm similar behaviour on windows. |
Hey all, apologies for the disruption. Pip by default does not use the system certificate store. It is an optional feature enabled by the integration of |
@rasmus91 @sweetlilmre Just to confirm, you're running into this issue without using the optional "Truststore" feature or while you're using it (ie |
Not using that option, no.
fre. 21. jun. 2024 16.06 skrev Seth Michael Larson ***@***.***
…:
@sweetlilmre <https://github.com/sweetlilmre> Just to confirm, you're
running into this issue *without* using the optional "Truststore" feature
or while you're using it (ie --with-feature=truststore)? Have to clarify
because it's not clear from your posted reproduction steps.
—
Reply to this email directly, view it on GitHub
<#12779 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AARQ7C5RHVTYOACO42J24XDZIQXNXAVCNFSM6AAAAABJVQC7RSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOBSHAZDGNRWHE>
.
You are receiving this because you authored the thread.Message ID:
***@***.***>
|
@sethmlarson, while using it:
Sorry for the ambiguity. |
Yes, it did until 24.1. i dont understand why it wouldn't when the CA cert is installed system wide? |
So I'm actually suspecting that a defect in Requests is the source of the issues here, pip uses Requests 2.32.0 but there have been 3 bugfix releases since that release to Requests with some relating to certificates. |
I've always wanted to know how pip's vendoring process worked, so I've created two branches to test if requests is the issue: Branch with latest version of requests: Can someone please try their workflow that works on 24.0 but breaks on 24.1 on both of these branches and report the results for both please (may need to use |
Looking at the requests changelog, I strongly suspect that psf/requests#6655 is implicated. psf/requests#6716 which landed in requests v2.32.3 restores the ability to specify custom SSLContexts in sub-classes of HTTPAdapter which pip does have. pip/src/pip/_internal/network/session.py Line 290 in 2753c77
Please try @notatallshaw's branches to help us determine whether requests is the root problem and upgrading requests fixes the bug. Thanks! |
Let me preface this with: I am no python / pip expert, in fact I probably don't know what I'm doing.
Results: 2.32.3:
Set REQUESTS_CA_BUNDLE environment variable
Clear REQUESTS_CA_BUNDLE environment variable Check: 2.31.0
Set REQUESTS_CA_BUNDLE environment variable
Clear REQUESTS_CA_BUNDLE environment variable Check:
|
Thanks for running the tests, I'll fix the issue with the 2.31.0 branch later today and confirm. Am I correct in understanding though that the 2.32.3 branch now succeeds where pip 24.1 fails? |
As per my tests, yes. |
I fixed the 2.31.0 branch if you want to reinstall it and try, though it's a little less important seeing that 2.32.3 works, it at least prooves definitively it was a requests regression and unrelated to truststore. |
I reran the 2.31.0 branch for completeness sake. 2.31.0
Set REQUESTS_CA_BUNDLE environment variable
Clear REQUESTS_CA_BUNDLE environment variable Check:
|
As a final comment on this issue, |
While I'm sure you didn't mean to offend, "debacle" is a bit strong here. We published a beta with the new requests release, and didn't get any feedback on this issue. The pip maintainers themselves don't have access to environments that need this feature, and so we weren't able to trigger the issue ourselves. If you have any suggestions (or better still a PR!) for how we could test this within our test suite then that would be appreciated - although it's arguable that this is something that should be covered in the requests test suite, so maybe proposing something at their end would also be useful. We try to avoid regressions like this, but there's a lot of pip functionality that's only used by particular parts of our user base, and often those users don't (for perfectly good reasons) get involved in testing and proving new releases for us. Negative feedback tends to reduce our motivation for taking risks in those areas, which in turn slows the pace of developments and improvements for those users. It's difficult to find a good balance. |
@pfmoore first off apologies, this was in no way intended as a slight toward the pip or request maintainers. The debacle I was referring to is the general certificate handling case in the python ecosystem. Unfortunately I live in a corporate environment with SSL inspection. This causes havoc all the time with new starts and even folks that have come back to python after a while, or had to reinstall for whatever reason. REQUESTS_CA_BUNDLE is always quoted as the solution but works in only some cases (seems to work for python and pip, but with truststore or something like that, baked in would be a huge win for python as a whole and debacle was the word that (perhaps a little unfortunately) I chose to express my frustration with the state of play. Again apologies, and thank you to you, and all involved in this work. |
FWIW, back in my large corporation days I had to teach people to use px-proxy: https://github.com/genotrance/px?tab=readme-ov-file#installation Which I found as the most consistent workaround to Python http ecosystem vs. enterprise security. Although I didn't get to try truststore, I'm now at a relatively small company where I don't face super esoteric security protocols |
@notatallshaw thank you, I'll take a look at that. |
Description
I am using a gitlab python repo, that proxies pypi.
After upgrading to pip 24.1 the certificate I have installed in
/usr/local/share/ca-certificates
is no longer loaded it seems, to validate that source.After going back to pip 24.0 It works just fine once again.
Expected behavior
I would expect no problem installing packages through a private repo for which I have appropriate certificates installed systemwide.
pip version
24.1
Python version
3.10
OS
Ubuntu 22.04
How to Reproduce
/usr/local/share/ca-certificates
sudo update-ca-certificates
Output
(this is output from ansible, but its still pip)
Code of Conduct
The text was updated successfully, but these errors were encountered: