Could not find a suitable TLS CA certificate bundle, when installing dependencies

[jezeniel]

• I am on the latest Poetry version.
• I have searched the issues of this repo and believe that this is not a duplicate.
• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name: Docker Image python:3.10
• Poetry version: 1.1.14
• Link of a Gist with the contents of your pyproject.toml file: pyproject.toml

Issue

Encountering an error when building in docker image with poetry config virtualenvs.create false

#0 3.253    1  /usr/local/lib/python3.10/site-packages/requests/adapters.py:460 in send
#0 3.253
#0 3.253   OSError
#0 3.253
#0 3.253   Could not find a suitable TLS CA certificate bundle, invalid path: /usr/local/lib/python3.10/site-packages/certifi/cacert.pem
#0 3.253
#0 3.253   at /usr/local/lib/python3.10/site-packages/requests/adapters.py:263 in cert_verify

Though downgrading to 1.1.13 fixes the problem.

Looks like similar with #4414 at first glance

EDIT:
This is related to #1889, when using --no-dev flag in poetry install and virtualenvs.create false it uninstalls some unneeded dependencies which causes certifi to be uninstalled in the system before poetry installs the main dependencies, causing the missing ca cert errors.

The easiest workaround is to poetry add certifi so that the --no-dev flag will not uninstall it.

[dimbleby]

poetry is not responsible for your CA certificates, and there is no change between 1.1.13 and 1.1.14 that could possibly be relevant

[jezeniel]

I know. This is my initial hunch that poetry could not be the cause.

But when I export and use requirements.txt it works. I removed the virtualenvs.create config and it works for 1.1.14. but when setting it back to false it starts to spit out this error.

I tried building my image with both versions multiple times, and it works when I downgrade to 1.1.13, regardless if virtualenvs.create is true or false.

But I am using mac on docker when testing this. Haven't tried it on linux yet.

[dimbleby]

The best hint this project can give you is: whatever you think you are seeing that implicates poetry, it is highly unlikely that poetry is actually relevant to your problem.

If you can find a way to reproduce this that shows otherwise, please share it.

[jezeniel]

I think I saw what is causing the issue. It is related to #1889. When virtualenvs.create is false, and I ran poetry install with --no-dev flag, it is uninstalling the subdependencies of my dev dependencies in this case certifi.

Adding certifi as my dependency fixes the problem. And as per #1889 last comment this behavior will change in 1.2, I will update this issue, and let me know if my assumption is right so I can close this issue now @dimbleby. Thanks!

[dimbleby]

sure, you don't need my permission! if you're happy, please close

[jezeniel]

Closing issue now, need to wait for 1.2 to fix, but workaround is simple enough.

[wrighbr]

This issue still seems to be problem with 1.2.0 but works fine in 1.1.15

[neersighted]

If you are running into this, please post a detailed reproduction including a pyproject.toml and the exact commands Poetry is being invoked with.

This user ran into Poetry uninstalling its own dependencies because it was installed into an environment that was shared with the project being managed (as well as the OS) by using virtualenvs.create false. Installing Poetry and your project into the same environment as your project is going to be fraught, as Poetry will without hesitation break itself (like you see here) when the dependency trees overlap.

The best defense against this is to not do it at all -- even if it was seemingly stable (like the 1.1 branch), your own changes, or Poetry's changes may break it or cause unexpected runtime behavior at any time.

See #6398 for a roundup of discussion on this and plans to clarify the docs on this particular topic.

TL; DR if you are running into this without virtualenvs.create false or similar (e.g. installing Poetry and the project into one active virtual environment), please open a new issue -- that's new and bizarre. This issue tracks a situation where Poetry broke itself because two different Python codebases were installed into one environment, and that is to be expected to a large degree when doing so.

[reputed-victor]

Could it be related to certifi having a 2022.09.14 release today? As far as I understand, some of the path-determining logic was changed in that release: certifi/python-certifi@47fb7ab

[roniemartinez]

Just to add based on my observation. This happens every time certifi releases a new version. It happened in 2022.09.14 and it happened again in 2022.09.24.

Poetry version is 1.2.0.

[neersighted]

Just to add based on my observation. This happens every time certifi releases a new version. It happened in 2022.09.14 and it happened again in 2022.09.24.

Poetry version is 1.2.0.

If you're experiencing this, you're installing Poetry and your project such that Poetry is upgrading its own dependencies. If you install Poetry as recommended/supported, Poetry will not try to upgrade certifi under itself.

[roniemartinez]

If you install Poetry as recommended/supported

@neersighted What is the recommended way of installing dependencies inside Docker?

Running virtualenv inside a Python docker container looks a bit overkill. I am not sure why we need the double isolation.

[neersighted]

The recommended install methods are always the same, regardless of containerization. If you install your project and Poetry into the same environment (system or virtual), Poetry will happily break itself as you are asking it to install/manipulate packages that Poetry itself depends on.

You should review the install methods (in particular, the CI recs: https://python-poetry.org/docs/#ci-recommendations) and pick one. All of them will install Poetry into a virtual environment to keep from 'corrupting' it by mixing your project in.

Virtual environments are still very useful (and near mandatory, depending) in containers for your project as well. Depending on your base image, you may have Python code mixed in to your Poetry environment than can either cause unexpected behavior or cause Poetry to be unable to install. Isolating the choices of your Python distributor from your application is another important feature of virtual environments, and not one to be discounted (see #6398).

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.