Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Certifi downgrade fails with pdm 2.10.3 in virtualenv! #2423

Closed
1 task done
morfinPL opened this issue Nov 22, 2023 · 3 comments
Closed
1 task done

Certifi downgrade fails with pdm 2.10.3 in virtualenv! #2423

morfinPL opened this issue Nov 22, 2023 · 3 comments
Labels
🐛 bug Something isn't working

Comments

@morfinPL
Copy link

  • I have searched the issue tracker and believe that this is not a duplicate.

Make sure you run commands with -v flag before pasting the output.

Steps to reproduce

Run docker with docker run -ti python:3.11.5 /bin/bash. Then inside the docker container run the following commands:

mkdir project
cd project
python -m pip install --upgrade pip virtualenv
python -m virtualenv .venv
. .venv/bin/activate
pip install pdm
pdm init
pdm add -v certifi==2023.7.22

Actual behavior

Last command fails with the following output:

Adding packages to default dependencies: certifi==2023.7.22
STATUS: Resolving dependencies
pdm.termui: ======== Start resolving requirements ========
pdm.termui:   certifi==2023.7.22
pdm.termui:   python>=3.11
pdm.termui:   Adding requirement certifi==2023.7.22
pdm.termui:   Adding requirement python>=3.11
pdm.termui: ======== Starting round 0 ========
STATUS: Resolving: new pin python>=3.11
pdm.termui: Pinning: python None
pdm.termui: ======== Ending round 0 ========
pdm.termui: ======== Starting round 1 ========
STATUS: Resolving: new pin certifi 2023.7.22
pdm.termui: Pinning: certifi 2023.7.22
pdm.termui: ======== Ending round 1 ========
pdm.termui: ======== Starting round 2 ========
pdm.termui: ======== Resolution Result ========
pdm.termui: Stable pins:
pdm.termui:    python None
pdm.termui:   certifi 2023.7.22
STATUS: Fetching hashes for resolved packages...
pdm.termui: Fetching hashes for certifi@2023.7.22
🔒 Lock successful
Changes are written to pyproject.toml.
STATUS: Resolving packages from lockfile...
STATUS: Fetching hashes for resolved packages...
Synchronizing working set with resolved packages: 0 to add, 1 to update, 0 to remove

pdm.termui: Removing distribution certifi
  ✖ Update certifi 2023.11.17 -> 2023.7.22 failed
pdm.termui: Error occurs: 
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/project/.venv/lib/python3.11/site-packages/pdm/installers/synchronizers.py", line 308, in update_candidate
    self.manager.install(can)
  File "/project/.venv/lib/python3.11/site-packages/pdm/installers/manager.py", line 33, in install
    installer(str(prepared.build()), self.environment, prepared.direct_url())
                  ^^^^^^^^^^^^^^^^
  File "/project/.venv/lib/python3.11/site-packages/pdm/models/candidates.py", line 399, in build
    self.obtain(allow_all=False)
  File "/project/.venv/lib/python3.11/site-packages/pdm/models/candidates.py", line 453, in obtain
    self._unpack(validate_hashes=not allow_all)
  File "/project/.venv/lib/python3.11/site-packages/pdm/models/candidates.py", line 467, in _unpack
    result = finder.download_and_unpack(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/project/.venv/lib/python3.11/site-packages/unearth/finder.py", line 421, in download_and_unpack
    file = unpack_link(
           ^^^^^^^^^^^^
  File "/project/.venv/lib/python3.11/site-packages/unearth/preparer.py", line 328, in unpack_link
    with session.get(link.normalized, stream=True) as resp:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/project/.venv/lib/python3.11/site-packages/requests/sessions.py", line 602, in get
    return self.request("GET", url, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/project/.venv/lib/python3.11/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/project/.venv/lib/python3.11/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/project/.venv/lib/python3.11/site-packages/cachecontrol/adapter.py", line 76, in send
    resp = super().send(request, stream, timeout, verify, cert, proxies)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/project/.venv/lib/python3.11/site-packages/requests/adapters.py", line 458, in send
    self.cert_verify(conn, request.url, verify, cert)
  File "/project/.venv/lib/python3.11/site-packages/requests/adapters.py", line 261, in cert_verify
    raise OSError(
OSError: Could not find a suitable TLS CA certificate bundle, invalid path: /project/.venv/lib/python3.11/site-packages/certifi/cacert.pem
Retry failed jobs
pdm.termui: Removing distribution None
  ✖ Update certifi None -> 2023.7.22 failed
pdm.termui: Error occurs: 
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/project/.venv/lib/python3.11/site-packages/pdm/installers/synchronizers.py", line 308, in update_candidate
    self.manager.install(can)
  File "/project/.venv/lib/python3.11/site-packages/pdm/installers/manager.py", line 33, in install
    installer(str(prepared.build()), self.environment, prepared.direct_url())
                  ^^^^^^^^^^^^^^^^
  File "/project/.venv/lib/python3.11/site-packages/pdm/models/candidates.py", line 399, in build
    self.obtain(allow_all=False)
  File "/project/.venv/lib/python3.11/site-packages/pdm/models/candidates.py", line 453, in obtain
    self._unpack(validate_hashes=not allow_all)
  File "/project/.venv/lib/python3.11/site-packages/pdm/models/candidates.py", line 467, in _unpack
    result = finder.download_and_unpack(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/project/.venv/lib/python3.11/site-packages/unearth/finder.py", line 421, in download_and_unpack
    file = unpack_link(
           ^^^^^^^^^^^^
  File "/project/.venv/lib/python3.11/site-packages/unearth/preparer.py", line 328, in unpack_link
    with session.get(link.normalized, stream=True) as resp:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/project/.venv/lib/python3.11/site-packages/requests/sessions.py", line 602, in get
    return self.request("GET", url, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/project/.venv/lib/python3.11/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/project/.venv/lib/python3.11/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/project/.venv/lib/python3.11/site-packages/cachecontrol/adapter.py", line 76, in send
    resp = super().send(request, stream, timeout, verify, cert, proxies)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/project/.venv/lib/python3.11/site-packages/requests/adapters.py", line 458, in send
    self.cert_verify(conn, request.url, verify, cert)
  File "/project/.venv/lib/python3.11/site-packages/requests/adapters.py", line 261, in cert_verify
    raise OSError(
OSError: Could not find a suitable TLS CA certificate bundle, invalid path: /project/.venv/lib/python3.11/site-packages/certifi/cacert.pem
Traceback (most recent call last):
  File "/project/.venv/bin/pdm", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/project/.venv/lib/python3.11/site-packages/pdm/core.py", line 295, in main
    return Core().main(args or sys.argv[1:])
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/project/.venv/lib/python3.11/site-packages/pdm/core.py", line 208, in main
    raise cast(Exception, err).with_traceback(traceback) from None
  File "/project/.venv/lib/python3.11/site-packages/pdm/core.py", line 203, in main
    self.handle(project, options)
  File "/project/.venv/lib/python3.11/site-packages/pdm/core.py", line 157, in handle
    command.handle(project, options)
  File "/project/.venv/lib/python3.11/site-packages/pdm/cli/commands/add.py", line 69, in handle
    self.do_add(
  File "/project/.venv/lib/python3.11/site-packages/pdm/cli/commands/add.py", line 181, in do_add
    do_sync(
  File "/project/.venv/lib/python3.11/site-packages/pdm/cli/actions.py", line 215, in do_sync
    synchronizer.synchronize()
  File "/project/.venv/lib/python3.11/site-packages/pdm/installers/synchronizers.py", line 455, in synchronize
    raise InstallationError("Some package operations are not complete yet")
pdm.exceptions.InstallationError: Some package operations are not complete yet

The way to make it work is to call pip install certifi==2023.7.22 and then run pdm add certifi==2023.7.22. As described here: certifi/python-certifi#225. At the same time, everything works well if you skip part with virtualenv. And y

Expected behavior

Last command finished with success and output:

Adding packages to default dependencies: certifi==2023.7.22
🔒 Lock successful
Changes are written to pyproject.toml.
All packages are synced to date, nothing to do.

🎉 All complete!

Environment Information

Paste the output of pdm info && pdm info --env below:

PDM version:
  2.10.3
Python Interpreter:
  /project/.venv/bin/python (3.11)
Project Root:
  /project
Local Packages:
  
{
  "implementation_name": "cpython",
  "implementation_version": "3.11.5",
  "os_name": "posix",
  "platform_machine": "x86_64",
  "platform_release": "6.2.0-26-generic",
  "platform_system": "Linux",
  "platform_version": "#26~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Thu Jul 13 16:27:29 UTC 2",
  "python_full_version": "3.11.5",
  "platform_python_implementation": "CPython",
  "python_version": "3.11",
  "sys_platform": "linux"
}
@morfinPL morfinPL added the 🐛 bug Something isn't working label Nov 22, 2023
@frostming
Copy link
Collaborator

Don't manage dependencies of PDM itself using the same PDM.

If it's not intended you may want to set PDM_IGNORE_ACTIVATED_VENV=1 to ignore the current activated venv, which the PDM lies in.

@frostming frostming closed this as not planned Won't fix, can't repro, duplicate, stale Nov 23, 2023
@morfinPL
Copy link
Author

@frostming can you tell me how to use PDM_IGNORE_ACTIVATED_VENV=1?

I tried the following in the docker container:

mkdir project
cd project
python -m pip install --upgrade pip virtualenv
python -m virtualenv .venv
. .venv/bin/activate
pip install pdm
export PDM_IGNORE_ACTIVATED_VENV=1
pdm init
pdm add -v certifi==2023.7.22

And results is the same as before.

@frostming
Copy link
Collaborator

frostming commented Nov 23, 2023

Okay, the problem is you installed in the default location of venv .venv, and PDM will reuse that(and manage that) venv no matter how. This results in you are making PDM to uninstall components of itself! Imagine if you run some venv manage command like pdm venv remove in-project. So just don't do that. Install pdm in a dedicated location and start a new project.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🐛 bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants