Support python 3.12 #1012

bryankaplan · 2024-07-04T19:19:33Z

Currently, opencv-python fails to build under python 3.12. This patch fixes that problem by permitting the use of any setuptools>=59.2.0, rather than mandating that specific version.

This patch fixes #988. See discussion there for details.

I have tested by successfully installing opencv-python.

Avasam · 2024-08-01T15:50:52Z

This should close #993 as well, and possibly #1005

Avasam · 2024-08-01T16:23:04Z

pyproject.toml

@@ -9,5 +9,5 @@ requires = [
  "numpy>=2.0.0;  python_version>='3.9'",
  "pip",
  "scikit-build>=0.14.0",
-  "setuptools==59.2.0",
+  "setuptools>=59.2.0",


I normally wouldn't recommend pinning setuptools on an upperbound unless you have an explicit reason, but v71 changed how it vendors its dependencies in a major way. https://setuptools.pypa.io/en/latest/history.html#v71-0-0
Where it'll prefer already installed deps over its vendors. This can cause issues if the environment has an older version of a dep installed (which wasn't problematic before for setuptools, as it'd use it's own vendor anyway).

Note that running pip install setuptools[core] would install the necessary dependencies along with setuptools.

Given the various distributions and dockerization of opencv-python, it might be warranted to progressively relax this pin by first going to:

Suggested change

"setuptools>=59.2.0",

"setuptools>=59.2.0,<71.0.0",

Leaving this to the maintainers whether that's a concern 😄

I'm reticent to introduce an upper-bound with neither any observed problems nor a plan to fix any anticipated problems. But I'm happy to commit the suggestion if maintainers agree it's prudent.

Yeah same, figured I'd at least let it be known.

@bryankaplan Done, thank you

mm3509 · 2024-09-30T16:41:36Z

I tried installing OpenCV on macOS in a virtual environment, with the suggestion by @Avasam of setuptools==59.2.0 and setuptools>59.2.0,<71.0.0, and the installation fails:

Collecting setuptools<71.0.0,>59.2.0 (from -r requirements.txt (line 2))
  Using cached setuptools-70.3.0-py3-none-any.whl.metadata (5.8 kB)
Collecting opencv_python>=4.5.2.52 (from -r requirements.txt (line 4))
  Using cached opencv-python-4.10.0.84.tar.gz (95.1 MB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
...
  File "/private/var/folders/vp/8rjwz51s5wx5dptcn89czk8m0000gn/T/pip-build-env-h9pxukyj/overlay/lib/python3.12/site-packages/setuptools/__init__.py", line 10, in <module>
    import distutils.core
ModuleNotFoundError: No module named 'distutils'

I'm on macOS 11.7.10, Python 3.12.5. I upgraded to Python 3.12.6 and have the same issue. I tried in a new virtual environment, and I tried it twice.

bryankaplan · 2024-09-30T16:47:51Z

Thank you for testing, @mm3509. I run Linux, so I can't confirm whether the platform has any relevance here. But it's worth noting that pip used a cached setuptools. Does it succeed if you use pip's --no-cache-dir option? Relevant discussions online can be found here and here.

mm3509 · 2024-09-30T17:56:44Z

Thank you for testing, @mm3509. I run Linux, so I can't confirm whether the platform has any relevance here. But it's worth noting that pip used a cached setuptools. Does it succeed if you use pip's --no-cache-dir option? Relevant discussions online can be found here and here.

The option --no-cache-dir didn't help, but I tried on Python 3.11 and now it works.

bryankaplan · 2024-09-30T18:02:53Z

@mm3509 I'm glad you were able to install opencv-python.

N.B. Installation under python 3.11 always has worked. This patch aims to support 3.12. It will be helpful to determine whether this fix works under macOS, and if not what does.

mm3509 · 2024-09-30T18:12:04Z

Thank you Bryan. You can count on me to test any patches on macOS :) At the moment, I am out of ideas for how to make it work.

CanePlayz · 2024-10-28T20:16:56Z

This is the verbose output I get when building on a Windows arm64 machine with Python 3.13:

PS C:\Users\jjpfs\Documents\opencv-python> pip wheel .
Processing c:\users\jjpfs\documents\opencv-python
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
ERROR: Exception:
Traceback (most recent call last):
  File "C:\Users\jjpfs\AppData\Local\Programs\Python\Python313-arm64\Lib\site-packages\pip\_internal\cli\base_command.py", line 105, in _run_wrapper
    status = _inner_run()
  File "C:\Users\jjpfs\AppData\Local\Programs\Python\Python313-arm64\Lib\site-packages\pip\_internal\cli\base_command.py", line 96, in _inner_run
    return self.run(options, args)
           ~~~~~~~~^^^^^^^^^^^^^^^
  File "C:\Users\jjpfs\AppData\Local\Programs\Python\Python313-arm64\Lib\site-packages\pip\_internal\cli\req_command.py", line 67, in wrapper
    return func(self, options, args)
  File "C:\Users\jjpfs\AppData\Local\Programs\Python\Python313-arm64\Lib\site-packages\pip\_internal\commands\wheel.py", line 147, in run
    requirement_set = resolver.resolve(reqs, check_supported_wheels=True)
  File "C:\Users\jjpfs\AppData\Local\Programs\Python\Python313-arm64\Lib\site-packages\pip\_internal\resolution\resolvelib\resolver.py", line 76, in resolve
    collected = self.factory.collect_root_requirements(root_reqs)
  File "C:\Users\jjpfs\AppData\Local\Programs\Python\Python313-arm64\Lib\site-packages\pip\_internal\resolution\resolvelib\factory.py", line 545, in collect_root_requirements
    reqs = list(
        self._make_requirements_from_install_req(
    ...<2 lines>...
        )
    )
  File "C:\Users\jjpfs\AppData\Local\Programs\Python\Python313-arm64\Lib\site-packages\pip\_internal\resolution\resolvelib\factory.py", line 501, in _make_requirements_from_install_req
    cand = self._make_base_candidate_from_link(
        ireq.link,
    ...<2 lines>...
        version=None,
    )
  File "C:\Users\jjpfs\AppData\Local\Programs\Python\Python313-arm64\Lib\site-packages\pip\_internal\resolution\resolvelib\factory.py", line 233, in _make_base_candidate_from_link
    self._link_candidate_cache[link] = LinkCandidate(
                                       ~~~~~~~~~~~~~^
        link,
        ^^^^^
    ...<3 lines>...
        version=version,
        ^^^^^^^^^^^^^^^^
    )
    ^
  File "C:\Users\jjpfs\AppData\Local\Programs\Python\Python313-arm64\Lib\site-packages\pip\_internal\resolution\resolvelib\candidates.py", line 304, in __init__
    super().__init__(
    ~~~~~~~~~~~~~~~~^
        link=link,
        ^^^^^^^^^^
    ...<4 lines>...
        version=version,
        ^^^^^^^^^^^^^^^^
    )
    ^
  File "C:\Users\jjpfs\AppData\Local\Programs\Python\Python313-arm64\Lib\site-packages\pip\_internal\resolution\resolvelib\candidates.py", line 159, in __init__
    self.dist = self._prepare()
                ~~~~~~~~~~~~~^^
  File "C:\Users\jjpfs\AppData\Local\Programs\Python\Python313-arm64\Lib\site-packages\pip\_internal\resolution\resolvelib\candidates.py", line 236, in _prepare
    dist = self._prepare_distribution()
  File "C:\Users\jjpfs\AppData\Local\Programs\Python\Python313-arm64\Lib\site-packages\pip\_internal\resolution\resolvelib\candidates.py", line 315, in _prepare_distribution
    return preparer.prepare_linked_requirement(self._ireq, parallel_builds=True)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\jjpfs\AppData\Local\Programs\Python\Python313-arm64\Lib\site-packages\pip\_internal\operations\prepare.py", line 527, in prepare_linked_requirement
    return self._prepare_linked_requirement(req, parallel_builds)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\jjpfs\AppData\Local\Programs\Python\Python313-arm64\Lib\site-packages\pip\_internal\operations\prepare.py", line 642, in _prepare_linked_requirement
    dist = _get_prepared_distribution(
        req,
    ...<3 lines>...
        self.check_build_deps,
    )
  File "C:\Users\jjpfs\AppData\Local\Programs\Python\Python313-arm64\Lib\site-packages\pip\_internal\operations\prepare.py", line 72, in _get_prepared_distribution
    abstract_dist.prepare_distribution_metadata(
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        finder, build_isolation, check_build_deps
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "C:\Users\jjpfs\AppData\Local\Programs\Python\Python313-arm64\Lib\site-packages\pip\_internal\distributions\sdist.py", line 56, in prepare_distribution_metadata
    self._install_build_reqs(finder)
    ~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^
  File "C:\Users\jjpfs\AppData\Local\Programs\Python\Python313-arm64\Lib\site-packages\pip\_internal\distributions\sdist.py", line 126, in _install_build_reqs
    build_reqs = self._get_build_requires_wheel()
  File "C:\Users\jjpfs\AppData\Local\Programs\Python\Python313-arm64\Lib\site-packages\pip\_internal\distributions\sdist.py", line 103, in _get_build_requires_wheel
    return backend.get_requires_for_build_wheel()
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
  File "C:\Users\jjpfs\AppData\Local\Programs\Python\Python313-arm64\Lib\site-packages\pip\_internal\utils\misc.py", line 701, in get_requires_for_build_wheel
    return super().get_requires_for_build_wheel(config_settings=cs)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\jjpfs\AppData\Local\Programs\Python\Python313-arm64\Lib\site-packages\pip\_vendor\pyproject_hooks\_impl.py", line 166, in get_requires_for_build_wheel
    return self._call_hook('get_requires_for_build_wheel', {
           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        'config_settings': config_settings
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    })
    ^^
  File "C:\Users\jjpfs\AppData\Local\Programs\Python\Python313-arm64\Lib\site-packages\pip\_vendor\pyproject_hooks\_impl.py", line 321, in _call_hook
    raise BackendUnavailable(data.get('traceback', ''))
pip._vendor.pyproject_hooks._impl.BackendUnavailable: Traceback (most recent call last):
  File "C:\Users\jjpfs\AppData\Local\Programs\Python\Python313-arm64\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 77, in _build_backend
    obj = import_module(mod_path)
  File "C:\Users\jjpfs\AppData\Local\Programs\Python\Python313-arm64\Lib\importlib\__init__.py", line 88, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1310, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 1022, in exec_module
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "C:\Users\jjpfs\AppData\Local\Temp\pip-build-env-c96n0db_\overlay\Lib\site-packages\setuptools\__init__.py", line 10, in <module>
    import distutils.core
ModuleNotFoundError: No module named 'distutils'

cpatulea · 2024-12-14T22:55:16Z

FYI this PR does appear to work for getting opencv-python to install (in Home Assistant's "wheels" container which is based on Alpine Linux and Python 3.13); took me 35 minutes on a Intel CPU from 2021.

$ docker run --rm --entrypoint="" -it ghcr.io/home-assistant/wheels/amd64/musllinux_1_2/cp313:dev bash
75c590f38fa7:~# git clone --depth 1 -b 988-support-py3.12 https://github.com/bryankaplan/opencv-python.git
75c590f38fa7:~# cd opencv-python
75c590f38fa7:~/opencv-python# pip install .
Processing /root/opencv-python
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: numpy>=1.17.0 in /usr/local/lib/python3.13/site-packages (from opencv-python==4.10.0+11cb9fc) (2.2.0)
Building wheels for collected packages: opencv-python
  Building wheel for opencv-python (pyproject.toml) ... done
  Created wheel for opencv-python: filename=opencv_python-4.10.0+11cb9fc-cp313-cp313-linux_x86_64.whl size=30913164 sha256=d89fdf5385bf01cc750f27f2e3237e858d7f5a6528233a4f62e25370aade0559
  Stored in directory: /tmp/pip-ephem-wheel-cache-l0c1rv86/wheels/bb/2f/81/e07df07d397d712a66db6e36035f9b1cfb2048a44317e91e93
Successfully built opencv-python
Installing collected packages: opencv-python
Successfully installed opencv-python-4.10.0+11cb9fc

Avasam · 2024-12-14T23:46:22Z

@bryankaplan or @asmorkalov , could you retrigger this PR's CI? (closing & reopening should work), I don't recall the reason for test failure and I'm guessing it doesn't have to do with this PR.

Even if this doesn't fix everyone's issues. It should be an objective improvement.

Currently, opencv-python fails to build under python 3.12. This patch fixes that problem by permitting the use of any setuptools>=59.2.0, rather than mandating that specific version. This patch fixes opencv#988. See discussion there for details. I have tested by successfully installing opencv-python.

Avasam mentioned this pull request Aug 1, 2024

ModuleNotFoundError: No module named 'distutils' in latest release #993

Open

Avasam reviewed Aug 1, 2024

View reviewed changes

cpatulea mentioned this pull request Dec 14, 2024

pip install failed on Python 3.12+ #1059

Open

4 tasks

bryankaplan force-pushed the 988-support-py3.12 branch from 11cb9fc to 218a392 Compare December 15, 2024 00:23

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support python 3.12 #1012

Support python 3.12 #1012

bryankaplan commented Jul 4, 2024

Avasam commented Aug 1, 2024 •

edited

Loading

Avasam Aug 1, 2024 •

edited

Loading

bryankaplan Aug 1, 2024

Avasam Aug 1, 2024

mm3509 Sep 30, 2024

mm3509 commented Sep 30, 2024 •

edited

Loading

bryankaplan commented Sep 30, 2024

mm3509 commented Sep 30, 2024

bryankaplan commented Sep 30, 2024

mm3509 commented Sep 30, 2024

CanePlayz commented Oct 28, 2024

cpatulea commented Dec 14, 2024 •

edited

Loading

Avasam commented Dec 14, 2024 •

edited

Loading

Support python 3.12 #1012

Are you sure you want to change the base?

Support python 3.12 #1012

Conversation

bryankaplan commented Jul 4, 2024

Avasam commented Aug 1, 2024 • edited Loading

Avasam Aug 1, 2024 • edited Loading

Choose a reason for hiding this comment

bryankaplan Aug 1, 2024

Choose a reason for hiding this comment

Avasam Aug 1, 2024

Choose a reason for hiding this comment

mm3509 Sep 30, 2024

Choose a reason for hiding this comment

mm3509 commented Sep 30, 2024 • edited Loading

bryankaplan commented Sep 30, 2024

mm3509 commented Sep 30, 2024

bryankaplan commented Sep 30, 2024

mm3509 commented Sep 30, 2024

CanePlayz commented Oct 28, 2024

cpatulea commented Dec 14, 2024 • edited Loading

Avasam commented Dec 14, 2024 • edited Loading

Avasam commented Aug 1, 2024 •

edited

Loading

Avasam Aug 1, 2024 •

edited

Loading

mm3509 commented Sep 30, 2024 •

edited

Loading

cpatulea commented Dec 14, 2024 •

edited

Loading

Avasam commented Dec 14, 2024 •

edited

Loading