BUG: pyzmq v26 installation fails (build from sources)

[AlvinSchiller]

What pyzmq version?**

26.0.0a2

What libzmq version?**

4.3.5 (seperatly build and installed)

Python version (and how it was installed)

python 3.9 via apt-get

OS

Raspberry Pi OS Bullseye 32-bit

What happened?

In our projekt we build pyzmq from sources with a seperatly prebuild libzmq installation.
Prior to the v26 release this succeeded. Since the new version release this fails.

Traceback, if applicable

Using pip 24.0 from /home/testuser/RPi-Jukebox-RFID/.venv/lib/python3.9/site-packages/pip (python 3.9)
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Collecting pyzmq
Downloading pyzmq-26.0.0a2.tar.gz (366 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 366.6/366.6 kB 5.9 MB/s eta 0:00:00
Installing build dependencies: started
Running command pip subprocess to install build dependencies
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple, https://www.piwheels.org/simple
Ignoring cffi: markers 'implementation_name == "pypy"' don't match your environment
Collecting cython>=3.0.0
Downloading Cython-3.0.8-py2.py3-none-any.whl.metadata (3.2 kB)
Collecting packaging
Downloading https://www.piwheels.org/simple/packaging/packaging-23.2-py3-none-any.whl (53 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 53.0/53.0 kB 526.4 kB/s eta 0:00:00
Collecting scikit-build-core
Downloading https://www.piwheels.org/simple/scikit-build-core/scikit_build_core-0.8.1-py3-none-any.whl (139 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 139.4/139.4 kB 1.7 MB/s eta 0:00:00
Collecting exceptiongroup (from scikit-build-core)
Downloading https://www.piwheels.org/simple/exceptiongroup/exceptiongroup-1.2.0-py3-none-any.whl (16 kB)
Collecting tomli>=1.1 (from scikit-build-core)
Downloading https://www.piwheels.org/simple/tomli/tomli-2.0.1-py3-none-any.whl (12 kB)
Downloading Cython-3.0.8-py2.py3-none-any.whl (1.2 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 10.0 MB/s eta 0:00:00
Installing collected packages: tomli, packaging, exceptiongroup, cython, scikit-build-core
Successfully installed cython-3.0.8 exceptiongroup-1.2.0 packaging-23.2 scikit-build-core-0.8.1 tomli-2.0.1
Installing build dependencies: finished with status 'done'
Getting requirements to build wheel: started
Running command Getting requirements to build wheel
Getting requirements to build wheel: finished with status 'done'
Installing backend dependencies: started
Running command pip subprocess to install backend dependencies
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple, https://www.piwheels.org/simple
Collecting pyproject_metadata
Downloading https://www.piwheels.org/simple/pyproject-metadata/pyproject_metadata-0.7.1-py3-none-any.whl (7.4 kB)
Collecting cmake>=3.28
Downloading https://www.piwheels.org/simple/cmake/cmake-3.28.3-cp39-cp39-linux_armv7l.whl (19.6 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 19.6/19.6 MB 13.4 MB/s eta 0:00:00
Collecting pathspec
Downloading https://www.piwheels.org/simple/pathspec/pathspec-0.12.1-py3-none-any.whl (31 kB)
Collecting packaging>=19.0 (from pyproject_metadata)
Using cached https://www.piwheels.org/simple/packaging/packaging-23.2-py3-none-any.whl (53 kB)
Installing collected packages: cmake, pathspec, packaging, pyproject_metadata
Successfully installed cmake-3.28.3 packaging-23.2 pathspec-0.12.1 pyproject_metadata-0.7.1
Installing backend dependencies: finished with status 'done'
Preparing metadata (pyproject.toml): started
Running command Preparing metadata (pyproject.toml)
*** scikit-build-core 0.8.1 using CMake 3.28.3 (metadata_wheel)
Preparing metadata (pyproject.toml): finished with status 'done'
Building wheels for collected packages: pyzmq
Building wheel for pyzmq (pyproject.toml): started
Running command Building wheel for pyzmq (pyproject.toml)
*** scikit-build-core 0.8.1 using CMake 3.28.3 (wheel)
*** Configuring CMake...
loading initial cache file /tmp/tmp0idg8ze3/build/CMakeInit.txt
-- The C compiler identification is GNU 10.2.1
-- The CXX compiler identification is GNU 10.2.1
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found Python: /home/testuser/RPi-Jukebox-RFID/.venv/bin/python3 (found version "3.9.2") found components: Interpreter Development.Module
-- enabling ZMQ_DRAFT_API
-- Looking for libzmq in /usr/local
-- Looking for libzmq in /usr/local - /usr/local/lib/libzmq.so
-- Using Python Python /home/testuser/RPi-Jukebox-RFID/.venv/bin/python3
-- Building Cython backend
-- Linking libzmq target libzmq
-- Configuring done (6.9s)
-- Generating done (0.0s)
-- Build files have been written to: /tmp/tmp0idg8ze3/build
*** Building project with Unix Makefiles...
[ 33%] Generating zmq/backend/cython/_zmq.c
[Errno 2] No such file or directory: '/tmp/tmp0idg8ze3/build/zmq/backend/cython/_zmq.c'
gmake[2]: *** [CMakeFiles/_zmq.dir/build.make:74: zmq/backend/cython/_zmq.c] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:82: CMakeFiles/_zmq.dir/all] Error 2
gmake: *** [Makefile:136: all] Error 2

*** CMake build failed
error: subprocess-exited-with-error

× Building wheel for pyzmq (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.
full command: /home/testuser/RPi-Jukebox-RFID/.venv/bin/python3 /home/testuser/RPi-Jukebox-RFID/.venv/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py build_wheel /tmp/tmpruyy6f9q
cwd: /tmp/pip-install-uilurpo0/pyzmq_09b91c9e3c034aeb893a3e119336ad15
Building wheel for pyzmq (pyproject.toml): finished with status 'error'
ERROR: Failed building wheel for pyzmq
Failed to build pyzmq
ERROR: Could not build wheels for pyzmq, which is required to install pyproject.toml-based projects

More info

• libzmq prebuild and copied to "/usr/local".

• Build pyzmq
  ZMQ_PREFIX="/usr/local"
  ZMQ_DRAFT_API=1
  pip install -v --no-binary pyzmq --pre pyzmq

This also fails for Raspberry Pi OS Bookworm with python 3.11

[AlvinSchiller]

PS: I couldn't submit using the "Bug Report" template, so i created this "blank" issue, trying to add all relevant information.

[minrk]

Thanks for testing the prerelease!

I don't really understand why all the CI builds are passing, but I suspect explicitly creating the backend directory will do it.

PS: I couldn't submit using the "Bug Report" template, so i created this "blank" issue, trying to add all relevant information.

Thanks! Can you tell me what the problem was with the template? I don't see anything. Blank issue's always fine, the template's just there to help. It seems like it would have worked fine here, though, so I'd like to fix it if I can.

[AlvinSchiller]

Thanks for testing the prerelease!

It was actually an unwanted test, as we mistakenly used the --pre argument ^^
But will keep an eye on it nevertheless.

Thanks! Can you tell me what the problem was with the template? I don't see anything. Blank issue's always fine, the template's just there to help. It seems like it would have worked fine here, though, so I'd like to fix it if I can.

I filled out all forms and clicked Submit but nothing happend. Tried it on a new page and the same result.
From other bugs created here early it looks like it should work. Don't know what the problem was in my case.

[minrk]

we mistakenly used the --pre argument ^^

Thanks for reporting, anyway :). FWIW, I think it's a really good practice to run tests with --pre, because if downstream packages don't do this, prereleases have very little point.

I filled out all forms and clicked Submit but nothing happened.

Ah, weird. I'll keep an eye out if anything comes up, or if it was an intermittent issue.

I just published pyzmq 26a3, which I think fixes this.

[AlvinSchiller]

Our Build succeeds again with 26.0.0a3. Thanks for the quick help!

[minrk]

Wonderful, thanks again for testing!

[pabera]

I wanted to get back on this real quick. In our use case, we have a hard requirement on WebSockets, therefore we need to enable Draft support. The documentation specifically suggests to use the --pre flag. But if it is not required, shouldn't that be updated in the docs too?

pyzmq/docs/source/howto/draft.md

Lines 29 to 37 in 5bc400b

	And to install pyzmq with draft support:
	```bash
	export ZMQ_PREFIX=${PREFIX}
	export ZMQ_DRAFT_API=1
	pip install -v --no-binary pyzmq --pre pyzmq
	```
	By specifying `--no-binary pyzmq`, pip knows to not install wheels, and will compile pyzmq from source.

[minrk]

You're right, I don't know how many years it's been since you haven't needed a prerelease to use drafts. I'll update it.