-
Notifications
You must be signed in to change notification settings - Fork 543
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Jenkins using new python versions in the matrix #1174
Conversation
…her python versions
Problem reported by @weideng1 when trying to use the approach above (install ccm normally, then update PyYAML after the fact) with 3.11.x Python versions: $ pip install "pyyaml<6.0"
Collecting pyyaml<6.0
Using cached PyYAML-5.4.1.tar.gz (175 kB)
Installing build dependencies ... done
Getting requirements to build wheel ... error
error: subprocess-exited-with-error
× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> [68 lines of output]
/tmp/pip-build-env-sqpvlw_j/overlay/lib/python3.11/site-packages/setuptools/config/setupcfg.py:293: _DeprecatedConfig: Deprecated config in `setup.cfg`
!!
********************************************************************************
The license_file parameter is deprecated, use license_files instead.
By 2023-Oct-30, you need to update your project and remove deprecated calls
or your builds will no longer be supported.
See https://setuptools.pypa.io/en/latest/userguide/declarative_config.html for details.
********************************************************************************
!!
parsed = self.parsers.get(option_name, lambda x: x)(value)
running egg_info
writing lib3/PyYAML.egg-info/PKG-INFO
writing dependency_links to lib3/PyYAML.egg-info/dependency_links.txt
writing top-level names to lib3/PyYAML.egg-info/top_level.txt
Traceback (most recent call last):
File "/home/automaton/no-previous-cython-venv/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
main()
File "/home/automaton/no-previous-cython-venv/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/automaton/no-previous-cython-venv/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
return hook(config_settings)
^^^^^^^^^^^^^^^^^^^^^
File "/tmp/pip-build-env-sqpvlw_j/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 355, in get_requires_for_build_wheel
return self._get_build_requires(config_settings, requirements=['wheel'])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/tmp/pip-build-env-sqpvlw_j/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 325, in _get_build_requires
self.run_setup()
File "/tmp/pip-build-env-sqpvlw_j/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 341, in run_setup
exec(code, locals())
File "<string>", line 271, in <module>
File "/tmp/pip-build-env-sqpvlw_j/overlay/lib/python3.11/site-packages/setuptools/__init__.py", line 107, in setup
return distutils.core.setup(**attrs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/tmp/pip-build-env-sqpvlw_j/overlay/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 185, in setup
return run_commands(dist)
^^^^^^^^^^^^^^^^^^
File "/tmp/pip-build-env-sqpvlw_j/overlay/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
dist.run_commands()
File "/tmp/pip-build-env-sqpvlw_j/overlay/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
self.run_command(cmd)
File "/tmp/pip-build-env-sqpvlw_j/overlay/lib/python3.11/site-packages/setuptools/dist.py", line 1233, in run_command
super().run_command(command)
File "/tmp/pip-build-env-sqpvlw_j/overlay/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
cmd_obj.run()
File "/tmp/pip-build-env-sqpvlw_j/overlay/lib/python3.11/site-packages/setuptools/command/egg_info.py", line 319, in run
self.find_sources()
File "/tmp/pip-build-env-sqpvlw_j/overlay/lib/python3.11/site-packages/setuptools/command/egg_info.py", line 327, in find_sources
mm.run()
File "/tmp/pip-build-env-sqpvlw_j/overlay/lib/python3.11/site-packages/setuptools/command/egg_info.py", line 549, in run
self.add_defaults()
File "/tmp/pip-build-env-sqpvlw_j/overlay/lib/python3.11/site-packages/setuptools/command/egg_info.py", line 587, in add_defaults
sdist.add_defaults(self)
File "/tmp/pip-build-env-sqpvlw_j/overlay/lib/python3.11/site-packages/setuptools/command/sdist.py", line 113, in add_defaults
super().add_defaults()
File "/tmp/pip-build-env-sqpvlw_j/overlay/lib/python3.11/site-packages/setuptools/_distutils/command/sdist.py", line 251, in add_defaults
self._add_defaults_ext()
File "/tmp/pip-build-env-sqpvlw_j/overlay/lib/python3.11/site-packages/setuptools/_distutils/command/sdist.py", line 336, in _add_defaults_ext
self.filelist.extend(build_ext.get_source_files())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<string>", line 201, in get_source_files
File "/tmp/pip-build-env-sqpvlw_j/overlay/lib/python3.11/site-packages/setuptools/_distutils/cmd.py", line 107, in __getattr__
raise AttributeError(attr)
AttributeError: cython_sources
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error
× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip. Underlying problem here appears to be the same as that described in yaml/pyyaml#601. PyYAML itself uses cython and it looks like certain versions of cython don't correctly hook into the build infrastructure. The workaround suggested there appears to work, at least with 3.11.x: $ pip install "Cython<3.0" "pyyaml<6.0" --no-build-isolation
Collecting Cython<3.0
Obtaining dependency information for Cython<3.0 from https://files.pythonhosted.org/packages/43/c9/4c8a84731c0948b408d059c2c17c95cbb8b9c2baefe1c6985c0b29a9055e/Cython-0.29.36-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl.metadata
Downloading Cython-0.29.36-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl.metadata (3.1 kB)
Collecting pyyaml<6.0
Downloading PyYAML-5.4.1.tar.gz (175 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 175.1/175.1 kB 5.8 MB/s eta 0:00:00
Preparing metadata (pyproject.toml) ... done
Downloading Cython-0.29.36-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl (1.9 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.9/1.9 MB 49.7 MB/s eta 0:00:00
Building wheels for collected packages: pyyaml
Building wheel for pyyaml (pyproject.toml) ... done
Created wheel for pyyaml: filename=PyYAML-5.4.1-cp311-cp311-linux_x86_64.whl size=45658 sha256=53aa5944120645f9f03c4edd882ca0be069c7125935c64919305d9e285cd9625
Stored in directory: /home/automaton/.cache/pip/wheels/2f/2f/af/a062a8f866dc44812a825da11175d50d7f255900f3b38c79b5
Successfully built pyyaml
Installing collected packages: pyyaml, Cython
Successfully installed Cython-0.29.36 pyyaml-5.4.1
$ python -V
Python 3.11.5 Note that you'll also have to install the wheel package in order to avoid complaints about "bdist_wheel" missing on install: pip install wheel You should install wheel before trying to build the PyYAML package. |
ae19bb7
to
8f5f00d
Compare
8f5f00d
to
0ca0ddf
Compare
Thanks Bret for the latest investigation. Your suggested workaround did solve the ccm installation issue. Please review the latest commit and approve. I'll then run Jenkins tests using master branch. |
1055314
to
0ca0ddf
Compare
…, before PYTHON-1297 gets merged
d4a4e0d
to
59b36ce
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yup, this looks good to me. Only one small (and not really required) comment about documenting the weirdness with older PyYAML < 6.0 installs on Python 3.x but otherwise I think we're good here.
@@ -167,6 +167,8 @@ def initializeEnvironment() { | |||
sudo apt-get install socat | |||
pip install --upgrade pip | |||
pip install -U setuptools | |||
pip install wheel | |||
pip install "Cython<3.0" "pyyaml<6.0" --no-build-isolation |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Might be worth adding a short comment here about why this command is being invoked this way
@@ -186,7 +188,7 @@ def initializeEnvironment() { | |||
} | |||
|
|||
sh label: 'Install unit test modules', script: '''#!/bin/bash -lex | |||
pip install nose-ignore-docstring nose-exclude service_identity | |||
pip install pynose nose-ignore-docstring nose-exclude service_identity |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice, interesting find! It's at least a better short-term bridge until we can get onto pytest
d8ca18d
to
a3051f3
Compare
…sync_with_upstream_3.29.1 version 3.29.0 * tag '3.29.0' of https://github.com/datastax/python-driver: Documentation (and other) updates for 3.29.0 (datastax#1194) PYTHON-1331 ssl.match_hostname() is deprecated in 3.7 (datastax#1191) PYTHON-1371 Add explicit exception type for serialization failures (datastax#1193) Remove outdated Python pre-3.7 references (datastax#1186) PYTHON-1368 Avoid installing DSE deps + executing DSE tests for Python 3.12 (datastax#1188) PYTHON-1366 Handle removal of asyncore in Python 3.12 (datastax#1187) Removed backup(.bak) files (datastax#1185) docs: Fix typo in add_callbacks (datastax#1177) Remove some remnants of Python2 (datastax#1172) PYTHON-1313 Fix asyncio removals in Python 3.10 (datastax#1179) PYTHON-1364 Fix ssl.wrap_socket errors (from eventlet) for Python 3.12 (datastax#1181) Add Jenkins support for Python 3.12.0 (datastax#1180) Update redirects in docs.yaml (datastax#1178) Jenkins using new python versions in the matrix (datastax#1174) Update docs.yaml to point to most recent 3.28.0 docs changes CONN-38 Notes for 3.28.0 on PYTHON-1350 (datastax#1167) Include docs for 3.28.0 Fixed non-valid rst in README
No description provided.