-
Notifications
You must be signed in to change notification settings - Fork 218
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Tracking issue for manylinux2014 rollout #338
Comments
Minor comment, and probably too late now that the PEP has been accepted, but I feel it's relevant to avoid confusion: could we call it |
Regarding this task:
Is switching pip to use |
manylinux2014 platform tag is official per PEP599 see also pypa/manylinux#338
manylinux2014 platform tag is official per PEP599 see also pypa/manylinux#338
No, definitely not, just figured it'd be nice to roll this in together since it seems like both are being worked on at the same time. |
manylinux2014 platform tag is official per PEP599 see also pypa/manylinux#338
manylinux2014 platform tag is official per PEP599 see also pypa/manylinux#338
I just merged pypi/warehouse#6684, |
I've taken the liberty to add pypa/pip#7102 to the rollout. |
Support in |
FYI |
multibuild uses Travis CI to do builds. Some time ago Travis CI got support for doing AArch64 builds. Can you add aarch64 builds with manylinux2014 so users will not have to compile wheels on their systems? |
@hrw |
@di I wonder how much work it would be to move from "send us source+wheels and we publish" to "send us source, we build wheels and publish". Now it can take 45 minutes to do "pip install scikit-learn" on aarch64 compared to few seconds on x86-64 machine. Not counting time needed to find out which development packages to install first to get those Python packages built. |
@hrw A lot. PyPI currently does zero introspection or execution of any user-supplied code. However, there are other repositories (piwheels.org, conda-forge.org) that do something similar, so it's not unprecedented. However I think this is off-topic for this issue, a better place to discuss further would be in the PyPI issue tracker. |
@hrw - for some wheels, it's not very difficult - if they build useful wheels with the default recipe ( |
|
I've just created manylinux2014 repositories for x86_64, i686 and aarch64: Let me know if there's anything I've missed. I'm also happy to add you to the quay.io organisation - are you mayeut on quay.io as well? |
@takluyver, yes, I'm mayeut on quay.io as well, thanks ! |
OK, I've invited you. If you create any more repositories, remember to give the bot account write permission for them - you can copy the details from any existing ones. |
First draft for README pushed to #348 |
#348 has been merged in branch |
Twine can upload manylinux2014 wheels: https://test.pypi.org/project/pybase64/1.0.1/#files |
Is there a central location where all the supported pypa/manylinux* images are catalogued? The README here does not list them, and searching on quay.io does not seem to return anything for |
@mattip, @jayfurmanek, images for manylinux2014 are catalogued in the README of the manylinux2014 branch https://github.com/pypa/manylinux/tree/manylinux2014 |
Now that there is a manylinux2014 container the screwdrivercd project and the python-screwdrivercd packages used by the python_sdv4_templates screwdrivercd python templates have been updated to allow generating manylinux2010 and manylinux2014 packages and publish them using twine using the new containers. We have updated the redislite package to publish both formats. I'd be interested in knowing if we're missing anything and if/when it might be appropriate to start promoting the manylinux2014 support. |
@dwighthubbard next step will be building non-x86 wheels I hope. |
I see you are not publishing a manylinux1 wheel. Are you getting complaints from people with old pip versions who do |
Perhaps your top level README.rst file should already reference https://www.python.org/dev/peps/pep-0599 ? I got confused by manylinux2014 being listed but not referenced via a PEP. |
Is there a way to force older pip versions to install manylinux2014 wheels? I thought |
Oh, another data point:
There seems to be a PR against cpython to update the bundled versions here: python/cpython#16782 |
Now that Thanks to @brainwane, @mayeut, @lkollar, @pradyunsg, @takluyver, @zachW, and especially @pfmoore for everyone's work here (as well as anyone I may be forgetting). |
@di I still see this on the platform compat tagging page - am I missing something? |
@brainwane In pypa/packaging.python.org#656 I reworded the caveat to say "manylinux2014" instead of "manylinux2010", is that not what you intended? |
@di could we take a moment sometime this week to refresh this issue, see whether anything in the checklist is finished or obsolete, and see what (if anything) is still waiting to close this? |
I think we can call this finished! Thanks again everyone for your contributions here. |
Per pypa/manylinux#338 , manylinux2014 is now implemented in its entirety. Per PEP 1, since the reference implementation is now complete and incorporated into the main source code repository, this commit changes the status to "Final". Signed-off-by: Sumana Harihareswara <sh@changeset.nyc>
Per pypa/manylinux#338 , manylinux2014 is now implemented in its entirety.
Per pypa/manylinux#338 , manylinux2014 is now implemented in its entirety.
Now that PEP 599 has been accepted to adopt the
manylinux2014
standard, we need a tracking issue for implementation (I'm reusing structure and prose from #179).The dates are guesses but would (as I figure) help us get all the infrastructure in place well before Nov 2020 when CentOS 6 (supported in
manylinux2010
) will go EOL, and in time for the August 2020 date that @gunan asked for; please argue with me. ;-)There are a number of further steps needed before folks can actually make use of that new baseline, and the order of operations matters (since building
manylinux2014
wheels isn't very useful if installers won't install them):Prep (ideally done by 2 September 2019):
Essential client support (ideally done by 15 September 2019):
pip
19.x to installmanylinux2014
wheelspackaging.tags
to detectmanylinux2014
compatibility (Add support for manylinux2014 packaging#186)packaging
(19.2)manylinux2014
support topip._internal.pep425tags
(Add manylinux2014 support pip#7102)pip._internal.pep425tags
w/packaging.tags
(Replace pep425tags w/ packaging.tags pip#6908)pip
(19.3 due out in week 41, so, by 19 October)Enable publication of
manylinux2014
wheel archives (ideally done by 30 October 2019):Warehouse
(PyPI) to acceptmanylinux2014
wheel uploads (Allow manylinux2014 wheel upload pypi/warehouse#6684)auditwheel
to emitmanylinux2014
wheel archives (manylinux2014 policy auditwheel#192)auditwheel
to allow new architectures introduced withmanylinux2014
(Add support for new architectures in manylinux2014 auditwheel#202)auditwheel
(3.0.0.0rc1 Release 3.0.0 auditwheel#205)auditwheel
(3.0.0 Release 3.0.0 auditwheel#205)manylinux2014
build environment, probably a Docker image (manylinux2014 images #348)quay.io
repositoriestwine
will uploadmanylinux2014
wheel archivesManagement of transition from
manylinux1
andmanylinux2010
(ideally done by 15 December 2019):manylinux2014
, with the caveat that it doesn't actually have broad tooling support yet, and to explain (per @zackw's note) pros & cons of switching tomanylinux2014
, e.g., future security patches, old Linux distributions new wheels won't work with, etc.Additional projects to consider once core capability support has rolled out (ideally done by 1 March 2020):
pipenv
to installmanylinux2014
wheels (needed due topipenv
vendoringpip
, so the support depends on the version ofpipenv
being used, rather than the version ofpip
in the host or target environments)manylinux2014
instead ofmanylinux2010
in cibuildwheel (Add support for manylinux2014 cibuildwheel#216, Add manylinux2014 x86_64/i686 support cibuildwheel#215)manylinux2014
instead ofmanylinux1
ormanylinux2010
in multibuildmanylinux2014
,manylinux2010
andmanylinux1
images in dockcrossTagging @di as the PEP 599 author, @pfmoore as the BDFL-Delegate who requested further planning on rollout, and @ewdurbin from the PyPI/PSF side of things.
The text was updated successfully, but these errors were encountered: