Only install numpy and scipy from wheels in build jobs (Qiskit#1449)

* Only install numpy and scipy from wheels in build jobs With recent changes to the upstream numpy and scipy packaging our CI build jobs are spending most of their time build numpy and/or scipy from source to test the built wheels function correctly. This is unecessary as there are compatible wheels available on pypi. To correct this issue this commit updates the cibuildwheel config used in the wheel build jobs (both for ci and release artifacts) to only use binary wheels for numpy and scipy before testing. * Use manylinux 2014 for py310 wheel builds In the previous commit we switched to the test configuration to install numpy and scipy only from binary pre-compiled wheel packages. However, for scipy and numpy starting in Python 3.10 they only package for manylinux2014 while for aer we were still using manylinux2010 for all the builds. To enable building on python 3.10 we need to use manylinux2014, but for compatibility we still need to use manylinux2010. This commit makes this change so for python 3.6-3.9 we build on manylinux2010 and for python 3.10 we build on manylinux 2014. To facilitate this change the configuration for cibuildwheel is changed to leverage the pyproject.toml to set the configuration for cibuildwheel instead of using separate environment variables. This centralizes the configuration and makes it easier to work with. * Bump cibuildwheel version The pyproject.toml based configuration didn't support overrides (which was leveraged in the previous commit to use a different image for py310) until v2.2.0 and we were previously using a version older than that. This commit bumps the cibuildwheel version to the 2.2.x release to enable the use of this option. * Use manylinux2010 for all i686 builds The i686 py310 job was failing because centos7 (which manylinux2014 is based off of) doesn't have openblas packaged for i686. We orignally switched both the x86_64 and i686 py310 jobs to use manylinux2014 because numpy and scipy made python 3.10 manylinux2014 and only publish manylinux2014 wheels. However, numpy/scipy don't support i686 on py310 and we already skip test installs for py310. So regardless we have to build numpy from source to build aer. So using manylinux2010 which is centos6 based will enable us to at least build because there is an openblas package available.
hhorii · Feb 9, 2022 · 0b0913c · 0b0913c
1 parent c733c5c
commit 0b0913c
Show file tree

Hide file tree

Showing 3 changed files with 31 additions and 26 deletions.
diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml
@@ -132,14 +132,8 @@ jobs:
         uses: microsoft/setup-msbuild@v1.0.2
         if: runner.os == 'Windows'
       - name: Install deps
-        run: python -m pip install -U cibuildwheel==2.1.2
+        run: python -m pip install -U cibuildwheel==2.2.2
       - name: Build Wheels
-        env:
-          CIBW_BEFORE_ALL_LINUX: "yum install -y openblas-devel"
-          CIBW_SKIP: "pp*"
-          CIBW_TEST_SKIP: "cp310-win32 cp310-manylinux_i686"
-          CIBW_TEST_COMMAND: "python {project}/tools/verify_wheels.py"
-          CIBW_ENVIRONMENT_WINDOWS: "CMAKE_GENERATOR='Visual Studio 16 2019'"
         run: cibuildwheel --output-dir wheelhouse
       - uses: actions/upload-artifact@v2
         with:
@@ -156,10 +150,9 @@ jobs:
         with:
           python-version: 3.8
       - name: Install deps
-        run: python -m pip install -U cibuildwheel==2.1.2
+        run: python -m pip install -U cibuildwheel==2.2.2
       - name: Build Wheels
         env:
-          CIBW_SKIP: "pp*"
           CIBW_ARCHS_MACOS: arm64
         run: cibuildwheel --output-dir wheelhouse
       - uses: actions/upload-artifact@v2

diff --git a/.github/workflows/deploy.yml b/.github/workflows/deploy.yml
@@ -18,16 +18,9 @@ jobs:
           python-version: '3.7'
       - name: Install cibuildwheel
         run: |
-          python -m pip install cibuildwheel==2.1.2
+          python -m pip install cibuildwheel==2.2.2
       - name: Build wheels
-        env:
-          CIBW_BEFORE_ALL_LINUX: "yum install -y openblas-devel"
-          CIBW_SKIP: "pp*"
-          CIBW_TEST_SKIP: "cp310-win32 cp310-manylinux_i686"
-          CIBW_TEST_COMMAND: "python {project}/tools/verify_wheels.py"
-          CIBW_ENVIRONMENT_WINDOWS: "CMAKE_GENERATOR='Visual Studio 16 2019'"
-        run: |
-          python -m cibuildwheel --output-dir wheelhouse
+        run: python -m cibuildwheel --output-dir wheelhouse
       - uses: actions/upload-artifact@v2
         with:
           path: ./wheelhouse/*.whl
@@ -60,14 +53,12 @@ jobs:
           platforms: all
       - name: Install cibuildwheel
         run: |
-          python -m pip install cibuildwheel==2.1.2 twine
+          python -m pip install cibuildwheel==2.2.2 twine
       - name: Build wheels
         run: |
           python -m cibuildwheel --output-dir wheelhouse
         env:
           CIBW_BEFORE_ALL_LINUX: "yum install -y https://dl.fedoraproject.org/pub/epel/7/aarch64/Packages/e/epel-release-7-12.noarch.rpm && yum install -y openblas-devel"
-          CIBW_SKIP: "pp*"
-          CIBW_TEST_COMMAND: "python {project}/tools/verify_wheels.py"
           CIBW_ARCHS_LINUX: aarch64
       - uses: actions/upload-artifact@v2
         with:
@@ -90,10 +81,9 @@ jobs:
         with:
           python-version: 3.8
       - name: Install deps
-        run: python -m pip install -U cibuildwheel==2.1.2 twine
+        run: python -m pip install -U cibuildwheel==2.2.2 twine
       - name: Build Wheels
         env:
-          CIBW_SKIP: "pp*"
           CIBW_ARCHS_MACOS: arm64
         run: cibuildwheel --output-dir wheelhouse
       - uses: actions/upload-artifact@v2
@@ -141,13 +131,12 @@ jobs:
         if: runner.os == 'Windows'
       - name: Install cibuildwheel
         run: |
-          python -m pip install cibuildwheel==2.1.2
+          python -m pip install cibuildwheel==2.2.2
       - name: Build wheels
         env:
           CIBW_BEFORE_ALL: "yum install -y yum-utils wget && wget https://developer.download.nvidia.com/compute/cuda/10.1/Prod/local_installers/cuda-repo-rhel6-10-1-local-10.1.243-418.87.00-1.0-1.x86_64.rpm && rpm -i cuda-repo-rhel6-10-1-local-10.1.243-418.87.00-1.0-1.x86_64.rpm && yum clean all && yum -y install cuda-10-1 openblas-devel"
-          CIBW_SKIP: "*-manylinux_i686 pp*"
+          CIBW_SKIP: "*-manylinux_i686 pp* *musllinux*"
           CIBW_ENVIRONMENT: QISKIT_AER_PACKAGE_NAME=qiskit-aer-gpu AER_THRUST_BACKEND=CUDA CUDACXX=/usr/local/cuda/bin/nvcc
-          CIBW_TEST_COMMAND: "python {project}/tools/verify_wheels.py"
         run: |
           python -m cibuildwheel --output-dir wheelhouse
       - uses: actions/upload-artifact@v2

diff --git a/pyproject.toml b/pyproject.toml
@@ -11,3 +11,26 @@ requires = [
   "numpy==1.16.3; python_version<='3.7' and platform_machine!='aarch64' or platform_python_implementation=='PyPy'",
 ]
 build-backend = "setuptools.build_meta"
+
+[tool.cibuildwheel]
+manylinux-x86_64-image = "manylinux2014"
+manylinux-i686-image = "manylinux2010"
+skip = "pp* *musllinux*"
+test-skip = "cp310-win32 cp310-manylinux_i686"
+test-command = "python {project}/tools/verify_wheels.py"
+# We need to use pre-built versions of Numpy and Scipy in the tests; they have a
+# tendency to crash if they're installed from source by `pip install`, and since
+# Numpy 1.22 there are no i686 wheels, so we force pip to use older ones without
+# restricting any dependencies that Numpy and Scipy might have.
+before-test = "pip install --only-binary=numpy,scipy numpy scipy"
+
+[tool.cibuildwheel.linux]
+before-all = "yum install -y openblas-devel"
+
+[tool.cibuildwheel.windows]
+environment = "CMAKE_GENERATOR='Visual Studio 16 2019'"
+
+# For older Pythons we maintain our manylinux2010 support.
+[[tool.cibuildwheel.overrides]]
+select = "cp3{6,7,8,9}-manylinux*"
+manylinux-x86_64-image = "manylinux2010"