Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

macOS CI: Update macOS runner to macOS 11 #420

Merged
merged 2 commits into from
Dec 10, 2023

Conversation

aoirint
Copy link
Contributor

@aoirint aoirint commented Nov 6, 2023

This pull request updates macOS version on test CI to macOS 11.

GitHub no longer provides macOS 10.15 CI runners after 2022-12-01. And macOS 10.15 Catalina seems to be End-Of-Life.

All recent CI jobs on macOS 10.15 were failed due to timeout.

test (macos-10.15, 3.6, x64)
This request was automatically failed because there were no enabled runners online to process the request for more than 1 days.

We need to update macOS version to keep testing on CI.

About macOS 11 and pypy 3.7 exclusion

macOS 11 and pypy 3.7 combination test remains disabled on this pull request.

macOS 10.15 and pypy 3.7 combination test was disabled due to compilation error.

It seems able to compile numpy 1.21.6 from source on macOS 11 and pypy 3.7 but the compilation of numpy takes a long time, 5 minutes 26 seconds.
So I keep it disabled but if you want to test it, please edit or mention it.

CI log: pip install numpy pytest on macOS 11 and pypy 3.7
$ pip install numpy pytest
Collecting numpy
  Downloading numpy-1.21.6.zip (10.3 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 10.3/10.3 MB 26.9 MB/s eta 0:00:00
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Collecting pytest
  Downloading pytest-7.4.3-py3-none-any.whl.metadata (7.9 kB)
Collecting iniconfig (from pytest)
  Downloading iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Collecting packaging (from pytest)
  Downloading packaging-23.2-py3-none-any.whl.metadata (3.2 kB)
Collecting pluggy<2.0,>=0.12 (from pytest)
  Downloading pluggy-1.2.0-py3-none-any.whl.metadata (4.4 kB)
Collecting exceptiongroup>=1.0.0rc8 (from pytest)
  Downloading exceptiongroup-1.1.3-py3-none-any.whl.metadata (6.1 kB)
Collecting tomli>=1.0.0 (from pytest)
  Downloading tomli-2.0.1-py3-none-any.whl (12 kB)
Collecting importlib-metadata>=0.12 (from pytest)
  Downloading importlib_metadata-6.7.0-py3-none-any.whl.metadata (4.9 kB)
Collecting zipp>=0.5 (from importlib-metadata>=0.12->pytest)
  Downloading zipp-3.15.0-py3-none-any.whl (6.8 kB)
Collecting typing-extensions>=3.6.4 (from importlib-metadata>=0.12->pytest)
  Downloading typing_extensions-4.7.1-py3-none-any.whl.metadata (3.1 kB)
Downloading pytest-7.4.3-py3-none-any.whl (325 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 325.1/325.1 kB 18.5 MB/s eta 0:00:00
Downloading exceptiongroup-1.1.3-py3-none-any.whl (14 kB)
Downloading importlib_metadata-6.7.0-py3-none-any.whl (22 kB)
Downloading pluggy-1.2.0-py3-none-any.whl (17 kB)
Downloading packaging-23.2-py3-none-any.whl (53 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 53.0/53.0 kB 6.3 MB/s eta 0:00:00
Downloading typing_extensions-4.7.1-py3-none-any.whl (33 kB)
Building wheels for collected packages: numpy
  Building wheel for numpy (pyproject.toml): started
  Building wheel for numpy (pyproject.toml): still running...
  Building wheel for numpy (pyproject.toml): still running...
  Building wheel for numpy (pyproject.toml): still running...
  Building wheel for numpy (pyproject.toml): finished with status 'done'
  Created wheel for numpy: filename=numpy-1.21.6-pp37-pypy37_pp73-macosx_11_0_x86_64.whl size=4859840 sha256=05e4a537f9ef545651d3bc011a5ff4bf31b7f34963c0c1e6e0d0f87bf7581248
  Stored in directory: /Users/runner/Library/Caches/pip/wheels/ab/24/3b/c73b1bc7595de80fdfbce45adbabdb203ee622715ca4d45ffe
Successfully built numpy
Installing collected packages: zipp, typing-extensions, tomli, packaging, numpy, iniconfig, exceptiongroup, importlib-metadata, pluggy, pytest
Successfully installed exceptiongroup-1.1.3 importlib-metadata-6.7.0 iniconfig-2.0.0 numpy-1.21.6 packaging-23.2 pluggy-1.2.0 pytest-7.4.3 tomli-2.0.1 typing-extensions-4.7.1 zipp-3.15.0

Notice:  A new release of pip is available: 22.0.4 -> 23.3.1
Notice:  To update, run: pip install --upgrade pip

@bastibe
Copy link
Owner

bastibe commented Nov 12, 2023

Thank you for the pull request. It seems that most tests currently fail with this change. Frankly, I don't have the time to look into it, but it seems unrelated to this change.

Would you perhaps have any insight into this?

@aoirint
Copy link
Contributor Author

aoirint commented Dec 1, 2023

Sorry for the late response.

The CI error ModuleNotFoundError: No module named '_soundfile' appears to be due to a change in the behavior of the Python package manager on Python 3.7.

Specifically, the handling of files generated during an editable installation of a package seems to have changed.
On Python 3.6, they were generated in the current directory.
However, on Python 3.7 and later, they are stored in a temporary directory and discarded. As a result, _soundfile.py generated by cffi library is not included in the editable installed package.

I found a workaround that fixes most errors except Windows + Python 3.11 which has other causes for not passing the test.
Also #419 and #420 are needed.

This patch generates _soundfile.py before the editable installation of this package.

diff --git a/.github/workflows/python-package.yml b/.github/workflows/python-package.yml
index 844bf5c..88aacff 100644
--- a/.github/workflows/python-package.yml
+++ b/.github/workflows/python-package.yml
@@ -40,6 +40,10 @@ jobs:
           architecture: ${{ matrix.architecture }}
       - name: Install requirements
         run: pip install numpy pytest
+      - name: "Workaround: Generate _soundfile.py explicitly"
+        run: |
+          pip install cffi>=1.0
+          python soundfile_build.py
       - name: Install editable package
         run: pip install --editable . --verbose
       - name: Run tests

If you don't mind, I am going to create a new pull request to add this workaround.
If you want to resolve all CI related issues (including #419, #420, this workaround, and maybe Windows + Python 3.11 test issue) in one pull request, I will follow your instruction.

@bastibe bastibe merged commit db9ef8b into bastibe:master Dec 10, 2023
3 of 31 checks passed
@bastibe
Copy link
Owner

bastibe commented Dec 10, 2023

Thank you so much for your help with these issues!

I have merged #419 and #420, and I'd be grateful for the additional merge request for fixing the 3.7+ CI issue, as well as possibly the Windows/3.11 issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants