Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update build, publish mechanism #96

Merged
merged 6 commits into from
Jul 3, 2024
Merged

Conversation

rajadain
Copy link
Member

@rajadain rajadain commented Jun 28, 2024

Overview

Since gwlf-e is primarily installed on Linux VMs for MMW, it is useful for there to be binary Linux wheels for common Python versions available.

This PR builds and publishes binary Linux wheels for Python 3.8 (current version on MMW), 3.9, and 3.10 (next version on MMW).

It also adds a GitHub Action to do releases instead of the previous manual process, and updates the README. PyPI has been configured to trust this new GitHub Action as a publisher as well.

Demo

https://test.pypi.org/project/gwlf-e/3.2.0/

Testing Instructions

  • Ensure you are in a Python 3.10 environment and have pipenv installed
  • Check out this repo
  • Run pipenv sync --dev to install all dependencies
  • Run pipenv run python -m build --sdist to build a source distribution
  • Run pipenv run python -m cibuildwheel --output-dir dist to build wheels
  • In a Python 3.10 virtual environment, install the gwlf-e 3.2.0 from TestPyPI
    pip install -i https://test.pypi.org/simple/ gwlf-e==3.2.0
    
    • Ensure it works

Closes #92

@rajadain rajadain force-pushed the tt/92/deploy-via-github-actions branch from aa5720b to ee239a6 Compare June 28, 2024 20:04
@JN-Hernandez
Copy link

JN-Hernandez commented Jul 1, 2024

Dependency install fails for me unfortunately:

(gqlfe-env) ➜  gwlf-e git:(develop) ✗ python --version
Python 3.10.14

(gqlfe-env) ➜  gwlf-e git:(develop) ✗ pipenv --version
pipenv, version 2023.12.1

(gqlfe-env) ➜  gwlf-e git:(develop) ✗ pipenv sync --dev
Courtesy Notice: Pipenv found itself running within a virtual environment, so it will automatically use that environment, instead of creating its own for any project. You can set PIPENV_IGNORE_VIRTUALENVS=1 to force pipenv to ignore that environment and create its own instead. You can set PIPENV_VERBOSITY=-1 to suppress this warning.
Installing dependencies from Pipfile.lock (783625)...
[pipenv.exceptions.InstallError]: Collecting certifi==2021.5.30 (from -r /var/folders/ds/knxd_rlj057g0xx7vh_ry0q80000gn/T/pipenv-6jrl4avz-requirements/pipenv-bxyzxcl4-hashed-reqs.txt (line 1))
[pipenv.exceptions.InstallError]:   Using cached certifi-2021.5.30-py2.py3-none-any.whl (145 kB)
[pipenv.exceptions.InstallError]: Collecting funcsigs==1.0.2 (from -r /var/folders/ds/knxd_rlj057g0xx7vh_ry0q80000gn/T/pipenv-6jrl4avz-requirements/pipenv-bxyzxcl4-hashed-reqs.txt (line 2))
[pipenv.exceptions.InstallError]:   Using cached funcsigs-1.0.2-py2.py3-none-any.whl (17 kB)
[pipenv.exceptions.InstallError]: ERROR: Ignored the following versions that require a different python version: 0.36.0 Requires-Python >=3.6,<3.10; 0.37.0 Requires-Python >=3.7,<3.10
[pipenv.exceptions.InstallError]: ERROR: Could not find a version that satisfies the requirement llvmlite==0.37.0 (from versions: 0.2.0, 0.2.1, 0.2.2, 0.4.0, 0.5.0, 0.6.0, 0.7.0, 0.8.0, 0.9.0, 0.10.0, 0.11.0, 0.12.0.1, 0.12.1, 0.13.0, 0.14.0, 0.15.0, 0.16.0, 0.17.0, 0.17.1, 0.18.0, 0.19.0, 0.20.0, 0.21.0, 0.22.0, 0.23.0, 0.23.2, 0.24.0, 0.25.0, 0.26.0, 0.27.0, 0.27.1, 0.28.0, 0.29.0, 0.30.0, 0.31.0, 0.32.0, 0.32.1, 0.33.0, 0.34.0, 0.35.0, 0.38.0, 0.38.1, 0.39.0, 0.39.1, 0.40.0rc1, 0.40.0, 0.40.1rc1, 0.40.1, 0.41.0rc1, 0.41.0, 0.41.1, 0.42.0rc1, 0.42.0, 0.43.0rc1, 0.43.0)
[pipenv.exceptions.InstallError]: ERROR: No matching distribution found for llvmlite==0.37.0
ERROR: Couldn't install package: {}
 Package installation failed...

At first I thought it was because of being on Python 3.10.14 instead of 3.10, so I tried recreating my virtual environment to 3.10.0... still no dice:

(gqlfe-env) ➜  gwlf-e git:(develop) ✗ pyenv virtualenv 3.10.0 gqlfe-en

(gqlfe-env) ➜  gwlf-e git:(develop) ✗ pyenv activate gqlfe-en

(gqlfe-en) ➜  gwlf-e git:(develop) ✗ pipenv sync --dev
Courtesy Notice: Pipenv found itself running within a virtual environment, so it will automatically use that environment, instead of creating its own for any project. You can set PIPENV_IGNORE_VIRTUALENVS=1 to force pipenv to ignore that environment and create its own instead. You can set PIPENV_VERBOSITY=-1 to suppress this warning.
Installing dependencies from Pipfile.lock (783625)...
[pipenv.exceptions.InstallError]: Collecting certifi==2021.5.30 (from -r /var/folders/ds/knxd_rlj057g0xx7vh_ry0q80000gn/T/pipenv-b0q59aph-requirements/pipenv-u6ozy720-hashed-reqs.txt (line 1))
[pipenv.exceptions.InstallError]:   Using cached certifi-2021.5.30-py2.py3-none-any.whl (145 kB)
[pipenv.exceptions.InstallError]: Collecting funcsigs==1.0.2 (from -r /var/folders/ds/knxd_rlj057g0xx7vh_ry0q80000gn/T/pipenv-b0q59aph-requirements/pipenv-u6ozy720-hashed-reqs.txt (line 2))
[pipenv.exceptions.InstallError]:   Using cached funcsigs-1.0.2-py2.py3-none-any.whl (17 kB)
[pipenv.exceptions.InstallError]: ERROR: Ignored the following versions that require a different python version: 0.36.0 Requires-Python >=3.6,<3.10; 0.37.0 Requires-Python >=3.7,<3.10
[pipenv.exceptions.InstallError]: ERROR: Could not find a version that satisfies the requirement llvmlite==0.37.0 (from versions: 0.2.0, 0.2.1, 0.2.2, 0.4.0, 0.5.0, 0.6.0, 0.7.0, 0.8.0, 0.9.0, 0.10.0, 0.11.0, 0.12.0.1, 0.12.1, 0.13.0, 0.14.0, 0.15.0, 0.16.0, 0.17.0, 0.17.1, 0.18.0, 0.19.0, 0.20.0, 0.21.0, 0.22.0, 0.23.0, 0.23.2, 0.24.0, 0.25.0, 0.26.0, 0.27.0, 0.27.1, 0.28.0, 0.29.0, 0.30.0, 0.31.0, 0.32.0, 0.32.1, 0.33.0, 0.34.0, 0.35.0, 0.38.0, 0.38.1, 0.39.0, 0.39.1, 0.40.0rc1, 0.40.0, 0.40.1rc1, 0.40.1, 0.41.0rc1, 0.41.0, 0.41.1, 0.42.0rc1, 0.42.0, 0.43.0rc1, 0.43.0)
[pipenv.exceptions.InstallError]: ERROR: No matching distribution found for llvmlite==0.37.0
ERROR: Couldn't install package: {}
 Package installation failed...

Removed my virtual environment, and it made further progress but still ultimately did not succeed:

➜  gwlf-e git:(develop) ✗ pipenv sync --dev
Creating a virtualenv for this project...
Pipfile: /Users/jnhernandez/projects/gwlf-e/Pipfile
Using /Users/jnhernandez/.pyenv/versions/3.9.19/envs/cirrus-3.9/bin/python3 (3.9.19) to create virtualenv...
⠧ Creating virtual environment...created virtual environment CPython3.9.19.final.0-64 in 1981ms
  creator CPython3Posix(dest=/Users/jnhernandez/.local/share/virtualenvs/gwlf-e-NDAS7vRc, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/Users/jnhernandez/Library/Application Support/virtualenv)
    added seed packages: pip==23.3.1, setuptools==69.0.2, wheel==0.42.0
  activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator

✔ Successfully created virtual environment!
Virtualenv location: /Users/jnhernandez/.local/share/virtualenvs/gwlf-e-NDAS7vRc
Installing dependencies from Pipfile.lock (783625)...
[pipenv.exceptions.InstallError]: Collecting certifi==2021.5.30 (from -r /var/folders/ds/knxd_rlj057g0xx7vh_ry0q80000gn/T/pipenv-8wsh245r-requirements/pipenv-an7wqvuy-hashed-reqs.txt (line 1))
[pipenv.exceptions.InstallError]:   Using cached certifi-2021.5.30-py2.py3-none-any.whl (145 kB)
[pipenv.exceptions.InstallError]: Collecting funcsigs==1.0.2 (from -r /var/folders/ds/knxd_rlj057g0xx7vh_ry0q80000gn/T/pipenv-8wsh245r-requirements/pipenv-an7wqvuy-hashed-reqs.txt (line 2))
[pipenv.exceptions.InstallError]:   Using cached funcsigs-1.0.2-py2.py3-none-any.whl (17 kB)
[pipenv.exceptions.InstallError]: Collecting llvmlite==0.37.0 (from -r /var/folders/ds/knxd_rlj057g0xx7vh_ry0q80000gn/T/pipenv-8wsh245r-requirements/pipenv-an7wqvuy-hashed-reqs.txt (line 3))
[pipenv.exceptions.InstallError]:   Downloading llvmlite-0.37.0.tar.gz (125 kB)
[pipenv.exceptions.InstallError]:      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 126.0/126.0 kB 1.4 MB/s eta 0:00:00
[pipenv.exceptions.InstallError]:   Preparing metadata (setup.py): started
[pipenv.exceptions.InstallError]:   Preparing metadata (setup.py): finished with status 'done'
[pipenv.exceptions.InstallError]: Collecting nose==1.3.7 (from -r /var/folders/ds/knxd_rlj057g0xx7vh_ry0q80000gn/T/pipenv-8wsh245r-requirements/pipenv-an7wqvuy-hashed-reqs.txt (line 4))
[pipenv.exceptions.InstallError]:   Downloading nose-1.3.7-py3-none-any.whl (154 kB)
[pipenv.exceptions.InstallError]:      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 154.7/154.7 kB 3.0 MB/s eta 0:00:00
[pipenv.exceptions.InstallError]: Collecting numba==0.54.0 (from -r /var/folders/ds/knxd_rlj057g0xx7vh_ry0q80000gn/T/pipenv-8wsh245r-requirements/pipenv-an7wqvuy-hashed-reqs.txt (line 5))
[pipenv.exceptions.InstallError]:   Downloading numba-0.54.0.tar.gz (2.2 MB)
[pipenv.exceptions.InstallError]:      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.2/2.2 MB 4.7 MB/s eta 0:00:00
[pipenv.exceptions.InstallError]:   Preparing metadata (setup.py): started
[pipenv.exceptions.InstallError]:   Preparing metadata (setup.py): finished with status 'done'
[pipenv.exceptions.InstallError]: Collecting numpy==1.20.3 (from -r /var/folders/ds/knxd_rlj057g0xx7vh_ry0q80000gn/T/pipenv-8wsh245r-requirements/pipenv-an7wqvuy-hashed-reqs.txt (line 6))
[pipenv.exceptions.InstallError]:   Downloading numpy-1.20.3.zip (7.8 MB)
[pipenv.exceptions.InstallError]:      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7.8/7.8 MB 12.3 MB/s eta 0:00:00
[pipenv.exceptions.InstallError]:   Installing build dependencies: started
[pipenv.exceptions.InstallError]:   Installing build dependencies: finished with status 'done'
[pipenv.exceptions.InstallError]:   Getting requirements to build wheel: started
[pipenv.exceptions.InstallError]:   Getting requirements to build wheel: finished with status 'done'
[pipenv.exceptions.InstallError]:   Preparing metadata (pyproject.toml): started
[pipenv.exceptions.InstallError]:   Preparing metadata (pyproject.toml): finished with status 'done'
[pipenv.exceptions.InstallError]: Building wheels for collected packages: llvmlite, numba, numpy
[pipenv.exceptions.InstallError]:   Building wheel for llvmlite (setup.py): started
[pipenv.exceptions.InstallError]:   Building wheel for llvmlite (setup.py): finished with status 'error'
[pipenv.exceptions.InstallError]:   Running setup.py clean for llvmlite
[pipenv.exceptions.InstallError]:   Building wheel for numba (setup.py): started
[pipenv.exceptions.InstallError]:   Building wheel for numba (setup.py): finished with status 'error'
[pipenv.exceptions.InstallError]:   Running setup.py clean for numba
[pipenv.exceptions.InstallError]:   Building wheel for numpy (pyproject.toml): started
[pipenv.exceptions.InstallError]:   Building wheel for numpy (pyproject.toml): finished with status 'done'
[pipenv.exceptions.InstallError]:   Created wheel for numpy: filename=numpy-1.20.3-cp39-cp39-macosx_14_5_arm64.whl size=5712241 sha256=44b7a7b3b57f6dc3caf10152736af4c4807a66eceaac13ce058f7fe927a56286
[pipenv.exceptions.InstallError]:   Stored in directory: /Users/jnhernandez/Library/Caches/pipenv/wheels/7a/34/64/4b83a3f59d2eb2ba5757834a9c43b2e3389f3b8a309b4905ba
[pipenv.exceptions.InstallError]: Successfully built numpy
[pipenv.exceptions.InstallError]: Failed to build llvmlite numba
[pipenv.exceptions.InstallError]: error: subprocess-exited-with-error
[pipenv.exceptions.InstallError]:
[pipenv.exceptions.InstallError]:   × python setup.py bdist_wheel did not run successfully.
[pipenv.exceptions.InstallError]:   │ exit code: 1
[pipenv.exceptions.InstallError]:   ╰─> [11 lines of output]
[pipenv.exceptions.InstallError]:       running bdist_wheel
[pipenv.exceptions.InstallError]:       /Users/jnhernandez/.local/share/virtualenvs/gwlf-e-NDAS7vRc/bin/python /private/var/folders/ds/knxd_rlj057g0xx7vh_ry0q80000gn/T/pip-install-7gxgembv/llvmlite_7f36ee653a6340249ba23dca1d7b8d08/ffi/build.py
[pipenv.exceptions.InstallError]:       LLVM version... Traceback (most recent call last):
[pipenv.exceptions.InstallError]:         File "/private/var/folders/ds/knxd_rlj057g0xx7vh_ry0q80000gn/T/pip-install-7gxgembv/llvmlite_7f36ee653a6340249ba23dca1d7b8d08/ffi/build.py", line 220, in <module>
[pipenv.exceptions.InstallError]:           main()
[pipenv.exceptions.InstallError]:         File "/private/var/folders/ds/knxd_rlj057g0xx7vh_ry0q80000gn/T/pip-install-7gxgembv/llvmlite_7f36ee653a6340249ba23dca1d7b8d08/ffi/build.py", line 214, in main
[pipenv.exceptions.InstallError]:           main_posix('osx', '.dylib')
[pipenv.exceptions.InstallError]:         File "/private/var/folders/ds/knxd_rlj057g0xx7vh_ry0q80000gn/T/pip-install-7gxgembv/llvmlite_7f36ee653a6340249ba23dca1d7b8d08/ffi/build.py", line 134, in main_posix
[pipenv.exceptions.InstallError]:           raise RuntimeError(msg) from None
[pipenv.exceptions.InstallError]:       RuntimeError: Could not find a `llvm-config` binary. There are a number of reasons this could occur, please see: https://llvmlite.readthedocs.io/en/latest/admin-guide/install.html#using-pip for help.
[pipenv.exceptions.InstallError]:       error: command '/Users/jnhernandez/.local/share/virtualenvs/gwlf-e-NDAS7vRc/bin/python' failed with exit code 1
[pipenv.exceptions.InstallError]:       [end of output]
[pipenv.exceptions.InstallError]:
[pipenv.exceptions.InstallError]:   note: This error originates from a subprocess, and is likely not a problem with pip.
[pipenv.exceptions.InstallError]:   ERROR: Failed building wheel for llvmlite
[pipenv.exceptions.InstallError]:   error: subprocess-exited-with-error
[pipenv.exceptions.InstallError]:
[pipenv.exceptions.InstallError]:   × python setup.py bdist_wheel did not run successfully.
[pipenv.exceptions.InstallError]:   │ exit code: 1
[pipenv.exceptions.InstallError]:   ╰─> [8 lines of output]
[pipenv.exceptions.InstallError]:       Traceback (most recent call last):
[pipenv.exceptions.InstallError]:         File "<string>", line 2, in <module>
[pipenv.exceptions.InstallError]:         File "<pip-setuptools-caller>", line 34, in <module>
[pipenv.exceptions.InstallError]:         File "/private/var/folders/ds/knxd_rlj057g0xx7vh_ry0q80000gn/T/pip-install-7gxgembv/numba_9be0bc8325af49c98de3b2e35e3f1144/setup.py", line 414, in <module>
[pipenv.exceptions.InstallError]:           metadata['ext_modules'] = get_ext_modules()
[pipenv.exceptions.InstallError]:         File "/private/var/folders/ds/knxd_rlj057g0xx7vh_ry0q80000gn/T/pip-install-7gxgembv/numba_9be0bc8325af49c98de3b2e35e3f1144/setup.py", line 141, in get_ext_modules
[pipenv.exceptions.InstallError]:           import numpy.distutils.misc_util as np_misc
[pipenv.exceptions.InstallError]:       ModuleNotFoundError: No module named 'numpy'
[pipenv.exceptions.InstallError]:       [end of output]
[pipenv.exceptions.InstallError]:
[pipenv.exceptions.InstallError]:   note: This error originates from a subprocess, and is likely not a problem with pip.
[pipenv.exceptions.InstallError]:   ERROR: Failed building wheel for numba
[pipenv.exceptions.InstallError]: ERROR: Could not build wheels for llvmlite, numba, which is required to install pyproject.toml-based projects
ERROR: Couldn't install package: {}
 Package installation failed...

As a final effort, I set the PIPENV_IGNORE_VIRTUALENVS variable to 1 as indicated in the warning message:

(gqlfe-en) ➜  gwlf-e git:(develop) ✗ export PIPENV_IGNORE_VIRTUALENVS=1

(gqlfe-en) ➜  gwlf-e git:(develop) ✗ echo $PIPENV_IGNORE_VIRTUALENVS
1

(gqlfe-en) ➜  gwlf-e git:(develop) ✗ pipenv sync --dev
Installing dependencies from Pipfile.lock (783625)...
[pipenv.exceptions.InstallError]: Collecting certifi==2021.5.30 (from -r /var/folders/ds/knxd_rlj057g0xx7vh_ry0q80000gn/T/pipenv-x8601bnm-requirements/pipenv-gnh_n21n-hashed-reqs.txt (line 1))
[pipenv.exceptions.InstallError]:   Using cached certifi-2021.5.30-py2.py3-none-any.whl (145 kB)
[pipenv.exceptions.InstallError]: Collecting funcsigs==1.0.2 (from -r /var/folders/ds/knxd_rlj057g0xx7vh_ry0q80000gn/T/pipenv-x8601bnm-requirements/pipenv-gnh_n21n-hashed-reqs.txt (line 2))
[pipenv.exceptions.InstallError]:   Using cached funcsigs-1.0.2-py2.py3-none-any.whl (17 kB)
[pipenv.exceptions.InstallError]: Collecting llvmlite==0.37.0 (from -r /var/folders/ds/knxd_rlj057g0xx7vh_ry0q80000gn/T/pipenv-x8601bnm-requirements/pipenv-gnh_n21n-hashed-reqs.txt (line 3))
[pipenv.exceptions.InstallError]:   Using cached llvmlite-0.37.0.tar.gz (125 kB)
[pipenv.exceptions.InstallError]:   Preparing metadata (setup.py): started
[pipenv.exceptions.InstallError]:   Preparing metadata (setup.py): finished with status 'done'
[pipenv.exceptions.InstallError]: Collecting nose==1.3.7 (from -r /var/folders/ds/knxd_rlj057g0xx7vh_ry0q80000gn/T/pipenv-x8601bnm-requirements/pipenv-gnh_n21n-hashed-reqs.txt (line 4))
[pipenv.exceptions.InstallError]:   Using cached nose-1.3.7-py3-none-any.whl (154 kB)
[pipenv.exceptions.InstallError]: Collecting numba==0.54.0 (from -r /var/folders/ds/knxd_rlj057g0xx7vh_ry0q80000gn/T/pipenv-x8601bnm-requirements/pipenv-gnh_n21n-hashed-reqs.txt (line 5))
[pipenv.exceptions.InstallError]:   Using cached numba-0.54.0.tar.gz (2.2 MB)
[pipenv.exceptions.InstallError]:   Preparing metadata (setup.py): started
[pipenv.exceptions.InstallError]:   Preparing metadata (setup.py): finished with status 'done'
[pipenv.exceptions.InstallError]: Collecting numpy==1.20.3 (from -r /var/folders/ds/knxd_rlj057g0xx7vh_ry0q80000gn/T/pipenv-x8601bnm-requirements/pipenv-gnh_n21n-hashed-reqs.txt (line 6))
[pipenv.exceptions.InstallError]:   Using cached numpy-1.20.3.zip (7.8 MB)
[pipenv.exceptions.InstallError]:   Installing build dependencies: started
[pipenv.exceptions.InstallError]:   Installing build dependencies: finished with status 'done'
[pipenv.exceptions.InstallError]:   Getting requirements to build wheel: started
[pipenv.exceptions.InstallError]:   Getting requirements to build wheel: finished with status 'done'
[pipenv.exceptions.InstallError]:   Preparing metadata (pyproject.toml): started
[pipenv.exceptions.InstallError]:   Preparing metadata (pyproject.toml): finished with status 'done'
[pipenv.exceptions.InstallError]: Building wheels for collected packages: llvmlite, numba, numpy
[pipenv.exceptions.InstallError]:   Building wheel for llvmlite (setup.py): started
[pipenv.exceptions.InstallError]:   Building wheel for llvmlite (setup.py): finished with status 'error'
[pipenv.exceptions.InstallError]:   Running setup.py clean for llvmlite
[pipenv.exceptions.InstallError]:   Building wheel for numba (setup.py): started
[pipenv.exceptions.InstallError]:   Building wheel for numba (setup.py): finished with status 'error'
[pipenv.exceptions.InstallError]:   Running setup.py clean for numba
[pipenv.exceptions.InstallError]:   Building wheel for numpy (pyproject.toml): started
[pipenv.exceptions.InstallError]:   Building wheel for numpy (pyproject.toml): finished with status 'done'
[pipenv.exceptions.InstallError]:   Created wheel for numpy: filename=numpy-1.20.3-cp39-cp39-macosx_14_5_arm64.whl size=5712233 sha256=9711dde8499c1df71d31b5e73b41500b737360d546c385e9e910755da73d087b
[pipenv.exceptions.InstallError]:   Stored in directory: /Users/jnhernandez/Library/Caches/pipenv/wheels/7a/34/64/4b83a3f59d2eb2ba5757834a9c43b2e3389f3b8a309b4905ba
[pipenv.exceptions.InstallError]: Successfully built numpy
[pipenv.exceptions.InstallError]: Failed to build llvmlite numba
[pipenv.exceptions.InstallError]: error: subprocess-exited-with-error
[pipenv.exceptions.InstallError]:
[pipenv.exceptions.InstallError]:   × python setup.py bdist_wheel did not run successfully.
[pipenv.exceptions.InstallError]:   │ exit code: 1
[pipenv.exceptions.InstallError]:   ╰─> [11 lines of output]
[pipenv.exceptions.InstallError]:       running bdist_wheel
[pipenv.exceptions.InstallError]:       /Users/jnhernandez/.local/share/virtualenvs/gwlf-e-NDAS7vRc/bin/python /private/var/folders/ds/knxd_rlj057g0xx7vh_ry0q80000gn/T/pip-install-3olms5dh/llvmlite_0994eb86ede5457b9beb7a00500d97cb/ffi/build.py
[pipenv.exceptions.InstallError]:       LLVM version... Traceback (most recent call last):
[pipenv.exceptions.InstallError]:         File "/private/var/folders/ds/knxd_rlj057g0xx7vh_ry0q80000gn/T/pip-install-3olms5dh/llvmlite_0994eb86ede5457b9beb7a00500d97cb/ffi/build.py", line 220, in <module>
[pipenv.exceptions.InstallError]:           main()
[pipenv.exceptions.InstallError]:         File "/private/var/folders/ds/knxd_rlj057g0xx7vh_ry0q80000gn/T/pip-install-3olms5dh/llvmlite_0994eb86ede5457b9beb7a00500d97cb/ffi/build.py", line 214, in main
[pipenv.exceptions.InstallError]:           main_posix('osx', '.dylib')
[pipenv.exceptions.InstallError]:         File "/private/var/folders/ds/knxd_rlj057g0xx7vh_ry0q80000gn/T/pip-install-3olms5dh/llvmlite_0994eb86ede5457b9beb7a00500d97cb/ffi/build.py", line 134, in main_posix
[pipenv.exceptions.InstallError]:           raise RuntimeError(msg) from None
[pipenv.exceptions.InstallError]:       RuntimeError: Could not find a `llvm-config` binary. There are a number of reasons this could occur, please see: https://llvmlite.readthedocs.io/en/latest/admin-guide/install.html#using-pip for help.
[pipenv.exceptions.InstallError]:       error: command '/Users/jnhernandez/.local/share/virtualenvs/gwlf-e-NDAS7vRc/bin/python' failed with exit code 1
[pipenv.exceptions.InstallError]:       [end of output]
[pipenv.exceptions.InstallError]:
[pipenv.exceptions.InstallError]:   note: This error originates from a subprocess, and is likely not a problem with pip.
[pipenv.exceptions.InstallError]:   ERROR: Failed building wheel for llvmlite
[pipenv.exceptions.InstallError]:   error: subprocess-exited-with-error
[pipenv.exceptions.InstallError]:
[pipenv.exceptions.InstallError]:   × python setup.py bdist_wheel did not run successfully.
[pipenv.exceptions.InstallError]:   │ exit code: 1
[pipenv.exceptions.InstallError]:   ╰─> [8 lines of output]
[pipenv.exceptions.InstallError]:       Traceback (most recent call last):
[pipenv.exceptions.InstallError]:         File "<string>", line 2, in <module>
[pipenv.exceptions.InstallError]:         File "<pip-setuptools-caller>", line 34, in <module>
[pipenv.exceptions.InstallError]:         File "/private/var/folders/ds/knxd_rlj057g0xx7vh_ry0q80000gn/T/pip-install-3olms5dh/numba_61374ef1b6eb4af9b0e1c9d141b46007/setup.py", line 414, in <module>
[pipenv.exceptions.InstallError]:           metadata['ext_modules'] = get_ext_modules()
[pipenv.exceptions.InstallError]:         File "/private/var/folders/ds/knxd_rlj057g0xx7vh_ry0q80000gn/T/pip-install-3olms5dh/numba_61374ef1b6eb4af9b0e1c9d141b46007/setup.py", line 141, in get_ext_modules
[pipenv.exceptions.InstallError]:           import numpy.distutils.misc_util as np_misc
[pipenv.exceptions.InstallError]:       ModuleNotFoundError: No module named 'numpy'
[pipenv.exceptions.InstallError]:       [end of output]
[pipenv.exceptions.InstallError]:
[pipenv.exceptions.InstallError]:   note: This error originates from a subprocess, and is likely not a problem with pip.
[pipenv.exceptions.InstallError]:   ERROR: Failed building wheel for numba
[pipenv.exceptions.InstallError]: ERROR: Could not build wheels for llvmlite, numba, which is required to install pyproject.toml-based projects
ERROR: Couldn't install package: {}
 Package installation failed...

@rajadain
Copy link
Member Author

rajadain commented Jul 1, 2024

This is very useful, thanks! You're on an M1 Mac?

@rajadain
Copy link
Member Author

rajadain commented Jul 1, 2024

It looks like you're doing all the above on develop. Can you try doing so on this branch?

@JN-Hernandez
Copy link

JN-Hernandez commented Jul 1, 2024

Thanks so much for the reminder on switching branches! This helped me progress further but unfortunately attempts to install gwlf-e 3.2.0 from TestPyPI produces an error:

(gqlfe-en) ➜  gwlf-e git:(tt/92/deploy-via-github-actions) ✗ pip install -i https://test.pypi.org/simple/ gwlf-e==3.2.0
Looking in indexes: https://test.pypi.org/simple/
Collecting gwlf-e==3.2.0
  Using cached https://test-files.pythonhosted.org/packages/9c/4d/2eb9a9215131d32db4c7a1d66689e47d9552350d238bc7974c3360cddbc5/gwlf_e-3.2.0.tar.gz (1.3 MB)
  Installing build dependencies ... error
  ERROR: Command errored out with exit status 1:
   command: /Users/jnhernandez/.pyenv/versions/3.10.0/envs/gqlfe-en/bin/python3.10 /private/var/folders/ds/knxd_rlj057g0xx7vh_ry0q80000gn/T/pip-standalone-pip-l3q6qh39/__env_pip__.zip/pip install --ignore-installed --no-user --prefix /private/var/folders/ds/knxd_rlj057g0xx7vh_ry0q80000gn/T/pip-build-env-2nqdfvam/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://test.pypi.org/simple/ -- 'setuptools>=42' 'wheel>=0.31' 'cibuildwheel>=2.13' 'twine>=1.11' 'numba==0.55.*'
       cwd: None
  Complete output (3 lines):
  Looking in indexes: https://test.pypi.org/simple/
  ERROR: Could not find a version that satisfies the requirement setuptools>=42 (from versions: none)
  ERROR: No matching distribution found for setuptools>=42
  ----------------------------------------
WARNING: Discarding https://test-files.pythonhosted.org/packages/9c/4d/2eb9a9215131d32db4c7a1d66689e47d9552350d238bc7974c3360cddbc5/gwlf_e-3.2.0.tar.gz#sha256=047076920f2b003207221a39f3efce5bfb8e550712e3b36a5445cb2550a77f80 (from https://test.pypi.org/simple/gwlf-e/) (requires-python:>=3.7). Command errored out with exit status 1: /Users/jnhernandez/.pyenv/versions/3.10.0/envs/gqlfe-en/bin/python3.10 /private/var/folders/ds/knxd_rlj057g0xx7vh_ry0q80000gn/T/pip-standalone-pip-l3q6qh39/__env_pip__.zip/pip install --ignore-installed --no-user --prefix /private/var/folders/ds/knxd_rlj057g0xx7vh_ry0q80000gn/T/pip-build-env-2nqdfvam/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://test.pypi.org/simple/ -- 'setuptools>=42' 'wheel>=0.31' 'cibuildwheel>=2.13' 'twine>=1.11' 'numba==0.55.*' Check the logs for full command output.
ERROR: Could not find a version that satisfies the requirement gwlf-e==3.2.0 (from versions: 3.0.0b0, 3.1.0, 3.2.0)
ERROR: No matching distribution found for gwlf-e==3.2.0

I was surprised to see the error says there's no matching distribution found for gwlf-e, when I can see it's present in TestPyPI. I am on an M1 device, if that makes a difference.

@rajadain
Copy link
Member Author

rajadain commented Jul 2, 2024

So the true error for the above is this one:

ERROR: Could not find a version that satisfies the requirement setuptools>=42 (from versions: none)

which can be solved by the --no-build-isolation flag https://stackoverflow.com/a/76866374.

The package is really designed to work in a Linux environment, and the wheels added as part of this PR are for Linux x64 environments. On an ARM MacOS, we'd have to build it locally. This can be done as such:

$ cd <project_dir>
$ mkdir test && cd test/
$ python3.10 -m venv .venv
$ source .venv/bin/activate
(.venv) $ pip install -r ../requirements.txt
(.venv) $ pip install build wheel
(.venv) $ pip install -i https://test.pypi.org/simple/ --no-build-isolation gwlf-e==3.2.0

This will create a new virtual environment, install the correct dependencies from the main PyPI index (for they will not be available in TestPyPI so cannot be installed transitively), install the build dependencies, and then compile and install the final piece.

Let me know if the above works.

@JN-Hernandez
Copy link

JN-Hernandez commented Jul 2, 2024

The revised steps worked, install completed successfully:

Successfully built gwlf-e
Installing collected packages: gwlf-e
Successfully installed gwlf-e-3.2.0

Do these steps (the original and revised for ARM MacOS) need to be documented in TestPyPI?

Screenshot 2024-07-02 at 11 14 34

Base automatically changed from tt/94/python-3.10 to develop July 3, 2024 01:23
@rajadain
Copy link
Member Author

rajadain commented Jul 3, 2024

Good catch. Added instructions in 0bc505d, and made that go to PyPI in 3afb5e6. See demo here: https://test.pypi.org/project/gwlf-e/3.2.1/

image

@rajadain
Copy link
Member Author

rajadain commented Jul 3, 2024

Reverted accidental version bump in c0bb882. I had done that locally to push 3.2.1 to TestPyPI. The real version bump will happen when exercising the release workflow, after this is merged.

Copy link

@JN-Hernandez JN-Hernandez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Look good, great job! +1

With this gone, PyPI will now use README for its documentation,
which is a lot more relevant.
@rajadain rajadain force-pushed the tt/92/deploy-via-github-actions branch from c0bb882 to 9f199c0 Compare July 3, 2024 15:58
@rajadain
Copy link
Member Author

rajadain commented Jul 3, 2024

Thanks for shepherding this in! Squashed fixups, will wait for CI to pass before merging.

@rajadain rajadain merged commit 0119840 into develop Jul 3, 2024
2 checks passed
@rajadain rajadain deleted the tt/92/deploy-via-github-actions branch July 3, 2024 16:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

AWS 2-4: Add GitHub Action for Deployments
2 participants