Debugging wheel‐build issues

Problem

• https://github.com/single-cell-data/TileDB-SOMA/blob/main/.github/workflows/python-packaging.yml uses pypa/cibuildwheel which invokes various commands
• These are layered within various docker run
• The pypy/cibuildwheel YAML is opaque and does not say everything it's doing
• This means that if you want to go into a failed wheel-build CI run, click through to the logs, and copy/paste commands to try to repro a CI wheel-build failure, you simply cannot

Running wheel-build CI at will

• Our "sdist & wheels" CI only auto-runs on releases and on nightly crons. (Please also see our established procedure.)
• In this repo to go Actions
• Then on the left pick the "sdist & wheels" action after "show more workflows"
• At the right "Run workflow", then your branch

Solution

The below is a known-good method to reproduce wheel-build issues outside of CI. They are a truthful record of commands that have been invoked.

• Use Linux. It has been found that if you are on a MacOS machine -- version 14 being current -- and do python setup.py bdist_wheel you'll get a wheel build for MacOS 11, which you will of course be promptly unable to install.

• Use a clean docker run environment

$ alias dreb
alias dreb='docker run --rm -it --entrypoint=/bin/bash'

$ dreb python:3.10
root@1892dc61dd4d:/#

• Build a source dist, then extract it to somewhere else, then install from there. This step is crucial as it verifies that the install will be seeing only files listed in MANIFEST.in, not everything in your local checkout (which is the way the CI YAML works)

cd /tmp/

git clone https://github.com/single-cell-data/TileDB-SOMA

cd TileDB-SOMA/apis/python/

tag=1.9.0 # or whatever tag

git checkout $tag

python setup.py sdist

ls -lrt dist/

mkdir /tmp/install

cp dist/tiledbsoma-${tag}.tar.gz /tmp/install

cd /tmp/install

tar zxf tiledbsoma-${tag}.tar.gz

cd tiledbsoma-${tag}

apt-get update

apt install -y cmake zip

pip install pybind11

python setup.py bdist_wheel

ls -l dist/tiledbsoma-*.whl

pip install dist/tiledbsoma-*.whl

cd /tmp/TileDB-SOMA/apis/python

python -m pytest tests/