Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create an openblas wheel #87

Merged
merged 23 commits into from
Sep 19, 2023
Merged
Show file tree
Hide file tree
Changes from 18 commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
87 changes: 0 additions & 87 deletions .github/workflows/build.yml

This file was deleted.

46 changes: 26 additions & 20 deletions .github/workflows/multibuild.yml → .github/workflows/posix.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: multibuild
name: posix

on:
push:
Expand All @@ -19,38 +19,29 @@ jobs:
fail-fast: false
matrix:
os: [ubuntu-latest, macos-11]
platform: [x64]
PLAT: [i686, x86_64]
INTERFACE64: ['0', '1']
MB_ML_VER: ['', 2010, 2014]
MB_ML_VER: ['2014']
include:
- os: macos-11
PLAT: arm64
INTERFACE64: '1'
platform: [x64]
- os: macos-11
PLAT: arm64
INTERFACE64: '0'
platform: [x64]
- os: ubuntu-latest
PLAT: x86_64
INTERFACE64: '1'
MB_ML_LIBC: musllinux
MB_ML_VER: _1_1
platform: [x64]
- os: ubuntu-latest
PLAT: x86_64
INTERFACE64: '0'
MB_ML_LIBC: musllinux
MB_ML_VER: _1_1
platform: [x64]
exclude:
- os: macos-11
PLAT: i686
- os: macos-11
MB_ML_VER: 2010
- os: macos-11
MB_ML_VER: 2014
- PLAT: i686
os: macos-11
- PLAT: i686
INTERFACE64: '1'
env:
Expand All @@ -71,14 +62,13 @@ jobs:
with:
submodules: recursive
fetch-depth: 0
- name: Set up Python 3.8
uses: actions/setup-python@v2
- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: 3.8
python-version: 3.7
- name: Set extra env
run: |
if [ "macos-11" == "${{ matrix.os }}" ]; then
ls /Library/Developer/CommandLineTools/SDKs
echo "TRAVIS_OS_NAME=osx" >> $GITHUB_ENV;
echo "LDFLAGS=-L/Library/Developer/CommandLineTools/SDKs/MacOSX12.1.sdk/usr/lib" >> $GITHUB_ENV;
echo "LIBRARY_PATH=-L/Library/Developer/CommandLineTools/SDKs/MacOSX12.1.sdk/usr/lib" >> $GITHUB_ENV;
Expand All @@ -93,9 +83,9 @@ jobs:
echo "DOCKER_TEST_IMAGE: ${DOCKER_TEST_IMAGE}"
- name: Install VirtualEnv
run: |
python -m pip install --upgrade pip
python3 -m pip install --upgrade pip
pip install virtualenv
- name: Build and Install Wheels
- name: Build OpenBLAS
run: |
set -xeo pipefail
if [[ "$PLAT" == "arm64" ]]; then
Expand All @@ -118,11 +108,27 @@ jobs:
build_lib "$PLAT" "$INTERFACE64" "0"
fi

- name: Build and test wheel
run: |
if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then
source travis-ci/build_wheel.sh
else
libc=${MB_ML_LIBC:-manylinux}
docker_image=quay.io/pypa/${libc}${MB_ML_VER}_${PLAT}
docker run --rm -e INTERFACE64="${INTERFACE64}" -v $(pwd):/openblas $docker_image /bin/bash -xe /openblas/travis-ci/build_wheel.sh
fi

- uses: actions/upload-artifact@v3
with:
name: openblas
path: libs/openblas*.tar.gz

- name: Upload tarballs
- uses: actions/upload-artifact@v3
with:
name: wheels
path: dist/scipy_openblas*.whl

- name: Upload
run: |
set -ex
TOKEN=${{ secrets.ANACONDA_SCIENTIFIC_PYTHON_UPLOAD }}
Expand Down
158 changes: 158 additions & 0 deletions .github/workflows/windows.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,158 @@
name: Win

on:
push:
branches: [ main ]
pull_request:
branches: [ main ]

env:
OPENBLAS_COMMIT: "c2f4bdb"
OPENBLAS_ROOT: "c:\\opt"
# Preserve working directory for calls into bash
# Without this, invoking bash will cd to the home directory
CHERE_INVOKING: "yes"
BASH_PATH: "c:\\rtools40\\usr\\bin\\bash.exe"

jobs:
build:
strategy:
matrix:
plat: ['x64', 'x86']
INTERFACE64: ['1', '0']
os: [windows-latest]
exclude:
- plat: x86
INTERFACE64: '1'
fail-fast: false
runs-on: ${{ matrix.os }}

steps:
- uses: actions/checkout@v3
- name: install-rtools
run: |
# rtools 42+ does not support 32 bits builds.
choco install -y rtools --noprogress --force --version=4.0.0.20220206

- name: Set env variables
run: |
echo "START_DIR=$PWD" >> $env:GITHUB_ENV
# For interpretation of MSYSTEM, see:
# https://sourceforge.net/p/msys2/discussion/general/thread/b7dfdac8/#3939
if ( "${{ matrix.plat }}" -eq "x86") {
echo "PLAT=i686" >> $env:GITHUB_ENV
echo "WHEEL_PLAT=win32" >> $env:GITHUB_ENV
echo "MSYSTEM=MINGW32" >> $env:GITHUB_ENV
echo "LDFLAGS=-static -static-libgcc" >> $env:GITHUB_ENV
echo "BUILD_BITS=32" >> $env:GITHUB_ENV
} else {
echo "PLAT=x86_64" >> $env:GITHUB_ENV
echo "WHEEL_PLAT=x86_64" >> $env:GITHUB_ENV
echo "MSYSTEM=UCRT64" >> $env:GITHUB_ENV
echo "LDFLAGS=-lucrt -static -static-libgcc" >> $env:GITHUB_ENV
echo "BUILD_BITS=64" >> $env:GITHUB_ENV
}
if ( ${{ matrix.INTERFACE64 }} -eq "1" ) {
echo "INTERFACE64=1" >> $env:GITHUB_ENV
}

- name: Build
run: |
git submodule update --init --recursive
& $env:BASH_PATH -lc tools/build_openblas.sh

- name: Test
run: |
& $env:BASH_PATH -lc tools/build_gfortran.sh
echo "Static test"
.\for_test\test.exe
echo "Dynamic test"
.\for_test\test_dyn.exe

- name: Copy
run: |
cp for_test\test*.exe builds

- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: 3.7
architecture: ${{ matrix.plat }}


- name: Build wheel
shell: bash
run: |
set -xeo pipefail
python -m pip install wheel
# This will fail if there is more than one file in libs
unzip -d local/scipy_openblas64 builds/openblas*.zip
if [[ -d local/scipy_openblas64/64 ]]; then
mv local/scipy_openblas64/64/* local/scipy_openblas64
else
mv local/scipy_openblas64/32/* local/scipy_openblas64
fi
mv local/scipy_openblas64/bin/*.dll local/scipy_openblas64/lib
rm local/scipy_openblas64/lib/*.a
rm local/scipy_openblas64/lib/*.exp
rm local/scipy_openblas64/lib/*.def
if [[ -d local/scipy_openblas64/64 ]]; then
rm -rf local/scipy_openblas64/64
else
rm -rf local/scipy_openblas64/32
fi
if [[ "${INTERFACE64}" != "1" ]]; then
mv local/scipy_openblas64 local/scipy_openblas32
# rewrite the name of the project to scipy_openblas32
# this is a hack, but apparently there is no other way to change the name
# of a pyproject.toml project
sed -e "s/openblas64/openblas32/" -i pyproject.toml
sed -e "s/openblas_get_config64_/openblas_get_config/" -i local/scipy_openblas32/__init__.py
sed -e "s/openblas64/openblas32/" -i local/scipy_openblas32/__main__.py
fi

python -m pip wheel -w dist -vv .
# move the mis-named scipy_openblas64-none-any.whl to a platform-specific name
for f in dist/*.whl; do mv $f "${f/%any.whl/$WHEEL_PLAT.whl}"; done

- name: Set up different Python
uses: actions/setup-python@v4
with:
python-version: 3.11
architecture: ${{ matrix.plat }}

- uses: actions/upload-artifact@v3
with:
name: openblas
path: builds/openblas*.zip

- uses: actions/upload-artifact@v3
with:
name: wheels
path: dist/scipy_openblas*.whl

- name: Test 64-bit interface wheel
if: matrix.INTERFACE64 == '1'
shell: bash
run: |
python -m pip install --no-index --find-links dist scipy_openblas64
python -m scipy_openblas64

- name: Test 32-bit interface wheel
if: matrix.INTERFACE64 != '1'
shell: bash
run: |
python -m pip install --no-index --find-links dist scipy_openblas32
python -m scipy_openblas32

- uses: conda-incubator/setup-miniconda@v2
with:
activate-environment: upload

- name: Upload
env:
ANACONDA_SCIENTIFIC_PYTHON_UPLOAD: ${{ secrets.ANACONDA_SCIENTIFIC_PYTHON_UPLOAD }}
run: |
# Pin urllib3<2 due to github.com/Anaconda-Platform/anaconda-client/issues/654
conda install "urllib3<2" anaconda-client
& $env:BASH_PATH -lc tools/upload_to_anaconda_staging.sh
13 changes: 13 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -4,3 +4,16 @@ builds/
*.pyc
*.swp
for_test/
test.exe
dist/
build/
libs/
# These are artifacts when using editable builds
local/lib
local/include
local/scipy_openblas64.egg-info
local/scipy_openblas64/lib
local/scipy_openblas64/include
local/scipy_openblas64/*.so
local/scipy_openblas64/*.pyd
local/scipy_openblas64/*.dylib
Loading
Loading