Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build windows with conan support #9

Merged
merged 15 commits into from
Jan 4, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
61 changes: 48 additions & 13 deletions .github/workflows/wheels.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,26 +2,34 @@ name: Wheels

on:
push:
branches:
- 'main'
jonathf marked this conversation as resolved.
Show resolved Hide resolved
tags:
- '*'
- '*'
paths-ignore:
- '*.md'
pull_request:
branches:
- 'main'
paths-ignore:
- '*.md'

jobs:
build_sdist:
name: Build SDist
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2

- name: Build SDist
run: pipx run build --sdist
- uses: actions/checkout@v2

- name: Check metadata
run: pipx run twine check dist/*
- name: Build SDist
run: pipx run build --sdist

- uses: actions/upload-artifact@v2
with:
path: dist/*.tar.gz
- name: Check metadata
run: pipx run twine check dist/*

- uses: actions/upload-artifact@v2
with:
path: dist/*.tar.gz
nilsnolde marked this conversation as resolved.
Show resolved Hide resolved

build_wheels:
name: Wheels on ${{ matrix.os }}
Expand All @@ -32,16 +40,43 @@ jobs:
os:
- ubuntu-latest
- macos-latest
# - windows-latest
- windows-latest

steps:
- uses: actions/checkout@v2
with:
submodules: recursive

- name: Cache Conan
id: cache-conan
uses: actions/cache@v2
if: matrix.os == 'windows-latest'
with:
path: |
conan_build
conan_data
key: conan-${{ matrix.os }}-${{ hashFiles('conanfile.txt') }}

- name: Set up Windows Python ${{ matrix.python_version }}
uses: actions/setup-python@v2
if: matrix.os == 'windows-latest' && steps.cache-conan.outputs.cache-hit != 'true'
with:
python-version: '3.x'

- name: Install conan for Windows
if: matrix.os == 'windows-latest' && steps.cache-conan.outputs.cache-hit != 'true'
run: |
pip install pip --upgrade
pip install conan
conan profile new default --detect
conan profile update "settings.compiler=Visual Studio" default
conan profile update "settings.compiler.version=16" default
conan config set "storage.path=$env:GITHUB_WORKSPACE/conan_data"
conan install --build=openssl --install-folder conan_build .
nilsnolde marked this conversation as resolved.
Show resolved Hide resolved

- uses: pypa/cibuildwheel@v2.3.1
env:
MACOSX_DEPLOYMENT_TARGET: 10.15
MACOSX_DEPLOYMENT_TARGET: 10.14
nilsnolde marked this conversation as resolved.
Show resolved Hide resolved

- name: Verify clean directory
run: git diff --exit-code
Expand All @@ -52,9 +87,9 @@ jobs:
with:
path: wheelhouse/*.whl


upload_all:
name: Upload if release
if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags')
needs: [build_wheels, build_sdist]
runs-on: ubuntu-latest

Expand Down
5 changes: 4 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ __pycache__/

# Distribution / packaging
.Python
build/
*build/
nilsnolde marked this conversation as resolved.
Show resolved Hide resolved
develop-eggs/
dist/
downloads/
Expand Down Expand Up @@ -117,6 +117,9 @@ venv.bak/
# Rope project settings
.ropeproject

# pycharm project settings
.idea

# mkdocs documentation
/site

Expand Down
13 changes: 12 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,14 +19,25 @@ VROOM can also solve any mix of the above problem types.

## Installation

Pyvroom currently makes binaries for on MacOS and Linux.
Pyvroom currently makes binaries for on MacOS and Linux (Windows is WIP).

Installation should be as simple as:

```bash
pip install pyvroom
```

## Building

Building the source distributions on another OS requires:
- the `./build-requirements.txt` Python dependencies
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah, didn't include that file yet, but would be helpful no?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added it

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still think it can't hurt to have it here for the old people like me who are still used to python setup.py xx;) happy to remove it as well though

- `asio` headers installed
- `openssl` & `crypto` libraries & headers installed

Optionally the C++ dependencies can be installed with [`conan`](https://github.com/conan-io/conan):
```shell script
conan install --build=openssl --install-folder conan_build .
```
## Basic usage

```python
Expand Down
5 changes: 5 additions & 0 deletions build-requirements.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
setuptools>=45
wheel
setuptools_scm>=6.2
setuptools_scm_git_archive
pybind11>=2.8.0
6 changes: 6 additions & 0 deletions conanfile.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
[requires]
openssl/1.1.1m
nilsnolde marked this conversation as resolved.
Show resolved Hide resolved
asio/1.21.0

[generators]
json
Comment on lines +5 to +6
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this will instruct conan to write the build information of the 2 libraries to a json which we can then parse in the setup.py

5 changes: 0 additions & 5 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -46,8 +46,3 @@ apk add openssl-dev
before-all = """
brew install asio
"""

[tool.cibuildwheel.windows]
before-all = """
choco install -y openssl
"""
49 changes: 39 additions & 10 deletions setup.py
Original file line number Diff line number Diff line change
@@ -1,34 +1,63 @@
import json
import logging
import os
import platform
from pathlib import Path
from setuptools import setup
from pybind11.setup_helpers import Pybind11Extension, build_ext

extra_compile_args = [
"-MMD",
"-MP",
"-Wextra",
"-Wpedantic",
"-Wall",
"-O3",
"-DASIO_STANDALONE",
"-DNDEBUG",
]
extra_link_args = [
"-lpthread",
"-lssl",
"-lcrypto",
]
extra_compile_args = [
"-MMD",
"-MP",
"-Wextra",
"-Wpedantic",
"-Wall",
"-O3",
"-DASIO_STANDALONE",
"-DNDEBUG",
]
include_dirs = [os.path.join("vroom", "src")]
libraries = []
library_dirs = []

# try conan dependency resolution
conanfile = tuple(Path(__file__).parent.resolve().rglob('conanbuildinfo.json'))
if conanfile:
logging.info("Using conan to resolve dependencies.")
with conanfile[0].open() as f:
conan_deps = json.load(f)['dependencies']
for dep in conan_deps:
include_dirs.extend(dep['include_paths'])
libraries.extend(dep['libs'])
libraries.extend(dep['system_libs'])
library_dirs.extend(dep['lib_paths'])
else:
logging.warning('Conan not installed and/or no conan build detected. Assuming dependencies are installed.')

if platform.system() == "Darwin":
# Homebrew puts include folders in weird places.
include_dirs.append("/usr/local/opt/openssl@1.1/include")
extra_link_args.insert(0, "-L/usr/local/opt/openssl@1.1/lib")
Comment on lines 43 to 45
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actually, using conan on mac for ci would remove these lines as well. which are not very robust I think. brew is pretty updated always I found, some users might actually have 3.x soon? anyways, more of a works-now-why-change thing. but maybe good to keep in mind in worst case

elif platform.system() == "Windows":
extra_compile_args = [
"-DNOGDI",
"-DNOMINMAX",
"-DWIN32_LEAN_AND_MEAN",
"-DASIO_STANDALONE"
nilsnolde marked this conversation as resolved.
Show resolved Hide resolved
]
extra_link_args = []
nilsnolde marked this conversation as resolved.
Show resolved Hide resolved

ext_modules = [
Pybind11Extension(
"_vroom",
[os.path.join("src", "_vroom.cpp")],
library_dirs=library_dirs,
libraries=libraries,
extra_compile_args=extra_compile_args,
extra_link_args=extra_link_args,
),
Expand Down