Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI use GitHub actions #615

Merged
merged 65 commits into from
May 29, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
65 commits
Select commit Hold shift + click to select a range
74f48d1
[CI] First draft for ci building and testing library
Jan 9, 2022
faa622a
[CI] Fix upload coverage
Jan 9, 2022
76867cf
[CI] Enable temporarly on push run
Jan 9, 2022
7199c1e
[CI] Use repository name for jobs
Jan 9, 2022
a81f319
[CI] Fix python issue installation
Jan 9, 2022
57498cc
[CI] Add boost dependency
Jan 9, 2022
3717a24
[CI] Disabling support to Python3.10 because pyflagser does not suppo…
Jan 9, 2022
4faa5c0
[CI] Fix build on Windows
Jan 9, 2022
98f6a07
[CI] Try to fix boost detection
Jan 10, 2022
9457dc7
[CI] Allow continue on failure if test fails
Jan 10, 2022
8fae575
[CI] Allow error with jupyter notebooks
Jan 10, 2022
c11db46
[CI] Add coverage generation on Mac
MonkeyBreaker Jan 10, 2022
3e2d864
[CI] Delete Azure related CI
MonkeyBreaker Jan 10, 2022
1e683a4
[CI] flake exclude boost directory
MonkeyBreaker Jan 10, 2022
65fb58a
[CI] Raise CI timeout to 15 min
MonkeyBreaker Jan 10, 2022
e80002f
[CI] Update flake8 use in CI
MonkeyBreaker Jan 10, 2022
7483f41
[CI] Update flake8 use in CI
MonkeyBreaker Jan 10, 2022
c2d2817
[CI] Add debug print for flake8
MonkeyBreaker Jan 10, 2022
b9ca33d
[CI] Add hardcoded folders to exclude for dlake8
MonkeyBreaker Jan 10, 2022
73a97c5
[CI] Add hardcoded folders to exclude for dlake8
MonkeyBreaker Jan 10, 2022
ad95922
[CI] Disable on push action
MonkeyBreaker Jan 10, 2022
321639b
[CI] Add wheels generations workflow
MonkeyBreaker Jan 10, 2022
11a81a0
[CI] Add boost install dir
MonkeyBreaker Jan 10, 2022
c172485
[CI] Update boost logic
MonkeyBreaker Jan 10, 2022
271fc10
[CI] Revert previous change
MonkeyBreaker Jan 10, 2022
42da492
[PY] Add modules in externals
MonkeyBreaker Jan 11, 2022
1db74b6
[TEST] Remove all __init__.py
MonkeyBreaker Jan 11, 2022
ee35824
[TEST] Rename test with duplicates file names
MonkeyBreaker Jan 11, 2022
ffb2efd
[TEST] Update in all externals/python test to use global import
MonkeyBreaker Jan 11, 2022
cc3abdd
[EXT] Remove Gudhi RipsComplex because it is unused
MonkeyBreaker Jan 11, 2022
cb48846
Revert "[EXT] Remove Gudhi RipsComplex because it is unused"
MonkeyBreaker Jan 11, 2022
86bb7b1
[CI] Disable in the wheels to download from action boost on linux worker
MonkeyBreaker Jan 12, 2022
8ae9976
[CI] Use /host for linux docker to prevent downloading each time boost
MonkeyBreaker Jan 12, 2022
6a87008
[CI] Update boost path variables
MonkeyBreaker Jan 12, 2022
e4c6952
[CI] Update boost path variables
MonkeyBreaker Jan 12, 2022
ea9b96d
[CI] Update boost path variables
MonkeyBreaker Jan 12, 2022
98df61e
[CI] Remove boost_install_dir variable
MonkeyBreaker Jan 12, 2022
65d9e53
[CI] Update boost path variables
MonkeyBreaker Jan 12, 2022
d116e59
[CI] Update boost path variables
MonkeyBreaker Jan 12, 2022
3211138
[CI] Update boost path variables
MonkeyBreaker Jan 12, 2022
c8386f5
[CI] Partial fallback to working version
MonkeyBreaker Jan 12, 2022
e690254
[CI] Update boost path variables
MonkeyBreaker Jan 12, 2022
4054174
[CI] Update boost path variables
MonkeyBreaker Jan 12, 2022
e91f0be
[DOC] Add first draft of documentation concerning wheels generation
MonkeyBreaker Jan 13, 2022
f06398f
[CI] Fix notebook test
MonkeyBreaker Jan 13, 2022
d135df6
[CI] Enable again on push for ci
MonkeyBreaker Jan 13, 2022
be50412
[CI] Update timeout and specify shell
MonkeyBreaker Jan 13, 2022
8079510
[CI] Update timeout for papermill job
MonkeyBreaker Jan 13, 2022
2efab19
[CI] Update timeout for papermill job
MonkeyBreaker Jan 13, 2022
6de8f29
[CI] Update timeout for papermill job
MonkeyBreaker Jan 13, 2022
5613551
[CI] Update main timeout to 20 min
MonkeyBreaker Jan 13, 2022
de4aaf8
[CI] Disable on push trigger
MonkeyBreaker Jan 16, 2022
838ba27
[CI] Remove nightly build related stuff
MonkeyBreaker Jan 16, 2022
bc301f6
[CI] Add boost caching
MonkeyBreaker Jan 16, 2022
7c07c28
[CI] Add input variable to enable notebook verification
MonkeyBreaker Jan 16, 2022
787fb94
[CI] Enable again on push for CI
MonkeyBreaker Jan 16, 2022
ae44ec5
[CI] Fix wrong name of event
MonkeyBreaker Jan 16, 2022
a7731ea
[CI] Hardcode boost directory, because when cache is used, the variab…
MonkeyBreaker Jan 16, 2022
ac48e06
[CI] Add debug information
MonkeyBreaker Jan 16, 2022
1f384bb
[CI] Add debug information
MonkeyBreaker Jan 16, 2022
c11696b
[CI] Add debug information
MonkeyBreaker Jan 16, 2022
c84e419
[DOC] Add first draft of documentation related to the CI
MonkeyBreaker Jan 16, 2022
4e3a9ce
[CI] Disable on push event
MonkeyBreaker Jan 16, 2022
d836157
[CI] Remove 3.6 enable 3.10
MonkeyBreaker Aug 30, 2022
a014bc9
[CI] Remove 3.6 enable 3.10
MonkeyBreaker Aug 30, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 0 additions & 7 deletions .azure-ci/build_manylinux2014.sh

This file was deleted.

58 changes: 0 additions & 58 deletions .azure-ci/docker_scripts.sh

This file was deleted.

11 changes: 0 additions & 11 deletions .azure-ci/setup_ccache.sh

This file was deleted.

193 changes: 193 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,193 @@
name: Build package on different Os and Python versions

on :
pull_request:
workflow_dispatch:
inputs:
testNotebooks:
description: 'Enable notebook verification step'
type: boolean
required: false
default: false

jobs:

build_package:
name: Build ${{ github.event.repository.name }} on ${{ matrix.os }} for Python-${{ matrix.python-version }}
runs-on: ${{ matrix.os }}
timeout-minutes: 20
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, windows-latest, macos-latest]
python-version: [3.7, 3.8, 3.9, '3.10']
include:
- os: ubuntu-latest
path: ~/.cache/pip
papermill_timeout: 120
# ccache requires absolute path
path_ccache: /home/runner/cache/ccache
- os: macos-latest
path: ~/Library/Caches/pip
papermill_timeout: 240
# ccache requires absolute path
path_ccache: /Users/runner/Library/Caches/ccache
- os: windows-latest
path: ~\AppData\Local\pip\Cache
papermill_timeout: 300
# FIXME: ccache is currently not available on Windows
# path_ccache: ~\AppData\Local\ccache\Cache
env:
BOOST_VERSION: 1.76.0

steps:
- uses: actions/checkout@v2

- uses: actions/setup-python@v2
name: Install Python-${{ matrix.python-version }}
with:
python-version: ${{ matrix.python-version }}

###################
# Caching stuff #
###################

- name: Activating Python cache
uses: actions/cache@v2
id: cache_python
continue-on-error: true
with:
path: ${{ matrix.path }}
key: ${{ runner.os }}-pip-${{ matrix.python-version }}-${{ hashFiles('**/requirements.txt') }}

- name: Cache boost ${{env.BOOST_VERSION}}
# FIXME For an unknow reason on windows runner, when the cache is hit
# The compilation fails because it doesn't find some headers
# I don't know if the cache is corrupted, for the moment hard to debug
if: ${{ runner.os == 'Linux' || runner.os == 'macOS' }}
uses: actions/cache@v2
id: cache-boost
with:
# Set the default path as the path to cache
path: ${{ github.workspace }}/boost/boost
# Use the version as the key to only cache the correct version
key: boost-${{env.BOOST_VERSION}}

# FIXME: This is quite costly but at the moment I don't have an alternative
- name: Install ccache for Linux and Mac
if: ${{ runner.os == 'Linux' || runner.os == 'macOS' }}
run: |
if [ "$RUNNER_OS" == "Linux" ]; then
sudo apt -y install ccache
elif [ "$RUNNER_OS" == "macOS" ]; then
brew install ccache
fi
shell: bash

# Windows does not support ccache with visual studio compiler
- name: ccache cache files
if: ${{ runner.os != 'Windows' }}
uses: actions/cache@v2
id: cache_ccache
continue-on-error: true
with:
path: ${{ matrix.path_ccache }}
# TODO: When updating c++ backend, use new date (yy-mm-dd)
key: ${{ runner.os }}-ccache-${{ matrix.python-version }}-22-01-11

- name: Upgrade pip and setuptools
run: |
python -m pip install --upgrade pip setuptools

- name: Install boost ${{env.BOOST_VERSION}}
uses: MarkusJx/install-boost@v2.1.0
id: install-boost
if: steps.cache-boost.outputs.cache-hit != 'true'
with:
boost_version: ${{env.BOOST_VERSION}}


######################
# Building package #
######################

- name: Build ${{ github.event.repository.name }} on Linux and MacOs
if: ${{ runner.os != 'Windows' }}
run: |
echo $BOOST_ROOT
python -m pip install -e ".[dev]"
env:
CC: "ccache gcc"
CXX: "ccache g++"
CCACHE_DIR: ${{ matrix.path_ccache }}
CCACHE_COMPRESS: 1
CCACHE_MAXSIZE: 4G
BOOST_ROOT: ${{ github.workspace }}/boost/boost

# Windows does not support ccache with visual studio compiler
- name: Build ${{ github.event.repository.name }} on Windows
if: ${{ runner.os == 'Windows' }}
run: |
python -m pip install -e ".[dev]"
env:
BOOST_ROOT: ${{ steps.install-boost.outputs.BOOST_ROOT }}

- name: Install test tools
run: |
python -m pip install pandas pytest pytest-cov pytest-benchmark hypothesis


#############
# Testing #
#############

- name: Test package with pytest on Linux and Windows
if: ${{ runner.os == 'Linux' || runner.os == 'Windows' }}
run: |
pytest --pyargs gtda --no-cov

- name: Test package with pytest Mac with coverage
if: ${{ runner.os == 'macOS' }}
run: |
pytest gtda --cov --cov-report xml:coverage.xml

# FIXME: It does not use config.cfg, I don't know why ...
- name: flake8 Lint on MacOS
if: ${{ runner.os == 'macOS' }}
uses: py-actions/flake8@v2
with:
exclude: "boost,gtda/externals,doc,examples"

- name: Test jupyter notebooks with papermill
if: ${{ github.event.inputs.testNotebooks == true }}
run: |
python -m pip install openml matplotlib
python -m pip install papermill
cd examples
# Get all .ipynb notebook but discard the MNIST_classification
# Because it is too long
# This command also removes the .pynb extension
FILES_WITHOUT_EXT=`find . ! -name '*MNIST_classification*' -iname '*.ipynb' -exec bash -c 'printf "%s\n" "${@%.*}"' _ {} +`
PAPERMILL_DIR=papermill_results

mkdir -p $PAPERMILL_DIR

for FILENAME in $FILES_WITHOUT_EXT
do
SAVE_FILE=$PAPERMILL_DIR/$FILENAME.txt
papermill --execution-timeout ${{ matrix.papermill_timeout }} $FILENAME.ipynb $SAVE_FILE
done
shell: bash

######################
# Upload Artifacts #
######################

- name: Upload coverage for Mac
if: ${{ runner.os == 'macOS' }}
uses: actions/upload-artifact@v2
with:
name: ${{ runner.os }}-coverage
path: coverage.xml
if-no-files-found: warn

Loading