Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Convert main.yml CI into callable workflows #3529

Merged
merged 39 commits into from
Sep 18, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
33cbb0b
Change main CI workflow to use callable workflows
byrnHDF Sep 11, 2023
0ab8de9
Fix special autotools
byrnHDF Sep 11, 2023
96ae0ed
Change job names
byrnHDF Sep 11, 2023
c8b6eae
Correct name syntax
byrnHDF Sep 11, 2023
d698b5a
Fix job syntax
byrnHDF Sep 11, 2023
3fd150a
correct job name
byrnHDF Sep 11, 2023
9efc068
Correct syntax
byrnHDF Sep 11, 2023
e04810e
fix with for uses syntax
byrnHDF Sep 11, 2023
6e6364a
fix matrix syntax
byrnHDF Sep 11, 2023
d0de4d8
Try if syntax
byrnHDF Sep 11, 2023
21d5d62
Split out parallel special workflows
byrnHDF Sep 11, 2023
bd3fb38
Add parallel workflow
byrnHDF Sep 11, 2023
ed0e07f
Fix parallel and add intel
byrnHDF Sep 11, 2023
0cc8e53
Correct configure command syntax and name
byrnHDF Sep 11, 2023
c689f0b
Add ifortvars.sh for fortran
byrnHDF Sep 11, 2023
b367635
Remove unknown file
byrnHDF Sep 11, 2023
efddcc3
disable Intel fortran tests until path fixed
byrnHDF Sep 11, 2023
0e2a79f
Add other option combos
byrnHDF Sep 12, 2023
5f8ef1d
Rework workflow job names
byrnHDF Sep 12, 2023
b366498
Group jobs
byrnHDF Sep 12, 2023
9441cab
Add job name
byrnHDF Sep 12, 2023
16cebcc
jobs syntax change
byrnHDF Sep 12, 2023
abd06a6
Add steps syntax
byrnHDF Sep 12, 2023
a0f19d6
Add dash to steps syntax
byrnHDF Sep 12, 2023
14820e9
add runs on item
byrnHDF Sep 12, 2023
6865223
add missing dash
byrnHDF Sep 12, 2023
b13c8bb
split out autools/cmake to callable workflows
byrnHDF Sep 12, 2023
8e865ee
rework job syntax
byrnHDF Sep 12, 2023
2efabed
remove name lines
byrnHDF Sep 12, 2023
db5144a
remove incorrect line
byrnHDF Sep 12, 2023
a1de1fc
match names
byrnHDF Sep 12, 2023
4d5628b
update checkout action version
byrnHDF Sep 12, 2023
4a62a20
Add job names
byrnHDF Sep 12, 2023
8254347
Add build mode back to name in autotools
byrnHDF Sep 12, 2023
8d2512f
Split autotools into two jobs
byrnHDF Sep 12, 2023
8f48239
fix input type
byrnHDF Sep 12, 2023
bed7892
Remove workdir in configure
byrnHDF Sep 12, 2023
3aaa182
Correct variable syntax
byrnHDF Sep 12, 2023
42fcb39
add workdir
byrnHDF Sep 12, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
54 changes: 54 additions & 0 deletions .github/workflows/autotools.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
name: hdf5 dev autools CI

# Controls when the action will run. Triggers the workflow on push or pull request
on:
workflow_call:

permissions:
contents: read

# A workflow run is made up of one or more jobs that can run sequentially or
# in parallel. We just have one job, but the matrix items defined below will
# run in parallel.
jobs:
call-workflow-special-autotools:
name: "Autotools Special Workflows"
uses: ./.github/workflows/main-auto-spc.yml

call-parallel-special-autotools:
name: "Autotools Parallel Special Workflows"
uses: ./.github/workflows/main-auto-par.yml

call-debug-thread-autotools:
name: "Autotools Debug Thread-Safety Workflows"
uses: ./.github/workflows/main-auto.yml
with:
thread_safety: enable
build_mode: "debug"

call-release-thread-autotools:
name: "Autotools Release Thread-Safety Workflows"
uses: ./.github/workflows/main-auto.yml
with:
thread_safety: enable
build_mode: "production"

call-debug-autotools:
name: "Autotools Debug Workflows"
uses: ./.github/workflows/main-auto.yml
with:
thread_safety: disable
build_mode: "debug"

call-release-autotools:
name: "Autotools Release Workflows"
uses: ./.github/workflows/main-auto.yml
with:
thread_safety: disable
build_mode: "production"

call-release-auto-intel:
name: "Autotools Intel Workflows"
uses: ./.github/workflows/intel-auto.yml
with:
build_mode: "production"
2 changes: 1 addition & 1 deletion .github/workflows/clang-format-check.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ jobs:
runs-on: ubuntu-latest
if: "!contains(github.event.head_commit.message, 'skip-ci')"
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- name: Run clang-format style check for C and Java code
uses: DoozyX/clang-format-lint-action@v0.13
with:
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/clang-format-fix.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ jobs:
permissions:
contents: write # In order to allow EndBug/add-and-commit to commit changes
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- name: Fix C and Java formatting issues detected by clang-format
uses: DoozyX/clang-format-lint-action@v0.13
with:
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/cmake-ctest.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
name: hdf5 dev ctest runs

# Controls when the action will run. Triggers the workflow on a schedule
# Controls when the action will run. Triggers the workflow on a call
on:
workflow_call:
inputs:
Expand Down
46 changes: 46 additions & 0 deletions .github/workflows/cmake.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
name: hdf5 dev cmake CI

# Controls when the action will run. Triggers the workflow on push or pull request
on:
workflow_call:

permissions:
contents: read

# A workflow run is made up of one or more jobs that can run sequentially or
# in parallel. We just have one job, but the matrix items defined below will
# run in parallel.
jobs:
call-debug-thread-cmake:
name: "CMake Debug Thread-Safety Workflows"
uses: ./.github/workflows/main-cmake.yml
with:
thread_safety: true
build_mode: "Debug"

call-release-thread-cmake:
name: "CMake Release Thread-Safety Workflows"
uses: ./.github/workflows/main-cmake.yml
with:
thread_safety: true
build_mode: "Release"

call-debug-cmake:
name: "CMake Debug Workflows"
uses: ./.github/workflows/main-cmake.yml
with:
thread_safety: false
build_mode: "Debug"

call-release-cmake:
name: "CMake Release Workflows"
uses: ./.github/workflows/main-cmake.yml
with:
thread_safety: false
build_mode: "Release"

call-release-cmake-intel:
name: "CMake Intel Workflows"
uses: ./.github/workflows/intel-cmake.yml
with:
build_mode: "Release"
2 changes: 1 addition & 1 deletion .github/workflows/codespell.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ jobs:
name: Check for spelling errors
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- uses: codespell-project/actions-codespell@master
with:
skip: ./.github/workflows/codespell.yml,./bin/trace,./hl/tools/h5watch/h5watch.c,./tools/test/h5jam/tellub.c,./config/sanitizer/LICENSE,./config/sanitizer/sanitizers.cmake,./tools/test/h5repack/testfiles/*.dat,./test/API/driver,./configure,./bin/ltmain.sh,./bin/depcomp,./bin/config.guess,./bin/config.sub,./autom4te.cache,./m4/libtool.m4,./c++/src/*.html
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/cve.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ jobs:
name: CVE regression
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4

- name: Install Autotools Dependencies (Linux)
run: |
Expand All @@ -40,7 +40,7 @@ jobs:
make
sudo make install
- name: Checkout CVE test repository
uses: actions/checkout@v3
uses: actions/checkout@v4
with:
repository: HDFGroup/cve_hdf5
path: cve_hdf5
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/h5py.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ jobs:
sudo apt-get update
sudo apt-get install -y gfortran-12
- name: Checkout Spack
uses: actions/checkout@v3
uses: actions/checkout@v4
with:
repository: spack/spack
path: ./spack
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/hdfeos5.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ jobs:
name: Build hdfeos5
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4

- name: Install Autotools Dependencies (Linux)
run: |
Expand Down
97 changes: 97 additions & 0 deletions .github/workflows/intel-auto.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
name: hdf5 dev autotools icx CI

on:
workflow_call:
inputs:
build_mode:
description: "release vs. debug build"
required: true
type: string

permissions:
contents: read

jobs:
Intel_build_and_test:
name: "Intel ${{ inputs.build_mode }} -Werror (build only)"
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Install Dependencies
run: |
sudo apt-get update
sudo apt-get install autoconf automake libtool libtool-bin libaec-dev
- name: Add oneAPI to apt
shell: bash
run: |
cd /tmp
wget https://apt.repos.intel.com/intel-gpg-keys/GPG-PUB-KEY-INTEL-SW-PRODUCTS.PUB
sudo apt-key add GPG-PUB-KEY-INTEL-SW-PRODUCTS.PUB
rm GPG-PUB-KEY-INTEL-SW-PRODUCTS.PUB
sudo add-apt-repository -y "deb https://apt.repos.intel.com/oneapi all main"

- name: Install oneAPI dpcpp and fortran compiler
shell: bash
run: |
sudo apt update
sudo apt install -y intel-oneapi-compiler-dpcpp-cpp
sudo apt install -y intel-oneapi-compiler-fortran
sudo apt install -y intel-oneapi-mpi-devel
sudo apt-get install doxygen graphviz
sudo apt install libssl3 libssl-dev libcurl4 libcurl4-openssl-dev
echo "CC=icx" >> $GITHUB_ENV
echo "CXX=icpx" >> $GITHUB_ENV
echo "FC=ifx" >> $GITHUB_ENV

- name: Install oneAPI MKL library
shell: bash
run: |
sudo apt install -y intel-oneapi-mkl-devel

- name: Autotools Configure
shell: bash
run: |
source /opt/intel/oneapi/setvars.sh
export PATH=$PATH:/opt/intel/oneapi/compiler/2023.2.1/linux/bin
sh ./autogen.sh
mkdir "${{ runner.workspace }}/build"
cd "${{ runner.workspace }}/build"
$GITHUB_WORKSPACE/configure \
--enable-build-mode=${{ inputs.build_mode }} \
--enable-shared \
--disable-fortran

# BUILD
- name: Autotools Build
run: |
source /opt/intel/oneapi/setvars.sh
export PATH=$PATH:/opt/intel/oneapi/compiler/2023.2.1/linux/bin
make -j3
working-directory: ${{ runner.workspace }}/build

# RUN TESTS
# NORMAL
- name: Autotools Run Tests
run: |
source /opt/intel/oneapi/setvars.sh
export PATH=$PATH:/opt/intel/oneapi/compiler/2023.2.1/linux/bin
export SYCL_DEVICE_FILTER=opencl.cpu
make check -j2
working-directory: ${{ runner.workspace }}/build

# INSTALL (note that this runs even when we don't run the tests)
- name: Autotools Install
run: |
source /opt/intel/oneapi/setvars.sh
export PATH=$PATH:/opt/intel/oneapi/compiler/2023.2.1/linux/bin
export SYCL_DEVICE_FILTER=opencl.cpu
make install
working-directory: ${{ runner.workspace }}/build

- name: Autotools Verify Install
run: |
source /opt/intel/oneapi/setvars.sh
export PATH=$PATH:/opt/intel/oneapi/compiler/2023.2.1/linux/bin
export SYCL_DEVICE_FILTER=opencl.cpu
make check-install
working-directory: ${{ runner.workspace }}/build
83 changes: 83 additions & 0 deletions .github/workflows/intel-cmake.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
name: hdf5 dev CMake icx CI

# Controls when the action will run. Triggers the workflow on a call
on:
workflow_call:
inputs:
build_mode:
description: "release vs. debug build"
required: true
type: string

permissions:
contents: read

jobs:
Intel_build_and_test:
name: "Intel ${{ inputs.build_mode }} -Werror (build only)"
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- name: add oneAPI to apt
shell: bash
run: |
cd /tmp
wget https://apt.repos.intel.com/intel-gpg-keys/GPG-PUB-KEY-INTEL-SW-PRODUCTS.PUB
sudo apt-key add GPG-PUB-KEY-INTEL-SW-PRODUCTS.PUB
rm GPG-PUB-KEY-INTEL-SW-PRODUCTS.PUB
sudo add-apt-repository -y "deb https://apt.repos.intel.com/oneapi all main"

- name: Install Linux Dependencies
run: |
sudo apt update
sudo apt-get install ninja-build doxygen graphviz
sudo apt install libssl3 libssl-dev libcurl4 libcurl4-openssl-dev

- name: install oneAPI dpcpp and fortran compiler
shell: bash
run: |
sudo apt update
sudo apt install -y intel-oneapi-compiler-dpcpp-cpp
sudo apt install -y intel-oneapi-compiler-fortran

- name: install oneAPI MKL library
shell: bash
run: |
sudo apt install -y intel-oneapi-mkl-devel

- name: CMake Configure with icx
shell: bash
run: |
source /opt/intel/oneapi/setvars.sh
export PATH=$PATH:/opt/intel/oneapi/compiler/2023.2.1/linux/bin
mkdir "${{ runner.workspace }}/build"
cd "${{ runner.workspace }}/build"
cmake -C $GITHUB_WORKSPACE/config/cmake/cacheinit.cmake -G Ninja \
-DCMAKE_BUILD_TYPE=${{ inputs.build_mode }} \
-DCMAKE_TOOLCHAIN_FILE=config/toolchain/intel.cmake \
-DMKL_ROOT="/opt/intel/oneapi/mkl/latest" \
-DTBB_ROOT="/opt/intel/oneapi/tbb/latest" \
-DBUILD_SZIP_WITH_FETCHCONTENT=ON \
-DLIBAEC_USE_LOCALCONTENT=OFF \
-DBUILD_ZLIB_WITH_FETCHCONTENT=ON \
-DZLIB_USE_LOCALCONTENT=OFF \
-DHDF5_BUILD_FORTRAN=OFF \
$GITHUB_WORKSPACE

# BUILD
- name: CMake Build
shell: bash
run: |
source /opt/intel/oneapi/setvars.sh
cmake --build . --parallel 3 --config ${{ inputs.build_mode }}
working-directory: ${{ runner.workspace }}/build

# RUN TESTS
- name: CMake Run Tests
shell: bash
run: |
source /opt/intel/oneapi/setvars.sh
export SYCL_DEVICE_FILTER=opencl.cpu
ctest . --parallel 2 -C ${{ inputs.build_mode }} -V
working-directory: ${{ runner.workspace }}/build
Loading