Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

More cleanups after new build system adoption #933

Merged
merged 24 commits into from
Nov 11, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
60f991d
clean more zsh
jaromil Sep 24, 2024
2d657f0
chore: remove old tests moved to bats
matteo-cristino Sep 24, 2024
74ef0d9
release also zencc for linux platform
jaromil Sep 24, 2024
43995be
fix: clean also lib/tinycc with distclean
jaromil Oct 8, 2024
f5969c7
fix: dockerfile to use new build
jaromil Oct 8, 2024
72285d0
fix: musl linux target to accept COMPILER env
jaromil Oct 8, 2024
92d7dac
further build system cleanup
jaromil Oct 10, 2024
3df7548
fix: use latest built executables in tests
jaromil Nov 6, 2024
87eefc9
fix: use meson only to run tests, remove build
jaromil Nov 6, 2024
e933c1e
try running linux test matrix without meson build
jaromil Nov 6, 2024
b3d0e1f
fix: build also posix shared library in makefile
jaromil Nov 6, 2024
1652ef2
fix: optimize wasm stderr printing by concatenating log prefix
jaromil Nov 6, 2024
c36cf1a
fix: generated signature tests compatible with js wrapper
jaromil Nov 6, 2024
2f61065
fix: sd_jwt test to use correct extension for zencode test script
jaromil Nov 6, 2024
4f10682
fix: new javascript wrapper for tests
jaromil Nov 6, 2024
6764de3
fix(ci): scenario qp in generic signature tests
jaromil Nov 6, 2024
929b12f
fix: some more test fixes and bats error message
jaromil Nov 6, 2024
8781c20
fix: build system with new makefile changes
jaromil Nov 7, 2024
013efbf
fix: trim down build targets, merge osx and posix
jaromil Nov 7, 2024
2b71f1d
fix: osx executables are copied instead of renamed
jaromil Nov 7, 2024
d6a2b84
fix: reorganize tests in suites
jaromil Nov 7, 2024
8015cd3
fix: cover time import overflow corner case in wasm
matteo-cristino Nov 7, 2024
35a35cd
fix: check-rs checks lua tests
matteo-cristino Nov 7, 2024
b6c124f
chore: adjust build targets in CI
jaromil Nov 11, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
191 changes: 77 additions & 114 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -84,67 +84,46 @@ jobs:
run: |
luacheck --config .luacheckrc --std lua54 .

meson-test:
name: 🥷 meson/ninja test
linux-test:
name: 🐧 Linux check tests (gcc and clang)
needs: [reuse, c-lint, lua-lint]
if: "!contains(github.event.pull_request.labels.*.name, 'SKIP_MESON')"
if: "github.event_name == 'pull_request'"
strategy:
matrix:
command: [CCACHE=1, CLANG=1 CCACHE=1]
command: [COMPILER=gcc, COMPILER=clang]
fail-fast: false
runs-on: ubuntu-latest
runs-on: "ubuntu-latest"
env:
CCACHE: 1
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: actions/setup-python@v5
with:
python-version: '3.10'
python-version: '3.13'
- uses: hendrikmuhs/ccache-action@v1.2
- name: install dependencies
run: |
sudo apt-get update
sudo apt-get install -y vim luajit lua-cjson
sudo apt install -yq jq libreadline-dev
${{ matrix.command }} sudo apt install -yq $COMPILER
pip3 install meson ninja
- name: Build with command ${{ matrix.command }}
- name: Build GNU/Linux ${{ matrix.command }}
run: |
make -f build/meson.mk ${{ matrix.command }}
- name: Run tests in meson
make linux-exe ${{ matrix.command }}
make -C src clean
make linux-lib ${{ matrix.command }}
- name: Check all tests on ${{ matrix.command }}
run: |
cp meson/zencode-exec test/
cp meson/zenroom test/
cp meson/zencc test/
ninja -C meson test

linux-build:
name: 🐧 Linux build tests (musl and linux)
needs: [reuse, c-lint, lua-lint]
if: "github.event_name == 'pull_request'"
runs-on: "ubuntu-latest"
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: hendrikmuhs/ccache-action@v1.2
- name: install dependencies
run: |
sudo apt install -yq jq musl-tools musl-dev libreadline-dev
- name: Build x86_64 with musl-system
run: |
make -f build/musl-linux.mk CCACHE=1
make clean
- name: Build x86_64 shlib with gcc
run: |
make -f build/linux.mk CCACHE=1
- name: Build x86_64 shlib with clang
run: |
make -f build/linux.mk CCACHE=1 COMPILER=clang
make check

windows-build:
name: 🪟 Windows build test
needs: [reuse, c-lint, lua-lint]
if: "github.event_name == 'pull_request'"
runs-on: "ubuntu-latest"
env:
CCACHE: 1
steps:
- uses: actions/checkout@v4
with:
Expand All @@ -155,26 +134,33 @@ jobs:
sudo apt install -y jq gcc-mingw-w64 g++-mingw-w64
- name: Build x86_64 windows .exe
run: |
make -f build/win-exe.mk CCACHE=1
make win-exe
make clean -C src
make -f build/win-dll.mk CCACHE=1
make win-dll

apple-ios-build:
name: 🍎 Apple iOS build test
osx-test:
name: 🍎 Apple OSX check tests
needs: [reuse, c-lint, lua-lint]
if: "github.event_name == 'pull_request'"
runs-on: macos-latest
env:
CCACHE: 1
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: actions/setup-python@v5
with:
python-version: '3.13'
- name: Install dependencies
run: |
pip3 install meson ninja
- uses: hendrikmuhs/ccache-action@v1.2
- run: |
make -f build/apple-osx.mk CCACHE=1
make clean
make -f build/apple-ios.mk ios-arm64 CCACHE=1
make clean
make -f build/apple-ios.mk ios-sim CCACHE=1
make osx-exe
make -C src clean
make osx-lib
make check-osx

go-build-check:
name: 🐹 go build & checks
Expand All @@ -184,6 +170,8 @@ jobs:
strategy:
matrix:
go: ['1.23.1', '1.22.7']
env:
CCACHE: 1
steps:
- uses: actions/checkout@v4
with:
Expand All @@ -195,12 +183,13 @@ jobs:
cache: false
- uses: actions/setup-python@v5
with:
python-version: '3.10'
- run: |
python-version: '3.13'
- name: Install dependencies
run: |
pip3 install meson ninja
sudo apt-get install -qy libreadline-dev
- run: |
make -f build/linux.mk deps zencode-exec CCACHE=1
make linux-exe
cp zencode-exec /usr/local/bin/
- name: 🧪 test bindings go-${{ matrix.go }}
working-directory: bindings/golang/zenroom
Expand All @@ -216,6 +205,8 @@ jobs:
strategy:
matrix:
rust: ['stable', 'nightly']
env:
CCACHE: 1
steps:
- uses: actions/checkout@v4
with:
Expand All @@ -224,40 +215,24 @@ jobs:
- uses: dtolnay/rust-toolchain@stable
with:
toolchain: ${{ matrix.rust }}
- run: |
- uses: actions/setup-python@v5
with:
python-version: '3.13'
- name: Install dependencies
run: |
pip3 install meson ninja
sudo apt-get install -qy libreadline-dev
- run: |
make -f build/linux.mk CCACHE=1
make linux-exe
cp zencode-exec /usr/local/bin/
- name: 🧪 test bindings rust-${{ matrix.rust }}
working-directory: bindings/rust
run: cargo test
- name: 🔐 check crypto with rust-${{ matrix.rust }}
run: make check-rs

# cortex-m-build-check:
# name: 📟 cortex-m build & checks
# needs: [reuse, c-lint, lua-lint]
# runs-on: ubuntu-latest
# if: ${{ contains(github.event.pull_request.labels.*.name, 'run cortex') || contains(github.event.head_commit.message, 'run cortex') }}
# steps:
# - uses: actions/checkout@v4
# with:
# fetch-depth: 0
# - uses: numworks/setup-arm-toolchain@2021-10
# - run: sudo apt install qemu-system-arm
# - name: Download cmsis
# run: |
# curl -LO https://github.com/ARM-software/CMSIS_5/archive/refs/tags/5.7.0.tar.gz
# tar xvf 5.7.0.tar.gz -C lib
# mv lib/CMSIS_5-5.7.0 lib/cmsis
# - run: make cortex-arm
# - name: 🔐 check crypto with cortex-m
# run: make check-cortex-m

js-build-check:
name: 🐯 js build & checks
name: 🐯 NodeJS WASM check tests
needs: [reuse, c-lint, lua-lint]
runs-on: ubuntu-latest
if: "!contains(github.event.pull_request.labels.*.name, 'SKIP_JS')"
Expand All @@ -273,25 +248,21 @@ jobs:
- uses: actions/setup-node@v4
with:
node-version: ${{ matrix.node-version }}
- uses: actions/setup-python@v5
with:
python-version: '3.13'
- uses: mymindstorm/setup-emsdk@v14
with:
version: ${{ env.EM_VERSION }}
- name: Install dependencies
run: |
sudo apt-get -qy update
sudo apt-get install -qy meson ninja-build
- name: Install yarn dependencies
run: |
yarn --cwd bindings/javascript
pip3 install meson ninja
- name: Build Zenroom Wasm
run: |
yarn --cwd bindings/javascript build
make node-wasm
env:
EMSCRIPTEN: ${{ env.EMSDK }}/upstream/emscripten
- name: 🧪 test bindings node-${{ matrix.node-version }}
run: |
yarn --cwd bindings/javascript test
- name: 🔐 check crypto with node-${{ matrix.node-version }}
- name: 🧪 test zencode with node-${{ matrix.node-version }}
run: |
make check-js

Expand Down Expand Up @@ -323,13 +294,13 @@ jobs:
- name: Build zencode-exec on Linux
run: |
cd ../..
make -f build/linux.mk deps zencode-exec
make linux-exe RELEASE=1
cp zencode-exec /usr/local/bin/
if: runner.os == 'Linux'
- name: Build zencode-exec on Apple/OSX
run: |
cd ../..
make -f build/apple-osx.mk deps zencode-exec.command
make osx-exe RELEASE=1
cp zencode-exec.command /usr/local/bin/zencode-exec
if: runner.os == 'macOS'
- run: |
Expand All @@ -341,7 +312,7 @@ jobs:

semantic-release:
name: 🤖 Semantic release
needs: [rust-build-check, go-build-check, meson-test, js-build-check]
needs: [rust-build-check, go-build-check, linux-test, js-build-check]
runs-on: ubuntu-latest
if: ${{ github.ref_name == 'master' && github.event_name == 'push' }}
outputs:
Expand Down Expand Up @@ -384,16 +355,9 @@ jobs:
- uses: mymindstorm/setup-emsdk@v14
with:
version: ${{ env.EM_VERSION }}
- name: Install dependencies
run: |
sudo apt-get -qy update
sudo apt-get install -qy meson ninja-build
- name: Install yarn dependencies
run: |
yarn --cwd bindings/javascript
- name: Build Zenroom Wasm
run: |
yarn --cwd bindings/javascript build
make node-wasm RELEASE=1
env:
EMSCRIPTEN: ${{ env.EMSDK }}/upstream/emscripten
- uses: actions/upload-artifact@v4
Expand All @@ -411,6 +375,9 @@ jobs:
runs-on: ubuntu-latest
needs: [semantic-release]
if: ${{ needs.semantic-release.outputs.release == 'True' }}
env:
CCACHE: 1
RELEASE: 1
steps:
- name: Checkout repository
uses: actions/checkout@v4
Expand All @@ -427,33 +394,34 @@ jobs:
sudo apt install -qy tree jq cmake make gcc g++ musl-tools musl-dev meson clang gcc-mingw-w64 g++-mingw-w64 libreadline-dev ccache
- name: Build x86_64 with musl-system
run: |
make -f build/musl-linux.mk CCACHE=1
make musl RELEASE=1
- name: Upload artifact linux-amd64
uses: actions/upload-artifact@v4
with:
name: release-bin-linux-amd64
path: |
zenroom
zencode-exec
zencc
- name: Clean for next build
run: make clean
run: |
make clean
- name: Build x86_64 shlib
run: |
make -f build/meson.mk
make linux-lib
- name: Upload artifact linux-lib-amd64
uses: actions/upload-artifact@v4
with:
name: release-lib-linux-amd64
path: |
meson/libzenroom.a
meson/libzenroom.so
libzenroom.so
- name: Clean for next build
run: make clean
- name: Build x86_64 windows .exe
run: |
make -f build/win-exe.mk CCACHE=1 RELEASE=1
make win-exe
make clean -C src
make -f build/win-dll.mk CCACHE=1 RELEASE=1
make win-dll
- name: Upload artifact win x86_64
uses: actions/upload-artifact@v4
with:
Expand Down Expand Up @@ -507,31 +475,26 @@ jobs:
needs: [semantic-release]
runs-on: macos-latest
if: ${{ needs.semantic-release.outputs.release == 'True' }}
env:
CCACHE: 1
RELEASE: 1
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: hendrikmuhs/ccache-action@v1.2
- run: |
make -f build/apple-osx.mk CCACHE=1
make clean
make -f build/apple-ios.mk ios-arm64 CCACHE=1
make clean
make -f build/apple-ios.mk ios-sim CCACHE=1
make osx-exe
make -C src clean
make osx-lib
- name: Upload artifacts for Apple OSX
uses: actions/upload-artifact@v4
with:
name: release-apple-osx
path: |
zenroom.command
zencode-exec.command
- name: Upload artifacts for Apple iOS
uses: actions/upload-artifact@v4
with:
name: release-apple-ios
path: |
zenroom-ios-arm64.a
zenroom-ios-x86_64.a
libzenroom.so

npm-release:
name: 📦 npm release
Expand Down
7 changes: 5 additions & 2 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,11 @@ COPY build /build
COPY Makefile Makefile

RUN apk add --no-cache linux-headers build-base cmake readline-dev
RUN make linux
RUN cp /src/zenroom /usr/local/bin/zenroom
RUN make musl-x86 COMPILER=gcc RELEASE=1
RUN mkdir -p /usr/local/bin/
RUN cp zenroom /usr/local/bin/
RUN cp zencode-exec /usr/local/bin/
RUN cp zencc /usr/local/bin/

ENTRYPOINT ["zenroom"]

Loading
Loading