Skip to content

Commit

Permalink
[native] Convert CircleCI jobs for PrestoC++ to Github actions
Browse files Browse the repository at this point in the history
This pull request converts the CircleCI workflows to GitHub actions workflows.
The workflows use the standard runners with Ubuntu 22.04.

Some modifications were made to allow a build with the limited resources available:

|name|build type|comments|expected runtime (no cache)|unit tests|E2E tests|
|----|----------|--------|----------------|----------|---------|
| prestocpp-linux-build | debug | All adapters enabled | 45m | N | N |
| prestocpp-linux-build-and-unit-test | release | JWT, remote function, Parquet enabled | 90m | Y | Y |
| prestocpp-macos-build | debug | Setup script plus build | 120m | N | N |
| prestocpp-linux-adapters-build | release | Adapters only build | 20m | N | N |

Caching is currently enabled for the prestocpp-linux-build-and-unit-test.
If the cache can be used, the time to run tests drops to 50m.

Additional workflows are prestocpp-format-check and prestocpp-header-check
That run in a few seconds.

For the original conversion see: #21545

Co-authored-by: Rob Anderson <robandpdx@github.com>
  • Loading branch information
czentgr and robandpdx committed Dec 12, 2024
1 parent ed176b9 commit f662fea
Show file tree
Hide file tree
Showing 7 changed files with 506 additions and 0 deletions.
24 changes: 24 additions & 0 deletions .github/workflows/prestocpp-format-check.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
name: prestocpp-format-check

on:
workflow_dispatch:
pull_request:
paths:
- 'presto-native-execution/**'

jobs:
prestocpp-format-check:
runs-on: ubuntu-latest
container:
image: public.ecr.aws/oss-presto/velox-dev:check
steps:
- uses: actions/checkout@v4
- name: Fix git permissions
# Usually actions/checkout does this but as we run in a container
# it doesn't work
run: git config --global --add safe.directory ${GITHUB_WORKSPACE}
- name: Check formatting
run: |
git fetch origin master
cd presto-native-execution
make format-check
24 changes: 24 additions & 0 deletions .github/workflows/prestocpp-header-check.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
name: prestocpp-header-check

on:
workflow_dispatch:
pull_request:
paths:
- 'presto-native-execution/**'

jobs:
prestocpp-header-check:
runs-on: ubuntu-latest
container:
image: public.ecr.aws/oss-presto/velox-dev:check
steps:
- uses: actions/checkout@v4
- name: Fix git permissions
# Usually actions/checkout does this but as we run in a container
# it doesn't work
run: git config --global --add safe.directory ${GITHUB_WORKSPACE}
- name: Check license headers
run: |
git fetch origin master
cd presto-native-execution
make header-check
31 changes: 31 additions & 0 deletions .github/workflows/prestocpp-linux-adapters-build.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
name: prestocpp-linux-adapters-build

on:
workflow_dispatch:
pull_request:
paths:
- 'presto-native-execution/scripts/**'

jobs:
prestocpp-linux-build-adapters:
runs-on: ubuntu-22.04
container:
image: prestodb/presto-native-dependency:0.290-20241014120930-e1fc090
steps:
- uses: actions/checkout@v4
- name: Fix git permissions
# Usually actions/checkout does this but as we run in a container
# it doesn't work
run: git config --global --add safe.directory ${GITHUB_WORKSPACE}
- name: Update submodules
run: |
cd presto-native-execution
make submodules
- name: Build all adapter dependencies
run: |
mkdir -p ${GITHUB_WORKSPACE}/adapter-deps/install
source /opt/rh/gcc-toolset-12/enable
set -xu
cd presto-native-execution
DEPENDENCY_DIR=${GITHUB_WORKSPACE}/adapter-deps PROMPT_ALWAYS_RESPOND=n ./velox/scripts/setup-adapters.sh
DEPENDENCY_DIR=${GITHUB_WORKSPACE}/adapter-deps PROMPT_ALWAYS_RESPOND=n ./scripts/setup-adapters.sh
Loading

0 comments on commit f662fea

Please sign in to comment.