Skip to content

Commit

Permalink
chore(ci): migrate engine size dashboard to pure bash and GH Actions (#…
Browse files Browse the repository at this point in the history
…5095)

Due to the lack of Nix buy-in from the team and with most of Nix users
leaving, and given the increased overhead of keeping multiple separate
build systems and workflows in sync after the introduction of the
WebAssembly tooling and difficulties in making changes in Nix code for
team members who don't use it, it was previously decided to decrease our
reliance on Nix and stop using it on CI, leaving it as optional and only
for local development, which mostly happened [in
February](#4713). However,
the engines size dashboard was still powered by Nix because we ran out
of the allocated time for the tech debt task.

After the Nix flake was updated last time, the workflow was broken
because `wasm-bindgen-cli` in the flake was at 0.2.95 while we are
currently pinned to 0.2.93 and are blocked from upgrading to a newer
version at the moment. Rather than pinning `wasm-bindgen-cli` to 0.2.93
in the flake by taking it from a different nixpkgs commit, it's a good
opportunity to start using the same infrastructure we use for other
GitHub Actions jobs instead.

With that, and given the fact that our workflows and build scripts are
heavily dependent on rustup and we even used rustup within the dev shell
instead of the toolchain from `rust-overlay`, there's not much benefit
for the local dev shell for Nix users to be a flake, a classic
`shell.nix` is more appropriate: pinning the state of the environment in
`flake.lock` is no longer useful and only gets in the way. As an added
bonus, classic Nix doesn't require copying the sources to the store,
which makes the shell startup a bit faster.

Fixes: prisma/team-orm#1444
Closes: #5072
  • Loading branch information
aqrln authored Dec 19, 2024
1 parent 3aa9260 commit 59c39b6
Show file tree
Hide file tree
Showing 10 changed files with 115 additions and 485 deletions.
3 changes: 3 additions & 0 deletions .editorconfig
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
[*.{yml,yaml}]
indent_style = space
indent_size = 2
7 changes: 2 additions & 5 deletions .envrc
Original file line number Diff line number Diff line change
Expand Up @@ -45,14 +45,11 @@ fi
# export TEST_CONNECTOR="postgres"
# export TEST_CONNECTOR_VERSION="10"

# Set up env vars and build inputs from flake.nix automatically for nix users.
# Set up env vars and build inputs from shell.nix automatically for nix users.
# If you don't use nix, you can safely ignore this.
# You can set the DISABLE_NIX environment variable if you're in an environment
# where nix is pre-installed (e.g. gitpod) but you don't want to use it.
if command -v nix &> /dev/null && [ -z ${DISABLE_NIX+x} ]
then
if nix flake metadata > /dev/null; then
watch_file nix/shell.nix nix/all-engines.nix nix/args.nix
use flake
fi
use nix
fi
70 changes: 59 additions & 11 deletions .github/workflows/on-push-to-main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,20 +12,68 @@ concurrency:
jobs:
publish-to-gh-pages:
runs-on: ubuntu-latest
strategy:
fail-fast: true

env:
CSV_PATH: engines-size/data.csv

steps:
- uses: actions/checkout@v4
- uses: cachix/install-nix-action@v30
with:
# we need internet access for the moment
extra_nix_config: |
sandbox = false

- run: |
- uses: ./.github/workflows/include/rust-wasm-setup

- name: Build native engines
run: |
cargo build --release -p query-engine -p query-engine-node-api
mv target/release/libquery_engine.{so,node}
- name: Build WASM engines
env:
WASM_BUILD_PROFILE: release
PKG_DIR: query-engine/query-engine-wasm/pkg
TARGET_DIR: target/query-engine-wasm
run: |
make build-qe-wasm-gz
mkdir -p $TARGET_DIR
for provider in "postgresql" "mysql" "sqlite"; do
cp $PKG_DIR/$provider/query_engine_bg.wasm $TARGET_DIR/query-engine-$provider.wasm
cp $PKG_DIR/$provider.gz $TARGET_DIR/query-engine-$provider.wasm.gz
done
- name: Check out gh-pages branch
run: |
git fetch --depth=1 origin gh-pages
git checkout origin/gh-pages
- name: Update engines size
run: |
files=(
target/release/query-engine
target/release/libquery_engine.node
target/query-engine-wasm/query-engine-postgresql.wasm.gz
target/query-engine-wasm/query-engine-postgresql.wasm
target/query-engine-wasm/query-engine-mysql.wasm.gz
target/query-engine-wasm/query-engine-mysql.wasm
target/query-engine-wasm/query-engine-sqlite.wasm.gz
target/query-engine-wasm/query-engine-sqlite.wasm
)
DATE_TIME="$(date -u --iso-8601=seconds)"
if [[ ! -f $CSV_PATH ]]; then
echo "date_time,branch,commit,file,size_bytes" > "$CSV_PATH"
fi
for file in "${files[@]}"; do
file_name=$(basename "$file")
size=$(stat -c %s "$file")
echo "$DATE_TIME,$GITHUB_REF_NAME,$GITHUB_SHA,$file_name,$size" >> "$CSV_PATH"
done
- name: Commit the changes
run: |
git config user.email "prismabots@gmail.com"
git config user.name "prisma-bot"
- name: Publish engines size to gh-pages branch
run: nix run .#publish-engine-size
git add "$CSV_PATH"
git commit --quiet -m "update engines size for $GITHUB_SHA"
git push origin '+HEAD:gh-pages'
121 changes: 0 additions & 121 deletions flake.lock

This file was deleted.

32 changes: 0 additions & 32 deletions flake.nix

This file was deleted.

5 changes: 0 additions & 5 deletions nix/README.md

This file was deleted.

15 changes: 0 additions & 15 deletions nix/args.nix

This file was deleted.

Loading

0 comments on commit 59c39b6

Please sign in to comment.