Minor CI improvements #4266
Minor CI improvements #4266
Conversation
.woodpecker.yml
Outdated
| - apt install git | ||
| - find migrations -type f -name '*.sql' -exec pg_format -i {} + | ||
| # ensure that formatting doesn't result in any git changes | ||
| - git diff --exit-code |
There was a problem hiding this comment.
git isn't the right way to handle this, because it would fail on a new migration. I'd recommend just editing the format_check.sh function to use find exec.
There was a problem hiding this comment.
No thats fine, git diff will only give exit code 1 if there are uncommitted changes. So if you add migrations in your PR they are part of a commit, and exit code is 0.
Find exec is complicated because then you need a different temp file for each input. Its annoying that pg_format doesnt have a --check option.
There was a problem hiding this comment.
But how is it going to make sure the new migration that got added is correct sql?
There was a problem hiding this comment.
Another easy way I can think of to speed it up, is in the sql_format_check script, use gnu parallel and make sure the tmp file is a new random one each time.
|
I noticed that drone-cache was silently broken, restarting minio server fixed it. To prevent this in the future, Im adding exit_code flag so that we notice future failures. |
|
I added a format check that can use parallelism now: as long as it fails, you can merge this one into yours. |
Nutomic
force-pushed
the
ci-faster-sql-fmt
branch
from
c9d6346 to
26c1761
Compare
Nutomic
changed the title
|
Not sure how this passed the CI, but its breaking some other PRs. I'll revert for now. |
This reverts commit af4d008.
Updated:
Locally Im getting 42s for running sql_format_check.sh, but only 16s for find -exec. Problem is that the repo files are owned by root, and I dont see any way to run steps as another user in Woodpecker.Edit: Switched from pgformatter image to perl so that it can run as root. However the apt install seems too slow so it still takes 60s+ for the step. Previously it was 1m40s so a bit faster.