Add github action running on each push

[tobiasdiez]

Add a github action that runs on each push. Since it uses the most recent docker image as the base, the runtime is relatively short with about 2h. The idea is to get quick feedback on PRs without the need to wait until the patchbot picks it up. After this ticket is merged, we can add a badge in the ticket similar to the linter workflow.

Example run: https://github.com/sagemath/sagetrac-mirror/actions/workflows/build.yml
(it works but one doctest and the pytests are failing; I think there are already tickets for these issues)

Depends on #33103

CC: @mkoeppe @fchapoton

Component: build

Author: Tobias Diez

Branch/Commit: 6329816

Reviewer: Matthias Koeppe

Issue created by migration from https://trac.sagemath.org/ticket/33263

[sagetrac-git]

Changed commit from 676799b to 1ab0755

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

1ab0755

Merge branch 'develop' of https://github.com/sagemath/sage into public/build/github_build

[mkoeppe]

comment:3

Why not just configure --prefix=/sage/local?

[mkoeppe]

comment:4

Which jobs does it cancel?

[mkoeppe]

comment:5

You may want to use #33233

[mkoeppe]

comment:6

Replying to @mkoeppe:

Which jobs does it cancel?

Also perhaps the new built-in concurrency stuff can be used? https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#concurrency

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

d27ca7f

Improve based on feedback

[sagetrac-git]

Changed commit from 1ab0755 to d27ca7f

[sagetrac-git]

Changed commit from d27ca7f to 4f5bcb9

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

4f5bcb9

Test that cancel works as desired

[tobiasdiez]

comment:9

Thanks for the feedback. I've now used --prefix and concurrency (to cancel previous runs of the same workflow for the same branch).

[tobiasdiez]

comment:10

Replying to @mkoeppe:

You may want to use #33233

The idea is that this workflow never fails (otherwise the branch is not merged). Thus the base line is "always green".

[tobiasdiez]

comment:11

Replying to @mkoeppe:

Why not just configure --prefix=/sage/local?

But now it seems to rebuild all python packages instead of using them from the docker image as before. Is this a bug in prefix or did I use it wrong?

[mkoeppe]

comment:13

Sorry, it should be configure --prefix=/sage/local --with-sage-venv

[mkoeppe]

comment:14

Replying to @tobiasdiez:

Replying to @mkoeppe:

You may want to use #33233

The idea is that this workflow never fails (otherwise the branch is not merged). Thus the base line is "always green".

The patchbot makes the same optimistic assumption, which is why it's broken so often...

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

3b0e99f

Use --with-sage-venv

[sagetrac-git]

Changed commit from 4f5bcb9 to 3b0e99f

[tobiasdiez]

comment:16

Replying to @mkoeppe:

Sorry, it should be configure --prefix=/sage/local --with-sage-venv

Thanks, that worked well!

[mkoeppe]

comment:17

Another idea, for a much faster build, would be to just work out of /sage, merging the current commit.

[tobiasdiez]

comment:18

I think with a bit more than 1.5h runtime, it is already reasonably fast. The actual build also only takes about 35mins, the rest is spent on the tests. Further optimization in my opinion will be a trade-off between time savings and reduced reliability, because you then have to think about invalidation of the cythonize cache etc. The point of this ticket is to pursue a different approach than the patchbot and really start from a clean state. I would propose to first gain experience with this, before we implement more fancy caching mechanisms (and in this case I would also prefer github actions builtin cache mechanism).

[mkoeppe]

comment:19

Sure, sounds good.

[mkoeppe]

comment:20

However I think it should only be run for branches on tickets set to needs_review.
After making git-trac available (see #33222), you can use something like git trac search --branch u/mkoeppe/upgrade_giac_to_1_7_0
and git trac print 33263 | grep '^Status: needs_review' for this

[mkoeppe]

comment:21

Hm... no, people set "needs_review" after a push, not before

[mkoeppe]

Dependencies: #33103

[mkoeppe]

comment:22

OK, let's just merge it and try it out. I'd suggest to set #33103 as a dependency and already put dev instead of 9.5 as image tag

[mkoeppe]

Reviewer: Matthias Koeppe

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

6329816

Use dev docker tag

[sagetrac-git]

Changed commit from 3b0e99f to 6329816

[tobiasdiez]

comment:24

Replying to @mkoeppe:

OK, let's just merge it and try it out. I'd suggest to set #33103 as a dependency and already put dev instead of 9.5 as image tag

Thanks for the review. It now uses dev and I set the ticket to positive review now (please intervene if I misunderstood you).

[tobiasdiez]

comment:25

Increasing priority as the badge is already added to the trac ticket (so that I don't have to login in the VM and change the config multiple times).

[vbraun]

Changed branch from public/build/github_build to 6329816