Next Gereration CI (10.9+ on GitHub Actions)

[h1dden-da3m0n]

General discussion thread and planing for the future of our build and CI structure and process.

Go wild 😉

[h1dden-da3m0n]

Regarding plugin CI centralization: I have a draft with 50% working so far.
Working:

• testing & building on pushes and PRs
• CodeQL
• label-sync
• Fix the /commands workflow & tirgger
• Testing the release drafter workflow

Missing:

• finish up the 'publish' workflow that will replace the webhook call on plugin releases and uploads directly

[h1dden-da3m0n]

Idea regarding Server & Web CI:

Move packaging (deb, rpm, portable) out of the repos. Containers may still be build so PRs could do incremental test builds, as well as pushes to the default branch. Both can be pushed to GHCR and tested be used to test.
So the only CI remaining in those repos:

• running tests
• building the regualr way (i.e. dotnet publish and npm ci)
• do static analysis (i.e. SonarQube, CodeQL, ...)
• utility such as bump_version (in some form)
• Commands workflows
• did I miss anything?

Have seperated Repositories for:

• Portable builds
• Deb builds
• (RPM builds)
• Contaienr (i.e. Docker/OCI)

Those separated repos build all the Server, Web and Meta packages.
They only build actual tags / releases / pre-releases of web and/or server.
Tags in those repositories represent the version of the meta packages (so implicitly the version of the server in it) and an incremental build number (this is just an idea/ proposal).
To keep track what version is bundled in meta packages there are multiple ways, I happen to have a idea of how to handle it due to DevOps Con and can explain if desired.

So what would be the proposed workflow:

1. development in server/web is independence and 'incremental' builds are provided / accessible as container and or packaged zip/tar.
2. ffmpeg is not affected as it has its own CI that can be updated if need be
3. once server or web hits a release (be that pre-release or release published) it triggers all the 'packaging' repo CI
4. the packaging repo CI will build everything to the latest available release or pre-release as well as a meta package
5. this then gets attached to a release within the packaging repo and uploaded to the repo server.
6. the CI in each of the packaging repos is independent and may fail, yet never breaks a release

this is all I currently can elaborate on this idea that I had in draft status for quite a while, so it is 100% not perfect and missing some aspects for sure

[ferferga]

I would keep all the building stuff in the respective repos for simplicity and because users cloning still need to build locally. So the idea is to share the same build process than an user cloning would. This would roughly simplify the process and decouple it perfectly imo:

In code's repo:

• Job 1

• Build Docker image for each arch we support. Publish to DockerHub and GHCR

• Job 2

• From the Docker images in job 1, build OS specific packages

• Job 3

• If the two previous job succeed, dispatch a workflow in metapackages.

So, for instance, target Linux-arm64 currently has three packages: portable, deb and rpm. We just build Linux-arm64 once in Job 1, but share it with all the target packages (for arm64) in Job 2

In metapackages:

• Job 1:

• Dockerfile pulls the current tagged unstable/stable images from jellyfin-server and jellyfin-web docker repos. Merges them for all arches and pushes to DockerHub and GHCR at jellyfin

• Job 2:

• Same mechanism than Job 1, but with OS specific and portable packages.

[ferferga]

Everything should also be versioned imo based on commit (they're not that important to show for unstable, specially since lately all the version have been a mess in regards that we might want to release X version but we need to retag again due to CI failures, so it ends up being X + 1 lol)

So exactly on the day of release there will be in reality two exact versions: one in the unstable "channel" with the commit number (released as a part of our commit pipelines) and another one in the "stable" channel with whatever version was tagged (released as part of our tagging pipeline)

The build behaviour of Job 1 and 2 in the code repo's is also easily replicable in a normal system with Docker, so it's not 100% tied to GH Actions and its easily shareable with users building from a clond

[thornbill]

Possible reference for cross-repo GitHub Action messaging: https://blog.marcnuri.com/triggering-github-actions-across-different-repositories

[h1dden-da3m0n]

I mean I successfully tested cross repo triggers b4 with https://github.com/peter-evans/repository-dispatch. but thank you, I shall give that a read too

okay they just use curl and do the same what Repository Dispatch abstracts. 👍

[thornbill]

Oh nice! I had just found that when Josh asked yesterday and didn't want it to get lost to the depths of the Matrix. 😂