Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: add treefmt as code formatting multiplexer, refactor CI to avoid duplication, reorg CI into DevOps workflow #4219

Merged
merged 123 commits into from
Oct 14, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
123 commits
Select commit Hold shift + click to select a range
8315d26
ci: add treefmt as code formatting multiplexer
JakobLichterfeld Sep 18, 2024
2e29513
style: linter findings in entrypoint.sh script
JakobLichterfeld Sep 18, 2024
3c6edd3
style: linter findings for yaml and yml
JakobLichterfeld Sep 18, 2024
f40c2e1
style: linter findings for json files
JakobLichterfeld Sep 18, 2024
7cfa0fd
style: linter findings for nix files
JakobLichterfeld Sep 18, 2024
7d72420
style: linter findings for js files
JakobLichterfeld Sep 18, 2024
fa612f1
style: linter findings for dashboards.sh
JakobLichterfeld Sep 18, 2024
aaf9949
style: linter findings for md and mdx files
JakobLichterfeld Sep 18, 2024
cc0283c
chore: remove unused clang formatter in treefmt config
JakobLichterfeld Sep 18, 2024
37633c0
style: linter findings for mdx files
JakobLichterfeld Sep 18, 2024
8b80e1f
ci: exclude Grafana dashboard JSON files from prettier formatting
JakobLichterfeld Sep 19, 2024
657c74d
Revert "style: linter findings for json files"
JakobLichterfeld Sep 19, 2024
24d49df
ci: exclude Grafana dashboard JSON files from all formatting as we us…
JakobLichterfeld Sep 19, 2024
a69c659
style: linter findings for json files
JakobLichterfeld Sep 19, 2024
953ecdf
doc: update changelog
JakobLichterfeld Sep 19, 2024
99682d7
ci(refactor): use composite action to avoid duplication in elixir wor…
JakobLichterfeld Sep 19, 2024
dbcb8a6
doc: update changelog
JakobLichterfeld Sep 19, 2024
25f8c71
ci: prevent workflow runs for certain conditions and allow scheduled …
JakobLichterfeld Sep 19, 2024
e41b510
ci(refactor): use reusable workflow to check paths
JakobLichterfeld Sep 19, 2024
a0d4c3e
ci(fix): correct output syntax for check_paths workflow and setting b…
JakobLichterfeld Sep 19, 2024
ccaa3dd
ci(refactor): use reusable workflows for streamlined DevOps pipeline
JakobLichterfeld Sep 19, 2024
19b91cc
ci(fix): add write permission for packages in DevOps workflow
JakobLichterfeld Sep 19, 2024
c256302
ci(test): test DevOps workflow
JakobLichterfeld Sep 19, 2024
81c7a11
ci(test): test DevOps workflow
JakobLichterfeld Sep 19, 2024
645da63
ci(fix): Update condition for spell_check, ensure_linting, elixir, an…
JakobLichterfeld Sep 19, 2024
7829116
ci: revert test DevOps
JakobLichterfeld Sep 19, 2024
7885ff0
ci(refactor): allow ghcr_build parallel to elixir test
JakobLichterfeld Sep 20, 2024
e6cf123
ci(refactor): Remove redundant check_paths job from elixir.yml, elixi…
JakobLichterfeld Sep 20, 2024
b58dbfa
feat: add treefmt-nix to nix flake (#4219 - @JakobLichterfeld)
JakobLichterfeld Sep 20, 2024
26d5742
ci: ensure proper linting via treefmt
JakobLichterfeld Sep 20, 2024
a67b17e
ci(test): test ensure_linting workflow
JakobLichterfeld Sep 20, 2024
075e518
ci(fix): checkout code for spell_checker to access file to check
JakobLichterfeld Sep 20, 2024
39ff286
ci(fix): allow impure in ensure_linting workflow
JakobLichterfeld Sep 20, 2024
aa90f55
Revert "ci(test): test ensure_linting workflow"
JakobLichterfeld Sep 20, 2024
df26d04
ci(fix): correct use of flake-utils for formatter and checks
JakobLichterfeld Sep 23, 2024
579548f
ci(fix): correct use of flake-utils for treefmt
JakobLichterfeld Sep 23, 2024
e1f3adb
refactor: Remove unnecessary imports in flake
JakobLichterfeld Sep 23, 2024
2da0319
ci(fix): correct syntax in flake
JakobLichterfeld Sep 23, 2024
155a69b
ci(refactor): Remove unused code in flake.nix
JakobLichterfeld Sep 23, 2024
2860bc3
style: standardised style for input url in flake
JakobLichterfeld Sep 23, 2024
0f1ac9d
ci(fix): treefmt-nix config with existing options
JakobLichterfeld Sep 23, 2024
4c14789
ci(feat): Add Nix binary cache and update treefmt command in CI workflow
JakobLichterfeld Sep 25, 2024
5412286
ci(refactor): Remove unused code in flake.nix
JakobLichterfeld Sep 28, 2024
402c009
fix: include devShell packages only on supported platforms
JakobLichterfeld Sep 30, 2024
027678b
fix: update hash for mix-deps package in flake.nix
JakobLichterfeld Sep 30, 2024
21b0643
ci(fix): Update treefmt command in CI workflow
JakobLichterfeld Sep 30, 2024
7c610f7
ci(test): test ensure_linting workflow
JakobLichterfeld Sep 30, 2024
143d1bd
feat: ensure mix deps are present in devShell
JakobLichterfeld Sep 30, 2024
cef1dec
ci(feat): use flake-parts to enable treefmt-nix
JakobLichterfeld Sep 30, 2024
c271d18
feat: use flake-parts
JakobLichterfeld Sep 30, 2024
eb0611c
fix: correct use of flake-parts for package build
JakobLichterfeld Sep 30, 2024
54cc87d
doc: update CI badge URL for devops workflow
JakobLichterfeld Oct 1, 2024
1da49ef
ci(fix): handle empty path filter output
JakobLichterfeld Oct 1, 2024
49f5dc2
ci: remove --impure flag from treefmt command in CI workflow
JakobLichterfeld Oct 1, 2024
c2817ad
fix: correct treefmt.config settings in formatter.nix
JakobLichterfeld Oct 2, 2024
7442780
fix: correct flake-parts inputs, avoid with in imports
JakobLichterfeld Oct 2, 2024
23028cb
fix: correct program name for mix-format in formatter.nix
JakobLichterfeld Oct 2, 2024
5aed05e
feat: devenv via flake-parts
JakobLichterfeld Oct 2, 2024
ea93892
fix: correct use of legacy nix code with flake-parts
JakobLichterfeld Oct 2, 2024
d637406
ci(fix): correct nix develop command in ensure_linting.yml
JakobLichterfeld Oct 2, 2024
2381bd6
refactor: list imports explicitly in flake, rename folder to flake-mo…
JakobLichterfeld Oct 3, 2024
33b8023
style: use tabs for indent size to format sh
JakobLichterfeld Oct 3, 2024
0820561
style: use nixfmt-rfc-style
JakobLichterfeld Oct 3, 2024
c780198
Revert "style: use nixfmt-rfc-style"
JakobLichterfeld Oct 3, 2024
08559da
style: use nixfmt style
JakobLichterfeld Oct 3, 2024
21daf90
fix: remove glibcLocales from optional dependencies to avoid "A defi…
JakobLichterfeld Oct 3, 2024
2098abb
fix: remove inotify-tools from optional dependencies to avoid "A def…
JakobLichterfeld Oct 3, 2024
b935cfc
fix: Remove inotify-tools and glibcLocales from optional dependencie
JakobLichterfeld Oct 3, 2024
27396b0
fix: correct file paths in flake.nix to version
JakobLichterfeld Oct 3, 2024
d02419c
fix: add ELIXIR_ERL_OPTIONS to shell environment to force utf8 locale
JakobLichterfeld Oct 3, 2024
d45f6e3
fix: add LOCALE_ARCHIVE to shell environment in flake.nix
JakobLichterfeld Oct 3, 2024
ee95db9
Revert "fix: add LOCALE_ARCHIVE to shell environment in flake.nix"
JakobLichterfeld Oct 3, 2024
14cff2f
ci(refactor): rename workflow to elixir_dep_verification_and_static_a…
JakobLichterfeld Oct 3, 2024
9b402f3
ci(debug): debug locale settings
JakobLichterfeld Oct 3, 2024
c1f7f9e
Revert "ci(debug): debug locale settings"
JakobLichterfeld Oct 3, 2024
2d844e5
Revert "fix: add ELIXIR_ERL_OPTIONS to shell environment to force utf…
JakobLichterfeld Oct 3, 2024
761b437
fix: add LOCALE_ARCHIVE to shell environment in flake.nix
JakobLichterfeld Oct 3, 2024
df2eca8
Revert "fix: add LOCALE_ARCHIVE to shell environment in flake.nix"
JakobLichterfeld Oct 3, 2024
4708b01
fix: add LANG=C.UTF-8 to shell environment in flake.nix
JakobLichterfeld Oct 3, 2024
591535e
fix: add mix local.rebar and mix local.hex commands to flake.nix
JakobLichterfeld Oct 3, 2024
8d2c191
fix: pin devenv to version without unix socket bug
JakobLichterfeld Oct 5, 2024
a6ea3f2
chore: update nixpkgs to nixos-24.05 and update dependencies
JakobLichterfeld Oct 5, 2024
4080528
doc: add treefmt config comments
JakobLichterfeld Oct 5, 2024
1ecfa45
ci: do not expose treefmt formatter programs in devshell
JakobLichterfeld Oct 5, 2024
5db417a
fix: correct use of module option to enable PostgreSQL server in flak…
JakobLichterfeld Oct 5, 2024
e670c9d
Revert "chore: update nixpkgs to nixos-24.05 and update dependencies"
JakobLichterfeld Oct 5, 2024
8b7c5a0
feat: consistent use of erlang 26 and elixir 1_16 in flake
JakobLichterfeld Oct 5, 2024
7b43066
ci: switch to macOS runner for linting workflow
JakobLichterfeld Oct 5, 2024
b46ece6
Revert "ci: do not expose treefmt formatter programs in devshell"
JakobLichterfeld Oct 5, 2024
417aed8
Revert "ci: switch to macOS runner for linting workflow"
JakobLichterfeld Oct 5, 2024
2345c14
ci: Remove nixpkgs channel specification in ensure_linting workflow
JakobLichterfeld Oct 5, 2024
07faec5
ci(debug): Add debug output for PATH and NIX_PATH in flake.nix
JakobLichterfeld Oct 5, 2024
a934e4a
Revert "ci(debug): Add debug output for PATH and NIX_PATH in flake.nix"
JakobLichterfeld Oct 5, 2024
ff10459
fix: avoid the need for impure for devenv
JakobLichterfeld Oct 6, 2024
0e4776b
fix: remove invalid custom build.check for formatter and use default
JakobLichterfeld Oct 6, 2024
7947a35
style: linter findings
JakobLichterfeld Oct 6, 2024
bb03033
fix: Add emptyTest to avoid nix flake check test execution on non-Lin…
JakobLichterfeld Oct 6, 2024
b83363c
chore: Remove LANG=C.UTF-8 from enterShell in flake.nix
JakobLichterfeld Oct 6, 2024
6aaddc2
ci(fix): Remove --impure flag from treefmt command in CI mode
JakobLichterfeld Oct 6, 2024
2613a3d
ci(fix): avoid impure mode in ensure_linting workflow
JakobLichterfeld Oct 7, 2024
ff41eef
style: linter findings
JakobLichterfeld Oct 7, 2024
7ecdc77
ci(debug): debug elixir version and locale
JakobLichterfeld Oct 7, 2024
575e7f1
chore: Update flake.lock dependencies
JakobLichterfeld Oct 7, 2024
44f281b
Merge remote-tracking branch 'origin/master' into treefmt
JakobLichterfeld Oct 7, 2024
822eb5a
feat: use newer devenv as unix socket bug is fixed in upstream
JakobLichterfeld Oct 7, 2024
154bf3a
fix: set rebar3 path in devenv
JakobLichterfeld Oct 8, 2024
93268ce
Revert "ci(debug): debug elixir version and locale"
JakobLichterfeld Oct 8, 2024
419170e
ci: re-enable path check in DevOps workflow
JakobLichterfeld Oct 8, 2024
9ca7dc4
doc: update Development and Contributing guide with nix and treefmt
JakobLichterfeld Oct 8, 2024
99f744e
ci: use PostgreSQL 17
JakobLichterfeld Oct 8, 2024
7c80666
style: linter findings
JakobLichterfeld Oct 9, 2024
02abb03
ci(fix): ensure cache name in build action does not contain invalid c…
JakobLichterfeld Oct 9, 2024
38926f3
doc: update changelog
JakobLichterfeld Oct 9, 2024
a1ce9de
Revert "ci(fix): ensure cache name in build action does not contain i…
JakobLichterfeld Oct 9, 2024
d440846
ci: remove branch restriction for check_paths workflow to increase sec
JakobLichterfeld Oct 9, 2024
574c027
ci(fix): run ghcr build workflow only for specific conditions
JakobLichterfeld Oct 9, 2024
96d518d
doc: update changelog
JakobLichterfeld Oct 9, 2024
c0507bc
fix: update hash for mix-deps package in flake.nix
JakobLichterfeld Oct 12, 2024
a7a3074
fix: disable flakeCheck for formatter, as mix format need the dep to …
JakobLichterfeld Oct 13, 2024
fafe5ef
ci(fix): run ghcr build workflow only for specific conditions
JakobLichterfeld Oct 13, 2024
37821c7
fix: move nixosModules.default to top-level attribute set
JakobLichterfeld Oct 13, 2024
e8e58b6
refactor: remove unnecessary config nesting in formatter.nix
JakobLichterfeld Oct 13, 2024
b48ac45
ci(fix): ensure version for buildx is set to correct name
JakobLichterfeld Oct 13, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/actions/build/action.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ inputs:
is_dockerhub_pushed:
description: "Need docker hub login?"
required: true
default: ''
default: ""
docker_password:
description: "Docker password"
required: true
Expand Down
75 changes: 75 additions & 0 deletions .github/actions/setup-elixir-and-cache-deps/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
name: "Setup Elixir and Cache Dependencies"
description: "Setup Elixir, OTP and cache dependencies"
inputs:
elixir-version:
description: "Elixir version"
required: false
default: "1.16.2"
otp-version:
description: "OTP version"
required: false
default: "26"
cache-name-deps:
description: "Cache name for dependencies"
required: true
cache-name-compiled:
description: "Cache name for compiled build"
required: true
mix-env:
description: "Mix environment"
required: false
default: "dev"
ELIXIR_ASSERT_TIMEOUT:
description: "Elixir assert timeout"
required: false
default: "1000"
outputs:
elixir-version:
description: "The Elixir version used in the setup"
value: ${{ steps.beam.outputs.elixir-version }}
otp-version:
description: "The OTP version used in the setup"
value: ${{ steps.beam.outputs.otp-version }}
runs:
using: "composite"
steps:
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7

- name: Setup Elixir and OTP
id: beam
uses: erlef/setup-beam@b9c58b0450cd832ccdb3c17cc156a47065d2114f # v1.18.1
with:
elixir-version: ${{ inputs.elixir-version }}
otp-version: ${{ inputs.otp-version }}

- name: Cache deps
id: cache-deps
uses: actions/cache@0c45773b623bea8c8e75f6c82b208c3cf94ea4f9 # v4.0.2
with:
path: deps
key: ${{ runner.os }}-mix-${{ inputs.cache-name-deps }}-${{ hashFiles('**/mix.lock') }}
restore-keys: |
${{ runner.os }}-mix-${{ inputs.cache-name-deps }}-

- name: Cache compiled build
id: cache-build
uses: actions/cache@0c45773b623bea8c8e75f6c82b208c3cf94ea4f9 # v4.0.2
with:
path: |
_build
priv/cldr/locales
key: ${{ runner.os }}-mix-${{ inputs.cache-name-compiled }}-${{ hashFiles('**/mix.lock') }}
restore-keys: |
${{ runner.os }}-mix-${{ inputs.cache-name-compiled }}-
${{ runner.os }}-mix-

- name: Clean to rule out incremental build as a source of flakiness
if: github.run_attempt > 3
run: |
mix deps.clean --all
mix clean
shell: sh

- name: Install dependencies
run: mix deps.get
shell: sh
23 changes: 7 additions & 16 deletions .github/workflows/buildx.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ name: Publish Docker images

on:
workflow_dispatch:
workflow_call:
schedule:
- cron: "0 3 * * *"
push:
Expand All @@ -10,7 +11,8 @@ on:
paths:
- "**/*"
- "!.github/**" # Important: Exclude PRs related to .github from auto-run
- "!.github/workflows/**" # Important: Exclude PRs related to .github from auto-run
- "!.github/workflows/**" # Important: Exclude PRs related to .github/workflows from auto-run
- "!.github/actions/**" # Important: Exclude PRs related to .github/actions from auto-run
env:
REGISTRY_IMAGE: teslamate/teslamate

Expand All @@ -20,22 +22,11 @@ permissions:

jobs:
check_paths:
runs-on: ubuntu-latest
outputs:
githubfolder: ${{ steps.filter.outputs.githubfolder }}
steps:
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7

- uses: dorny/paths-filter@de90cc6fb38fc0963ad72b210f1f284cd68cea36 # v3.0.2
id: filter
with:
filters: |
githubfolder:
- '.github/**'
uses: ./.github/workflows/check_paths.yml

teslamate_build:
needs: check_paths
if: needs.check_paths.outputs.githubfolder == 'false' || github.event_name == 'schedule'
if: needs.check_paths.outputs.githubfolder != 'true' || github.event_name == 'schedule'
strategy:
fail-fast: false
matrix:
Expand Down Expand Up @@ -71,7 +62,7 @@ jobs:
needs:
- check_paths
- teslamate_build
if: needs.check_paths.outputs.githubfolder == 'false' || github.event_name == 'schedule'
if: needs.check_paths.outputs.githubfolder != 'true' || github.event_name == 'schedule'
steps:
- name: Checkout
uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
Expand All @@ -92,7 +83,7 @@ jobs:
type=edge
grafana:
needs: check_paths
if: needs.check_paths.outputs.githubfolder == 'false' || github.event_name == 'schedule'
if: needs.check_paths.outputs.githubfolder != 'true' || github.event_name == 'schedule'
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
Expand Down
41 changes: 41 additions & 0 deletions .github/workflows/check_paths.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
name: Check paths

on:
workflow_call:
# Map the workflow outputs to job outputs
outputs:
githubfolder:
description: "changes to .github folder"
value: ${{ jobs.check_paths.githubfolder }}
push:
paths:
- "**/*"
- "!.github/**" # Important: Exclude PRs related to .github from auto-run
- "!.github/workflows/**" # Important: Exclude PRs related to .github/workflows from auto-run
- "!.github/actions/**" # Important: Exclude PRs related to .github/actions from auto-run
pull_request_target:
branches: ["master"]
paths:
- "**/*"
- "!.github/**" # Important: Exclude PRs related to .github from auto-run
- "!.github/workflows/**" # Important: Exclude PRs related to .github/workflows from auto-run
- "!.github/actions/**" # Important: Exclude PRs related to .github/actions from auto-run

permissions:
contents: read

jobs:
check_paths:
runs-on: ubuntu-latest
outputs:
githubfolder: ${{ steps.filter.outputs.githubfolder }}
steps:
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7

- uses: dorny/paths-filter@de90cc6fb38fc0963ad72b210f1f284cd68cea36 # v3.0.2
id: filter
with:
base: "master" # needed to set as a called workflow does not have direct access to repository.default_branch
filters: |
githubfolder:
- '.github/**'
58 changes: 58 additions & 0 deletions .github/workflows/devops.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
name: DevOps

on:
workflow_dispatch:
push:
paths:
- "**/*"
- "!.github/**" # Important: Exclude PRs related to .github from auto-run
- "!.github/workflows/**" # Important: Exclude PRs related to .github/workflows from auto-run
- "!.github/actions/**" # Important: Exclude PRs related to .github/actions from auto-run
pull_request:
branches: ["master"]
paths:
- "**/*"
- "!.github/**" # Important: Exclude PRs related to .github from auto-run
- "!.github/workflows/**" # Important: Exclude PRs related to .github/workflows from auto-run
- "!.github/actions/**" # Important: Exclude PRs related to .github/actions from auto-run

permissions:
contents: read
packages: write

jobs:
check_paths:
uses: ./.github/workflows/check_paths.yml

spell_check:
needs: check_paths
if: needs.check_paths.outputs.githubfolder != 'true' || github.event_name == 'schedule'
uses: ./.github/workflows/spell_check.yml

ensure_linting:
needs:
- check_paths
- spell_check
if: needs.check_paths.outputs.githubfolder != 'true' || github.event_name == 'schedule'
uses: ./.github/workflows/ensure_linting.yml

elixir_dep_verification_and_static_analysis:
needs:
- check_paths
- ensure_linting
if: needs.check_paths.outputs.githubfolder != 'true' || github.event_name == 'schedule'
uses: ./.github/workflows/elixir_dep_verification_and_static_analysis.yml

elixir_test:
needs:
- check_paths
- ensure_linting
if: needs.check_paths.outputs.githubfolder != 'true' || github.event_name == 'schedule'
uses: ./.github/workflows/elixir_test.yml

ghcr_build:
needs:
- check_paths
- elixir_dep_verification_and_static_analysis
if: needs.check_paths.outputs.githubfolder != 'true' || github.event_name == 'schedule'
uses: ./.github/workflows/ghcr_build.yml
Loading
Loading