fix(plausible): migrate to new common chart and upgrade to v2 #11186

schnerring · 2023-08-04T04:44:28Z

Description

cherry-picked changes from my PR that were never merged into from port/incubator into master: fix(plausible): Remove deprecated config options #7810
Added config options for Plausible v2.0.0
Migrated to new common chart

TODO:

upgrade questions.yaml to new common chart
migrate common.yaml / _env.tpl to new common chart format

⚒️ Fixes #

⚙️ Type of change

⚙️ Feature/App addition
🪛 Bugfix
⚠️ Breaking change (fix or feature that would cause existing functionality to not work as expected)
🔃 Refactor of current code

🧪 How Has This Been Tested?

📃 Notes:

✔️ Checklist:

⚖️ My code follows the style guidelines of this project
👀 I have performed a self-review of my own code
#️⃣ I have commented my code, particularly in hard-to-understand areas
📄 I have made corresponding changes to the documentation
⚠️ My changes generate no new warnings
🧪 I have added tests to this description that prove my fix is effective or that my feature works
⬆️ I increased versions for any altered app according to semantic versioning

➕ App addition

If this PR is an app addition please make sure you have done the following.

🪞 I have opened a PR on truecharts/containers adding the container to TrueCharts mirror repo.
🖼️ I have added an icon in the Chart's root directory called icon.png

Please don't blindly check all the boxes. Read them and only check those that apply.
Those checkboxes are there for the reviewer to see what is this all about and
the status of this PR with a quick glance.

charts/incubator/plausible/questions.yaml

charts/incubator/plausible/values.yaml

schnerring · 2023-08-04T15:33:04Z

Any idea what to do about this linting error?

Error: execution error at (plausible/templates/common.yaml:2:4): Operator [cloudnative-pg] has to be installed first

stavros-k · 2023-08-06T08:36:45Z

Any idea what to do about this linting error?

Error: execution error at (plausible/templates/common.yaml:2:4): Operator [cloudnative-pg] has to be installed first

Commented out the helm template from the linting stage, as it would require operators to be installed there as well.
Should be passing now

charts/incubator/plausible/values.yaml

charts/incubator/plausible/templates/_secret.tpl

charts/incubator/plausible/values.yaml

stavros-k · 2023-08-06T08:53:38Z

For catalog test failing, you can "generate" questions and see for errors
From the root of the project

./tools/gen-cat.sh charts/incubator/plausible

Output will go to a new dir on the root of the project called catalog

charts/incubator/plausible/values.yaml

schnerring · 2023-08-06T19:53:15Z

./tools/gen-cat.sh charts/incubator/plausible

That command returns the following error when running it:

Error: no cached repository for helm-manager-abaa37bfaf705b7726f614114b77dd23196b42f5087901e57467be7fa3fead01 found. (try 'helm repo update'): open /home/***/.cache/helm/repository/helm-manager-abaa37bfaf705b7726f614114b77dd23196b42f5087901e57467be7fa3fead01-index.yaml: no such file or directory

Any idea what I'm missing?

stavros-k · 2023-08-06T20:06:40Z

./tools/gen-cat.sh charts/incubator/plausible

That command returns the following error when running it:

Error: no cached repository for helm-manager-abaa37bfaf705b7726f614114b77dd23196b42f5087901e57467be7fa3fead01 found. (try 'helm repo update'): open /home/***/.cache/helm/repository/helm-manager-abaa37bfaf705b7726f614114b77dd23196b42f5087901e57467be7fa3fead01-index.yaml: no such file or directory

Any idea what I'm missing?

Mine seem to work, maybe you need to add the repositories

❯ ./tools/gen-cat.sh charts/incubator/plausible 
No owner configured, defaulting to truecharts
No repo configured, defaulting to charts
env #CR_TOKEN not found, defaulting to production=false
"truecharts" already exists with the same configuration, skipping
"truecharts-library" already exists with the same configuration, skipping
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "truecharts-library" chart repository
...Successfully got an update from the "truecharts-deps" chart repository
...Successfully got an update from the "truecharts" chart repository
...Successfully got an update from the "bitnami" chart repository
...Successfully got an update from the "prometheus" chart repository
Update Complete. ⎈Happy Helming!⎈
Start processing charts/incubator/plausible ...
Saving 3 charts
Downloading common from repo https://library-charts.truecharts.org
Downloading postgresql from repo https://deps.truecharts.org/
Downloading clickhouse from repo https://deps.truecharts.org/
Deleting outdated charts
Cleaning SCALE catalog for Chart: plausible
Copying Chart to Catalog: plausible
Applying SCALE patches for Chart: plausible
Making copy of charts/incubator/plausible/questions.yaml to catalog/incubator/plausible/4.0.25/questions.yaml
Including standardised questions.yaml includes for: plausible
Removing old versions for: plausible
/home/skois/projects/truecharts/apps
Done processing charts/incubator/plausible ...
Starting post-processing

schnerring · 2023-08-07T02:31:11Z

Mine seem to work, maybe you need to add the repositories

tools/get-cat.sh was missing the truecharts-deps repo, I added it to the prep_helm function of the script.

Regarding questions.yaml, I didn't really know what I was doing / looking for, so I just did a side-by-side comparison with the up-to-date paperless-ngx questions.yaml and changed the plausible questions.yaml accordingly.

Now the "install charts" CI step fails. It looks like ClickHouse is the issue. Here's the event log of the plausible-luh9hueop1-clickhouse-5bd7db77f8-qnfql pod:

Events:
  Type     Reason     Age                    From               Message
  ----     ------     ----                   ----               -------
  Normal   Scheduled  4m4s                   default-scheduler  Successfully assigned plausible-luh9hueop1/plausible-luh9hueop1-clickhouse-5bd7db77f8-qnfql to k3d-k3s-default-server-0
  Normal   Pulling    4m3s                   kubelet            Pulling image "tccr.io/truecharts/clickhouse:v23.7.1.2470@sha256:fbec51f0c6b09733fe4601bba2584dc750f0a8a5077702b86163744bd6f4573f"
  Normal   Pulled     3m46s                  kubelet            Successfully pulled image "tccr.io/truecharts/clickhouse:v23.7.1.2470@sha256:fbec51f0c6b09733fe4601bba2584dc750f0a8a5077702b86163744bd6f4573f" in 16.389378822s
  Normal   Pulled     102s (x11 over 3m46s)  kubelet            Container image "tccr.io/truecharts/clickhouse:v23.7.1.2470@sha256:fbec51f0c6b09733fe4601bba2584dc750f0a8a5077702b86163744bd6f4573f" already present on machine
  Warning  Failed     102s (x12 over 3m46s)  kubelet            Error: secret "clickhousecreds" not found

I guess this is due the common chart refactorings to the ClickHouse chart / library-chart? Gotta do some more debugging..........

Signed-off-by: Stavros Kois <47820033+stavros-k@users.noreply.github.com>

schnerring · 2023-08-08T23:14:56Z

Things are looking good! However, CNPG fails to start up in the Install charts Part 1 tests. Any idea why? I don't see anything in the logs.

 Events:
  Type     Reason     Age                    From               Message
  ----     ------     ----                   ----               -------
  Normal   Scheduled  3m36s                  default-scheduler  Successfully assigned plausible-vu2tnxzq9l/plausible-vu2tnxzq9l-cnpg-main-1 to k3d-k3s-default-server-0
  Normal   Pulled     3m37s                  kubelet            Container image "tccr.io/truecharts/cloudnative-pg:v1.20.2@sha256:07121d674c1de587044746f92cc3bccb4576f2fc125f6f2f5bcaa582dad3e3ad" already present on machine
  Normal   Created    3m37s                  kubelet            Created container bootstrap-controller
  Normal   Started    3m36s                  kubelet            Started container bootstrap-controller
  Normal   Pulled     3m36s                  kubelet            Container image "ghcr.io/cloudnative-pg/postgresql:15.3" already present on machine
  Normal   Created    3m36s                  kubelet            Created container postgres
  Normal   Started    3m36s                  kubelet            Started container postgres
  Warning  Unhealthy  3m33s (x3 over 3m35s)  kubelet            Readiness probe failed: HTTP probe failed with statuscode: 500

stavros-k · 2023-08-12T09:30:22Z

Things are looking good! However, CNPG fails to start up in the Install charts Part 1 tests. Any idea why? I don't see anything in the logs.

 Events:
  Type     Reason     Age                    From               Message
  ----     ------     ----                   ----               -------
  Normal   Scheduled  3m36s                  default-scheduler  Successfully assigned plausible-vu2tnxzq9l/plausible-vu2tnxzq9l-cnpg-main-1 to k3d-k3s-default-server-0
  Normal   Pulled     3m37s                  kubelet            Container image "tccr.io/truecharts/cloudnative-pg:v1.20.2@sha256:07121d674c1de587044746f92cc3bccb4576f2fc125f6f2f5bcaa582dad3e3ad" already present on machine
  Normal   Created    3m37s                  kubelet            Created container bootstrap-controller
  Normal   Started    3m36s                  kubelet            Started container bootstrap-controller
  Normal   Pulled     3m36s                  kubelet            Container image "ghcr.io/cloudnative-pg/postgresql:15.3" already present on machine
  Normal   Created    3m36s                  kubelet            Created container postgres
  Normal   Started    3m36s                  kubelet            Started container postgres
  Warning  Unhealthy  3m33s (x3 over 3m35s)  kubelet            Readiness probe failed: HTTP probe failed with statuscode: 500

I'm not sure that its stays Unhealthy, its just the last event logged. If it stayed unhealthy it would be somelike like x20-x30 over the time.

What I've seen is this

------------------------------------------------------------------------------------------------------------------------
==> Logs of init container plausible-vu2tnxzq9l-c6665c7b6-8rlr8
------------------------------------------------------------------------------------------------------------------------
Loading plausible..
Starting dependencies..
Starting repos..
create Plausible.Repo database if it doesn't exist
create Plausible.IngestRepo database if it doesn't exist
Creation of Db successful!
Loading plausible..
Starting dependencies..
Starting repos..
Running migrations for Elixir.Plausible.Repo
Running migrations for Elixir.Plausible.IngestRepo
Migrations successful!
------------------------------------------------------------------------------------------------------------------------
<== Logs of init container plausible-vu2tnxzq9l-c6665c7b6-8rlr8
------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------------------------------
==> Logs of init container plausible-vu2tnxzq9l-c6665c7b6-8rlr8
------------------------------------------------------------------------------------------------------------------------
/bin/sh: /bin/bash: not found
------------------------------------------------------------------------------------------------------------------------
<== Logs of init container plausible-vu2tnxzq9l-c6665c7b6-8rlr8
------------------------------------------------------------------------------------------------------------------------

Is there any chance the migration entrypoint.sh trying to use bash?

stavros-k · 2023-08-12T09:43:56Z

@schnerring Found the issue
https://github.com/truecharts/library-charts/commit/48b8e123edb431d114e402f51706d1a5b3d4380f

charts/incubator/plausible/Chart.yaml

Signed-off-by: Stavros Kois <47820033+stavros-k@users.noreply.github.com>

schnerring · 2023-08-12T13:34:41Z

@schnerring Found the issue truecharts/library-charts@48b8e12

Nice find, thanks for the help!

truecharts-admin · 2023-08-20T00:21:06Z

This PR is locked to prevent necro-posting on closed PRs. Please create a issue or contact staff on discord if you want to further discuss this

schnerring commented Aug 4, 2023

View reviewed changes

charts/incubator/plausible/questions.yaml Show resolved Hide resolved

schnerring commented Aug 4, 2023

View reviewed changes

charts/incubator/plausible/values.yaml Show resolved Hide resolved

schnerring force-pushed the upgrade-plausible branch 4 times, most recently from f300bdc to 32bbef2 Compare August 4, 2023 15:17

stavros-k reviewed Aug 6, 2023

View reviewed changes

charts/incubator/plausible/values.yaml Outdated Show resolved Hide resolved

stavros-k reviewed Aug 6, 2023

View reviewed changes

charts/incubator/plausible/templates/_secret.tpl Outdated Show resolved Hide resolved

stavros-k reviewed Aug 6, 2023

View reviewed changes

charts/incubator/plausible/values.yaml Outdated Show resolved Hide resolved

stavros-k reviewed Aug 6, 2023

View reviewed changes

charts/incubator/plausible/values.yaml Outdated Show resolved Hide resolved

schnerring and others added 14 commits August 9, 2023 00:45

fix(plausible): Remove deprecated config options

ea76481

fix(plausible): migrate Chart.yaml to new common chart

703462e

fix(plausible): use image from tccr.io

5b3a787

fix(plausible): migrate values.yaml to new common chart and use CNPG

a502fea

fix(plausible): add v2 configuration options

363abc4

fix(plausible): fix linting errors

33253b1

fix(plausible): migrate template files to new common format

907b49f

fix(plausible): fix CLICKHOUSE_DATABASE_URL

9437f0f

fix(plausible): remove invalid service protocol

162e35e

Update charts/incubator/plausible/values.yaml

2288f19

Signed-off-by: Stavros Kois <47820033+stavros-k@users.noreply.github.com>

fix(plausible): implement code review suggestions

7b08470

Add deps.truecharts.org helm repo to tools/gen-cat.sh

6c97040

fix(plausible): migrate questions.yaml to new common chart layout

5a5dfdd

fix(plausible): don't use existingSecret for ClickHouse

6656e7c

schnerring force-pushed the upgrade-plausible branch from aefb067 to 6656e7c Compare August 8, 2023 22:49

fix(plausible): upgrade ClickHouse chart dependency

0a3c848

stavros-k reviewed Aug 12, 2023

View reviewed changes

charts/incubator/plausible/Chart.yaml Outdated Show resolved Hide resolved

Update charts/incubator/plausible/Chart.yaml

4c9e1f4

Signed-off-by: Stavros Kois <47820033+stavros-k@users.noreply.github.com>

stavros-k marked this pull request as ready for review August 12, 2023 09:55

stavros-k requested review from PrivatePuffin and a team as code owners August 12, 2023 09:55

Merge branch 'master' into upgrade-plausible

05b5167

stavros-k approved these changes Aug 12, 2023

View reviewed changes

stavros-k enabled auto-merge (squash) August 12, 2023 10:05

stavros-k disabled auto-merge August 12, 2023 10:05

stavros-k merged commit 5b1058a into truecharts:master Aug 12, 2023
14 checks passed

truecharts locked as resolved and limited conversation to collaborators Aug 20, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix(plausible): migrate to new common chart and upgrade to v2 #11186

fix(plausible): migrate to new common chart and upgrade to v2 #11186

schnerring commented Aug 4, 2023 •

edited

Loading

schnerring commented Aug 4, 2023 •

edited

Loading

stavros-k commented Aug 6, 2023

stavros-k commented Aug 6, 2023

schnerring commented Aug 6, 2023

stavros-k commented Aug 6, 2023

schnerring commented Aug 7, 2023

schnerring commented Aug 8, 2023

stavros-k commented Aug 12, 2023

stavros-k commented Aug 12, 2023

schnerring commented Aug 12, 2023

truecharts-admin commented Aug 20, 2023

fix(plausible): migrate to new common chart and upgrade to v2 #11186

fix(plausible): migrate to new common chart and upgrade to v2 #11186

Conversation

schnerring commented Aug 4, 2023 • edited Loading

schnerring commented Aug 4, 2023 • edited Loading

stavros-k commented Aug 6, 2023

stavros-k commented Aug 6, 2023

schnerring commented Aug 6, 2023

stavros-k commented Aug 6, 2023

schnerring commented Aug 7, 2023

schnerring commented Aug 8, 2023

stavros-k commented Aug 12, 2023

stavros-k commented Aug 12, 2023

schnerring commented Aug 12, 2023

truecharts-admin commented Aug 20, 2023

schnerring commented Aug 4, 2023 •

edited

Loading

schnerring commented Aug 4, 2023 •

edited

Loading