operator: Add warning for old schema configuration #11158

btaani · 2023-11-07T13:06:51Z

What this PR does / why we need it:
This PR adds a new warning condition to the LokiStack in case the object storage schema version is older than V13

Which issue(s) this PR fixes:
Fixes LOG-4548

Special notes for your reviewer:

Checklist

Reviewed the CONTRIBUTING.md guide (required)
Documentation added
Tests updated
CHANGELOG.md updated
- If the change is worth mentioning in the release notes, add add-to-release-notes label
Changes that require user attention or interaction to upgrade are documented in docs/sources/setup/upgrade/_index.md
For Helm chart changes bump the Helm chart version in production/helm/loki/Chart.yaml and update production/helm/loki/CHANGELOG.md and production/helm/loki/README.md. Example PR
If the change is deprecating or removing a configuration option, update the deprecated-config.yaml and deleted-config.yaml files respectively in the tools/deprecated-config-checker directory. Example PR

periklis

In general this PR goes into the right direction. A couple of suggestions for better UX:

Keep the Warning condition that the user should upgrade to V13 if not done.
Add another Warning condition that indicates that some schemas are off the retention limit.
Move SchemaStatus into the ObjectStorageSchema struct to indicate which schemas are off the the retention limit.
Add a custom metric that collects Loki stacks requiring a schema upgrade (i.e. lokistack_schema_upgrades_required). We can even write and deploy a custom alert for this: https://github.com/grafana/loki/blob/main/operator/internal/manifests/internal/alerts/prometheus-alerts.yaml)

operator/internal/handlers/lokistack_create_or_update.go

operator/apis/loki/v1/lokistack_types.go

xperimental

Overall a good start 👍
I have not tested this on a cluster yet, only looked at the source so far, but I added some comments already.

I second Peri's hint for an alert. I'm currently assuming that this is how the "insights" feature works in OpenShift.

operator/apis/loki/v1/lokistack_types.go

operator/internal/status/storage.go

operator/internal/status/lokistack.go

operator/internal/status/storage.go

github-actions · 2023-11-20T13:06:19Z

Trivy scan found the following vulnerabilities:

HIGH, Target: docker.io/grafana/loki:main-724a841 (alpine 3.18.4), Type: alpine openssl: Incorrect cipher key and IV length processing in libcrypto3 v3.1.3-r0. Fixed in v3.1.4-r0
HIGH, Target: docker.io/grafana/loki:main-724a841 (alpine 3.18.4), Type: alpine openssl: Incorrect cipher key and IV length processing in libssl3 v3.1.3-r0. Fixed in v3.1.4-r0
\nTo see more details on these vulnerabilities, and how/where to fix them, please run docker build -t grafana/loki:main-724a841 -f cmd/loki/Dockerfile .
trivy i grafana/loki:main-724a841 on your branch. If these were not introduced by your PR, please considering fixing them in via a subsequent PR. Thanks!

operator/apis/loki/v1beta1/lokistack_types.go

operator/apis/loki/v1/lokistack_types.go

xperimental · 2023-12-06T16:23:31Z

operator/CHANGELOG.md

@@ -1,5 +1,6 @@
 ## Main

+- [11158](https://github.com/grafana/loki/pull/11158) **btaani**: operator: Add support to add additional storage schema V13


I think the feature we're adding is the warning nudging the user to upgrade the schema:

Suggested change

- [11158](https://github.com/grafana/loki/pull/11158) **btaani**: operator: Add support to add additional storage schema V13

- [11158](https://github.com/grafana/loki/pull/11158) **btaani**: operator: Add warning for old schema configuration

operator/internal/status/status.go

operator/internal/status/storage.go

6f7e8b8

operator/apis/loki/v1/lokistack_types.go

operator/internal/status/lokistack.go

xperimental

The new warning works nicely 👍

I left a few comments in the PR. I'll defer to @periklis for another pass.

operator/internal/manifests/internal/config/build_test.go

operator/internal/status/lokistack_test.go

operator/internal/status/lokistack.go

periklis

LGTM and smoke testing working fine too. A couple of nits and one remaining question about removed tests.

operator/internal/status/lokistack.go

operator/internal/status/lokistack_test.go

Co-authored-by: Periklis Tsirakidis <periklis@nefeli.eu>

Co-authored-by: Robert Jacob <rojacob@redhat.com> Co-authored-by: Periklis Tsirakidis <periklis@nefeli.eu>

pull-request-size bot added the size/XL label Nov 7, 2023

github-actions bot added the sig/operator label Nov 7, 2023

btaani marked this pull request as ready for review November 7, 2023 13:37

btaani requested review from periklis, xperimental and a team as code owners November 7, 2023 13:37

periklis requested changes Nov 10, 2023

View reviewed changes

operator/internal/handlers/lokistack_create_or_update.go Outdated Show resolved Hide resolved

operator/apis/loki/v1/lokistack_types.go Outdated Show resolved Hide resolved

pull-request-size bot added size/XXL and removed size/XL labels Nov 10, 2023

xperimental reviewed Nov 10, 2023

View reviewed changes

btaani force-pushed the log-4548 branch from a5d0c17 to fc98158 Compare November 21, 2023 11:53

pull-request-size bot added size/XL and removed size/XXL labels Nov 21, 2023

btaani force-pushed the log-4548 branch 5 times, most recently from 947d153 to 3e7382c Compare December 6, 2023 11:16

xperimental reviewed Dec 6, 2023

View reviewed changes

btaani force-pushed the log-4548 branch from eaf1394 to 379ca6c Compare December 12, 2023 23:40

pull-request-size bot added size/XXL and removed size/XL labels Dec 12, 2023

btaani force-pushed the log-4548 branch from 379ca6c to 862d1fe Compare December 14, 2023 13:49

pull-request-size bot added size/XL and removed size/XXL labels Dec 14, 2023

xperimental and others added 4 commits December 14, 2023 23:06

Support setting multiple conditions: cherry picked from f390f37

4b68265

Move handling of degraded state into status package: cherry-picked from

90d5ac8

6f7e8b8

add unit test for adding schema

e6b7688

add warning condition on old schemas

eab8176

btaani added 4 commits December 15, 2023 00:40

add reason to api

7f23e7b

add missing resturn statement in generateCondition

721a700

add unit test

0848741

modify CHANGELOG.md

8228bae

btaani force-pushed the log-4548 branch from 862d1fe to 8228bae Compare December 15, 2023 00:18

btaani changed the title ~~operator: Add support to add additional storage schema V13~~ operator: Add warning for old schema configuration Dec 15, 2023

xperimental reviewed Dec 15, 2023

View reviewed changes

btaani added 3 commits December 15, 2023 17:36

address review comments

9079b4b

remove setting condition status

017d250

minor refactoring

1b5a5e1

xperimental reviewed Dec 15, 2023

View reviewed changes

operator/internal/manifests/internal/config/build_test.go Outdated Show resolved Hide resolved

operator/internal/status/lokistack_test.go Outdated Show resolved Hide resolved

operator/internal/status/lokistack.go Outdated Show resolved Hide resolved

btaani and others added 2 commits December 15, 2023 20:45

another refactor

7e8415e

Merge branch 'main' into log-4548

5cd0c5a

periklis reviewed Dec 18, 2023

View reviewed changes

operator/internal/status/lokistack.go Outdated Show resolved Hide resolved

operator/internal/status/lokistack.go Show resolved Hide resolved

operator/internal/status/lokistack_test.go Show resolved Hide resolved

btaani and others added 5 commits December 18, 2023 11:47

Update operator/internal/status/lokistack.go

459b75a

Co-authored-by: Periklis Tsirakidis <periklis@nefeli.eu>

Update operator/internal/status/lokistack.go

68fac33

Co-authored-by: Periklis Tsirakidis <periklis@nefeli.eu>

undo change

cd7f71f

Merge branch 'main' into log-4548

b46a940

remove extra parameter in function call

9fad771

periklis approved these changes Dec 18, 2023

View reviewed changes

periklis enabled auto-merge (squash) December 18, 2023 14:01

btaani mentioned this pull request Dec 18, 2023

operator: Add a custom metric that collects Lokistacks requiring a schema upgrade #11513

Merged

8 tasks

periklis merged commit 8931f4a into grafana:main Dec 18, 2023
14 checks passed

rhnasc pushed a commit to inloco/loki that referenced this pull request Apr 12, 2024

operator: Add warning for old schema configuration (grafana#11158)

70ff95d

Co-authored-by: Robert Jacob <rojacob@redhat.com> Co-authored-by: Periklis Tsirakidis <periklis@nefeli.eu>

xperimental mentioned this pull request Jul 29, 2024

[release-5.8] Backport metrics and status handling from 5.9 openshift/loki#337

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

operator: Add warning for old schema configuration #11158

operator: Add warning for old schema configuration #11158

btaani commented Nov 7, 2023 •

edited

Loading

periklis left a comment •

edited

Loading

xperimental left a comment

github-actions bot commented Nov 20, 2023 •

edited

Loading

xperimental Dec 6, 2023

xperimental left a comment

periklis left a comment

		@@ -1,5 +1,6 @@
		## Main

		- [11158](https://github.com/grafana/loki/pull/11158) btaani: operator: Add support to add additional storage schema V13

operator: Add warning for old schema configuration #11158

operator: Add warning for old schema configuration #11158

Conversation

btaani commented Nov 7, 2023 • edited Loading

periklis left a comment • edited Loading

Choose a reason for hiding this comment

xperimental left a comment

Choose a reason for hiding this comment

github-actions bot commented Nov 20, 2023 • edited Loading

xperimental Dec 6, 2023

Choose a reason for hiding this comment

xperimental left a comment

Choose a reason for hiding this comment

periklis left a comment

Choose a reason for hiding this comment

btaani commented Nov 7, 2023 •

edited

Loading

periklis left a comment •

edited

Loading

github-actions bot commented Nov 20, 2023 •

edited

Loading