OCPBUGS-17113: Lazy updates for Prometheus #2395

rexagod · 2024-06-26T00:40:26Z

Incorporate openshift/library-go#1575 downstream.

I added CHANGELOG entry for this change.
No user facing changes, so no entry in CHANGELOG was needed.

openshift-ci-robot · 2024-06-26T00:40:31Z

@rexagod: This pull request references Jira Issue OCPBUGS-17113, which is invalid:

expected the bug to target either version "4.17." or "openshift-4.17.", but it targets "4.15.z" instead

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

In response to this:

Incorporate openshift/library-go#1575 downstream.

I added CHANGELOG entry for this change.

No user facing changes, so no entry in CHANGELOG was needed.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

openshift-ci · 2024-06-26T00:41:03Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: rexagod

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [rexagod]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

rexagod · 2024-06-26T00:41:54Z

/jira refresh

openshift-ci-robot · 2024-06-26T00:41:59Z

@rexagod: This pull request references Jira Issue OCPBUGS-17113, which is valid.

3 validation(s) were run on this bug

bug is open, matching expected state (open)
bug target version (4.17.0) matches configured target version for branch (4.17.0)
bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)

Requesting review from QA contact:
/cc @juzhao

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

rexagod · 2024-06-26T16:44:51Z

/retest-required

juzhao · 2024-06-28T03:43:40Z

/retest-required

juzhao · 2024-07-01T02:43:31Z

/retest-required

Incorporate openshift/library-go#1575 downstream. Signed-off-by: Pranshu Srivastava <rexagod@gmail.com>

rexagod · 2024-07-02T23:55:04Z

/test e2e-aws-ovn-techpreview

juzhao · 2024-07-04T08:15:08Z

/label qe-approved

machine424

Thanks!
I left some comments.

machine424 · 2024-07-08T08:46:26Z

pkg/client/client.go

+		_, err = c.dclient.
+			Resource(prometheusGVR).
+			Namespace(structuredRequiredPrometheus.GetNamespace()).
+			Create(ctx, unstructuredRequiredPrometheus, metav1.CreateOptions{})


I think we can delegate the creation to ApplyUnstructuredResourceImproved as it already does that.

I see. I tried to preserve the earlier behavior, albeit at the cost of no cache updates during creation, but this makes sense. Done!

machine424 · 2024-07-08T09:13:02Z

pkg/client/client.go

+	return &didUpdate, nil
+}
+
+func prometheusDefaultingFunc(unstructuredPrometheus *unstructured.Unstructured) {


I thought that after our discussion here openshift/library-go#1575 (comment), we "agreed" to give the "cache approach" a try for now, as it's simpler (less code/less maintenance), and with the cache, IIUC, I don't think we need to care about the defaults (if the wanted config is the one stored in the cache which mean the one deployed, skip the update).
Any reasons behind the change?

Referring to the same thread: openshift/library-go#1575 (comment), we do this in addition to the cache approach since that doesn't cache spec fields, in case of a metadata cache miss (same as what Trevor's patch targets in ef5f348#diff-2895f30116602d8fe6b0545c186d4f81247fa4eaab049dab174c6c91c0036e08R639-R667).

machine424 · 2024-07-08T09:15:03Z

pkg/client/client_test.go

+	c := &Client{}
+	c.dclient = dynamicFake.NewSimpleDynamicClient(runtime.NewScheme(), initialPrometheus.DeepCopy())
+	c.eventRecorder = events.NewInMemoryRecorder("create-or-update-prometheus-test")
+	c.resourceCache = resourceapply.NewResourceCache()


Note that the code seems to be using the "comparison approach" (I don't see any cache initialized there), but here we're testing the "cache approach", am I missing sth?

Both of these are in use (PTAL my comment https://github.com/openshift/cluster-monitoring-operator/pull/2395/files#r1696698968). Here, we initialize the cache, and down below we use the defaulting approach to reject default spec updates (shouldUpdate is false for this case).

pkg/client/client_test.go

machine424 · 2024-07-08T09:26:02Z

pkg/client/client.go

 	}
+	unstructuredRequiredPrometheusMetadataLabels := unstructuredRequiredPrometheus.GetLabels()


these may be worth a comment.
Just to be sure I understand, is this done to avoid

cluster-monitoring-operator/Documentation/development.md

Line 52 in 5341b80

#### NOTE:

?

That's done to preserve the past behavior: PTAL rexagod@529b005#diff-2895f30116602d8fe6b0545c186d4f81247fa4eaab049dab174c6c91c0036e08R301.

machine424 · 2024-07-08T09:31:07Z

pkg/client/client.go

-func (c *Client) CreateOrUpdatePrometheus(ctx context.Context, p *monv1.Prometheus) error {
-	pclient := c.mclient.MonitoringV1().Prometheuses(p.GetNamespace())
-	existing, err := pclient.Get(ctx, p.GetName(), metav1.GetOptions{})
+func (c *Client) CreateOrUpdatePrometheus(ctx context.Context, structuredRequiredPrometheus *monv1.Prometheus) (*bool, error) {


why do we need to return a pointer instead of just a bool here?
I don't think tests should rely on this, I think we should retrieve whether the object was updated from the API/client.

I looked at the usages and there doesn't seem to be a need for three states. Fixed! 👍🏼

openshift-ci · 2024-08-05T02:49:50Z

@rexagod: all tests passed!

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

rexagod · 2024-08-05T07:47:26Z

pkg/client/client_test.go

+			shouldUpdate: true,
 		},
 		{
 			description: "apply no value to the existing prometheus",
 			prometheus: &monv1.Prometheus{
 				TypeMeta:   initialPrometheus.TypeMeta,
 				ObjectMeta: initialPrometheus.ObjectMeta,
 			},
+			shouldUpdate: true,


With the no defaulting approach, there won't be a difference between default and non-default applications, as both will trigger updates as compared to just the latter earlier, at the benefit of not maintaining hardcoded default values in CMO. LMK if I should revert this or if this sounds good.

openshift-ci bot requested review from jan--f and slashpai June 26, 2024 00:41

openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jun 26, 2024

openshift-ci-robot added jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. and removed jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels Jun 26, 2024

openshift-ci bot requested a review from juzhao June 26, 2024 00:42

rexagod marked this pull request as draft June 26, 2024 06:51

openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jun 26, 2024

rexagod marked this pull request as ready for review June 26, 2024 06:52

openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jun 26, 2024

openshift-ci bot requested review from machine424 and marioferh June 26, 2024 06:53

rexagod force-pushed the mon-17113-lib-go-1575 branch 5 times, most recently from 484f320 to d394434 Compare June 26, 2024 13:53

rexagod force-pushed the mon-17113-lib-go-1575 branch from d394434 to 29846c5 Compare June 26, 2024 20:04

OCPBUGS-17113: Lazy updates for Prometheus

564d684

Incorporate openshift/library-go#1575 downstream. Signed-off-by: Pranshu Srivastava <rexagod@gmail.com>

rexagod force-pushed the mon-17113-lib-go-1575 branch from 29846c5 to 564d684 Compare July 2, 2024 20:39

openshift-ci bot added the qe-approved Signifies that QE has signed off on this PR label Jul 4, 2024

machine424 reviewed Jul 8, 2024

View reviewed changes

rexagod added 2 commits July 31, 2024 13:07

fixup! OCPBUGS-17113: Lazy updates for Prometheus

efb39df

fixup! fixup! OCPBUGS-17113: Lazy updates for Prometheus

4e75b4a

rexagod force-pushed the mon-17113-lib-go-1575 branch from 5b0d83b to 4e75b4a Compare August 5, 2024 00:12

rexagod commented Aug 5, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

OCPBUGS-17113: Lazy updates for Prometheus #2395

OCPBUGS-17113: Lazy updates for Prometheus #2395

rexagod commented Jun 26, 2024

openshift-ci-robot commented Jun 26, 2024

openshift-ci bot commented Jun 26, 2024

rexagod commented Jun 26, 2024

openshift-ci-robot commented Jun 26, 2024

rexagod commented Jun 26, 2024

juzhao commented Jun 28, 2024

juzhao commented Jul 1, 2024

rexagod commented Jul 2, 2024

juzhao commented Jul 4, 2024

machine424 left a comment

machine424 Jul 8, 2024

rexagod Jul 30, 2024

machine424 Jul 8, 2024

rexagod Jul 30, 2024

machine424 Jul 8, 2024

rexagod Jul 30, 2024

machine424 Jul 8, 2024

rexagod Jul 30, 2024

machine424 Jul 8, 2024

rexagod Jul 30, 2024

openshift-ci bot commented Aug 5, 2024

rexagod Aug 5, 2024

		}
		unstructuredRequiredPrometheusMetadataLabels := unstructuredRequiredPrometheus.GetLabels()

OCPBUGS-17113: Lazy updates for Prometheus #2395

Are you sure you want to change the base?

OCPBUGS-17113: Lazy updates for Prometheus #2395

Conversation

rexagod commented Jun 26, 2024

openshift-ci-robot commented Jun 26, 2024

openshift-ci bot commented Jun 26, 2024

rexagod commented Jun 26, 2024

openshift-ci-robot commented Jun 26, 2024

rexagod commented Jun 26, 2024

juzhao commented Jun 28, 2024

juzhao commented Jul 1, 2024

rexagod commented Jul 2, 2024

juzhao commented Jul 4, 2024

machine424 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

openshift-ci bot commented Aug 5, 2024

Choose a reason for hiding this comment