Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OCPBUGS-17113: Lazy updates for Prometheus #2395

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

rexagod
Copy link
Member

@rexagod rexagod commented Jun 26, 2024

Incorporate openshift/library-go#1575 downstream.

  • I added CHANGELOG entry for this change.
  • No user facing changes, so no entry in CHANGELOG was needed.

@openshift-ci-robot openshift-ci-robot added jira/severity-moderate Referenced Jira bug's severity is moderate for the branch this PR is targeting. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels Jun 26, 2024
@openshift-ci-robot
Copy link
Contributor

@rexagod: This pull request references Jira Issue OCPBUGS-17113, which is invalid:

  • expected the bug to target either version "4.17." or "openshift-4.17.", but it targets "4.15.z" instead

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

In response to this:

Incorporate openshift/library-go#1575 downstream.

  • I added CHANGELOG entry for this change.
  • No user facing changes, so no entry in CHANGELOG was needed.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci openshift-ci bot requested review from jan--f and slashpai June 26, 2024 00:41
@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jun 26, 2024
@rexagod
Copy link
Member Author

rexagod commented Jun 26, 2024

/jira refresh

@openshift-ci-robot openshift-ci-robot added jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. and removed jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels Jun 26, 2024
@openshift-ci-robot
Copy link
Contributor

@rexagod: This pull request references Jira Issue OCPBUGS-17113, which is valid.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target version (4.17.0) matches configured target version for branch (4.17.0)
  • bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)

Requesting review from QA contact:
/cc @juzhao

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci openshift-ci bot requested a review from juzhao June 26, 2024 00:42
@rexagod rexagod marked this pull request as draft June 26, 2024 06:51
@openshift-ci openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jun 26, 2024
@rexagod rexagod marked this pull request as ready for review June 26, 2024 06:52
@openshift-ci openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jun 26, 2024
@openshift-ci openshift-ci bot requested review from machine424 and marioferh June 26, 2024 06:53
@rexagod rexagod force-pushed the mon-17113-lib-go-1575 branch 5 times, most recently from 484f320 to d394434 Compare June 26, 2024 13:53
@rexagod
Copy link
Member Author

rexagod commented Jun 26, 2024

/retest-required

@rexagod rexagod force-pushed the mon-17113-lib-go-1575 branch from d394434 to 29846c5 Compare June 26, 2024 20:04
@juzhao
Copy link
Contributor

juzhao commented Jun 28, 2024

/retest-required

1 similar comment
@juzhao
Copy link
Contributor

juzhao commented Jul 1, 2024

/retest-required

@rexagod rexagod force-pushed the mon-17113-lib-go-1575 branch from 29846c5 to 564d684 Compare July 2, 2024 20:39
@rexagod
Copy link
Member Author

rexagod commented Jul 2, 2024

/test e2e-aws-ovn-techpreview

@juzhao
Copy link
Contributor

juzhao commented Jul 4, 2024

/label qe-approved

Comment on lines 1872 to 1882
shouldUpdate: true,
},
{
description: "apply no value to the existing prometheus",
prometheus: &monv1.Prometheus{
TypeMeta: initialPrometheus.TypeMeta,
ObjectMeta: initialPrometheus.ObjectMeta,
},
shouldUpdate: true,
Copy link
Member Author

@rexagod rexagod Aug 5, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With the no defaulting approach, there won't be a difference between default and non-default applications, as both will trigger updates as compared to just the latter earlier, at the benefit of not maintaining hardcoded default values in CMO.

@rexagod
Copy link
Member Author

rexagod commented Sep 11, 2024

/test versions

1 similar comment
@rexagod
Copy link
Member Author

rexagod commented Sep 18, 2024

/test versions

@rexagod rexagod force-pushed the mon-17113-lib-go-1575 branch 4 times, most recently from 80dc2e3 to 30dc110 Compare September 19, 2024 21:08
@rexagod
Copy link
Member Author

rexagod commented Sep 23, 2024

/test e2e-aws-ovn-single-node

Flaking.

@rexagod rexagod force-pushed the mon-17113-lib-go-1575 branch from 30dc110 to 70fc333 Compare October 7, 2024 06:30
@rexagod
Copy link
Member Author

rexagod commented Oct 9, 2024

Blocked by openshift/library-go#1823.

@rexagod
Copy link
Member Author

rexagod commented Oct 10, 2024

EDIT: Will update this PR once openshift/library-go#1833 is in.

@openshift-bot
Copy link
Contributor

Issues go stale after 90d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle stale

@openshift-ci openshift-ci bot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Jan 14, 2025
@openshift-merge-robot openshift-merge-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jan 14, 2025
@rexagod
Copy link
Member Author

rexagod commented Jan 15, 2025

/remove-lifecycle stale

@openshift-ci openshift-ci bot removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Jan 15, 2025
@rexagod
Copy link
Member Author

rexagod commented Jan 29, 2025

openshift/library-go#1833 (comment) is closed now, moving forward as there are no blockers.

@rexagod rexagod force-pushed the mon-17113-lib-go-1575 branch from 70fc333 to 37abec5 Compare January 29, 2025 16:14
@openshift-merge-robot openshift-merge-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jan 29, 2025
Copy link
Contributor

openshift-ci bot commented Jan 29, 2025

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: rexagod

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Incorporate openshift/library-go#1575
downstream.

Signed-off-by: Pranshu Srivastava <rexagod@gmail.com>
@rexagod rexagod force-pushed the mon-17113-lib-go-1575 branch from 37abec5 to 39addc0 Compare January 29, 2025 18:43
@rexagod
Copy link
Member Author

rexagod commented Jan 30, 2025

/test e2e-aws-ovn-single-node
/test e2e-aws-ovn-techpreview
/test okd-scos-e2e-aws-ovn

Copy link
Contributor

openshift-ci bot commented Jan 30, 2025

@rexagod: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/e2e-aws-ovn-techpreview 39addc0 link true /test e2e-aws-ovn-techpreview
ci/prow/versions 39addc0 link false /test versions
ci/prow/e2e-aws-ovn-single-node 39addc0 link false /test e2e-aws-ovn-single-node
ci/prow/okd-scos-e2e-aws-ovn 39addc0 link false /test okd-scos-e2e-aws-ovn

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@rexagod
Copy link
Member Author

rexagod commented Feb 3, 2025

Prow:

: [sig-arch] events should not repeat pathologically for ns/openshift-monitoring 0s
{  1 events happened too frequently

event happened 44 times, something is wrong: namespace/openshift-monitoring deployment/cluster-monitoring-operator hmsg/10ad659236 - reason/PrometheusUpdated Updated Prometheus.monitoring.coreos.com/k8s -n openshift-monitoring because it changed (10:04:01Z) result=reject }

Must-gather:

Time | Namespace | Component | RelatedObject | Reason | Message

10:19:01 (x52) | openshift-monitoring | cluster-monitoring-operator | cluster-monitoring-operator | PrometheusUpdated | Updated Prometheus.monitoring.coreos.com/k8s -n openshift-monitoring because it changed

CMO:

I0130 08:49:01.224480       1 event.go:377] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-monitoring", Name:"cluster-monitoring-operator", UID:"12db9f3b-5ff0-4a42-8c0f-6e7d201e46c9", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'PrometheusUpdated' Updated Prometheus.monitoring.coreos.com/k8s -n openshift-monitoring because it changed

The update logic introduced here updates twice on each call, causing a loop, I'll refactor that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. jira/severity-moderate Referenced Jira bug's severity is moderate for the branch this PR is targeting. jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. qe-approved Signifies that QE has signed off on this PR
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants