Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[release-4.16] OCPBUGS-38021: Integration of the OpenStack CRs into the insights-operator #974

Merged

Conversation

slawqo
Copy link
Contributor

@slawqo slawqo commented Aug 5, 2024

Control Plan of the Red Hat OpenStack starting from the version 18 (RHOSO 18) will be deployed using OpenShift operators as a set of OpenShift applications. Compute nodes will be deployed still as RHEL nodes and will be outside of the OpenShift cluster but all will be mangaged by the Dataplane-operator and only interface for OpenStack operators will be though the CRs:

  • OpenStackControlPlane CR for the control plane,
  • OpenStackDataplaneNodeSet and OpenStackDataplaneDeployment CRs for the compute nodes.

This PR adds those OpenStack CRs to the insights-operator so that whole configuration of the OpenStack control plane may be gathered by the insights-operator and send to analysis to the Insights server.

Categories

  • Bugfix
  • Data Enhancement
  • Feature
  • Backporting
  • Others (CI, Infrastructure, Documentation)

Sample Archive

  • docs/insights-archive-sample/customresources/core.openstack.org/openstackcontrolplanes/openstack/openstack-galera-network-isolation.json
  • docs/insights-archive-sample/customresources/dataplane.openstack.org/openstackdataplanedeployments/openstack/edpm-deployment.json
  • docs/insights-archive-sample/customresources/dataplane.openstack.org/openstackdataplanenodesets/openstack/openstack-edpm.json

Unit Tests

  • pkg/gatherers/clusterconfig/gather_openstack_controlplanes_test.go
  • pkg/gatherers/clusterconfig/gather_openstack_dataplane_deployments_test.go
  • pkg/gatherers/clusterconfig/gather_openstack_dataplane_node_sets_test.go

Privacy

Anonymization functions are added and used while collecting OpenStack CRs. Anonymized data:

  • OpenstackControlplanes
    • Spec.dns.template.options
  • OpenstackDataplaneNodeSets
    • ansibleUser
    • edpm_sshd_allowed_ranges
    • dnsClusterAddresses
    • Status.allHostnames

Changelog

No

Breaking Changes

No

References

https://issues.redhat.com/browse/OSPRH-5904

Additional Info

To test what data are collected by the insights-operator with those changes, OpenStack has to be deployed on top of the OpenShift cluster. The easiest way to do this is to use CRC OpenShift and install_yamls tool. Please follow the guide to deploy OpenStack Control Plane and OpenStackDataPlane CRs

Control Plan of the Red Hat OpenStack starting from the version 18
(RHOSO 18) will be deployed using OpenShift operators as a set of
OpenShift applications.
Compute nodes will be deployed still as RHEL nodes and will be outside
of the OpenShift cluster but all will be mangaged by the
Dataplane-operator and only interface for OpenStack operators will be
though the CRs:

- OpenStackControlPlane CR for the control plane,
- OpenStackDataplaneNodeSet and OpenStackDataplaneDeployment CRs for the
  compute nodes.

This patch adds OpenStackControlPlane CR to the insights-operator so that
whole configuration of the OpenStack control plane may be gathered by the
insights-operator and send to analysis to the Insights server.

Co-Authored-By: Yatin Karel <ykarel@redhat.com>

Conflicts:
    docs/gathered-data.md
    pkg/gatherers/clusterconfig/clusterconfig_gatherer.go

Related: #OSPRH-5904
(cherry picked from commit 999cbf5)
Some fields in the collected OpenStackControlPlane CRs should be
anonymized. This is for example field 'options' from the 'dns'
service spec.

Related: #OSPRH-5904
(cherry picked from commit d725c1f)
For now it will drop only
'metadata/annotations/kubectl.kubernetes.io/last-applied-configuration'
but other fields can be added to that list too.

(cherry picked from commit c697f3d)
Control Plan of the Red Hat OpenStack starting from the version 18
(RHOSO 18) will be deployed using OpenShift operators as a set of
OpenShift applications.
Compute nodes will be deployed still as RHEL nodes and will be outside
of the OpenShift cluster but all will be mangaged by the
Dataplane-operator and only interface for OpenStack operators will be
though the CRs:

- OpenStackControlPlane CR for the control plane,
- OpenStackDataplaneNodeSet and OpenStackDataplaneDeployment CRs for the
  compute nodes.

This patch adds OpenStackDataplaneNodeSet CR to the insights-operator so that
whole configuration of the OpenStack control plane may be gathered by the
insights-operator and send to analysis to the Insights server.

Co-Authored-By: Yatin Karel <ykarel@redhat.com>

Related: #OSPRH-5904
(cherry picked from commit 809ea80)
Control Plan of the Red Hat OpenStack starting from the version 18
(RHOSO 18) will be deployed using OpenShift operators as a set of
OpenShift applications.
Compute nodes will be deployed still as RHEL nodes and will be outside
of the OpenShift cluster but all will be mangaged by the
Dataplane-operator and only interface for OpenStack operators will be
though the CRs:

- OpenStackControlPlane CR for the control plane,
- OpenStackDataplaneNodeSet and OpenStackDataplaneDeployment CRs for the
  compute nodes.

This patch adds OpenStackDataplaneDeployment CR to the insights-operator so that
whole configuration of the OpenStack control plane may be gathered by the
insights-operator and send to analysis to the Insights server.

Co-Authored-By: Yatin Karel <ykarel@redhat.com>

Related: #OSPRH-5904
(cherry picked from commit 52659ff)
Related: #OSPRH-5904
(cherry picked from commit c6feae6)
Red Hat's openstack-operator in addition to the OpenStackControlPlane
and OpenStackDataPlane CRs uses also additonal CR 'openstackversions'
which reports version of the openstack(s) deployed in the OCP cluster.
This patch adds support for gathering also this CRs from the cluster if
they exists.

Related: #OSPRH-5904
(cherry picked from commit d0d5cc7)
@openshift-ci openshift-ci bot requested review from ncaak and rhrmo August 5, 2024 12:30
@slawqo slawqo changed the title Osprh 5904 release 4.16 [release-4.16] OSPRH-5904: Integration of the OpenStack CRs into the insights-operator Aug 5, 2024
@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Aug 5, 2024
@openshift-ci-robot
Copy link
Contributor

openshift-ci-robot commented Aug 5, 2024

@slawqo: This pull request references OSPRH-5904 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.16.z" version, but no target version was set.

In response to this:

Control Plan of the Red Hat OpenStack starting from the version 18 (RHOSO 18) will be deployed using OpenShift operators as a set of OpenShift applications. Compute nodes will be deployed still as RHEL nodes and will be outside of the OpenShift cluster but all will be mangaged by the Dataplane-operator and only interface for OpenStack operators will be though the CRs:

  • OpenStackControlPlane CR for the control plane,
  • OpenStackDataplaneNodeSet and OpenStackDataplaneDeployment CRs for the compute nodes.

This PR adds those OpenStack CRs to the insights-operator so that whole configuration of the OpenStack control plane may be gathered by the insights-operator and send to analysis to the Insights server.

Categories

  • Bugfix
  • Data Enhancement
  • Feature
  • Backporting
  • Others (CI, Infrastructure, Documentation)

Sample Archive

  • docs/insights-archive-sample/customresources/core.openstack.org/openstackcontrolplanes/openstack/openstack-galera-network-isolation.json
  • docs/insights-archive-sample/customresources/dataplane.openstack.org/openstackdataplanedeployments/openstack/edpm-deployment.json
  • docs/insights-archive-sample/customresources/dataplane.openstack.org/openstackdataplanenodesets/openstack/openstack-edpm.json

Unit Tests

  • pkg/gatherers/clusterconfig/gather_openstack_controlplanes_test.go
  • pkg/gatherers/clusterconfig/gather_openstack_dataplane_deployments_test.go
  • pkg/gatherers/clusterconfig/gather_openstack_dataplane_node_sets_test.go

Privacy

Anonymization functions are added and used while collecting OpenStack CRs. Anonymized data:

  • OpenstackControlplanes
  • Spec.dns.template.options
  • OpenstackDataplaneNodeSets
  • ansibleUser
  • edpm_sshd_allowed_ranges
  • dnsClusterAddresses
  • Status.allHostnames

Changelog

No

Breaking Changes

No

References

https://issues.redhat.com/browse/OSPRH-5904

Additional Info

To test what data are collected by the insights-operator with those changes, OpenStack has to be deployed on top of the OpenShift cluster. The easiest way to do this is to use CRC OpenShift and install_yamls tool. Please follow the guide to deploy OpenStack Control Plane and OpenStackDataPlane CRs

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@tremes
Copy link
Contributor

tremes commented Aug 6, 2024

@slawqo How far back do you want to backport this plese? I didn't know (or forget) that you wanted to backport it. I am afraid that you will need an OCPBUGS JIRA to backport.

@slawqo
Copy link
Contributor Author

slawqo commented Aug 6, 2024

@slawqo How far back do you want to backport this plese? I didn't know (or forget) that you wanted to backport it. I am afraid that you will need an OCPBUGS JIRA to backport.

We would like to have it in 4.16 release as this one is our target for RHOSO 18.0 GA release. I will check today this project on Jira and will open issue there.

@tremes
Copy link
Contributor

tremes commented Aug 6, 2024

/test e2e-gcp-ovn-techpreview
/approve

Copy link

openshift-ci bot commented Aug 6, 2024

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: slawqo, tremes

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Aug 6, 2024
@tremes
Copy link
Contributor

tremes commented Aug 6, 2024

/label backport-risk-assessed

@openshift-ci openshift-ci bot added the backport-risk-assessed Indicates a PR to a release branch has been evaluated and considered safe to accept. label Aug 6, 2024
@tremes tremes changed the title [release-4.16] OSPRH-5904: Integration of the OpenStack CRs into the insights-operator [release-4.16] OCPBUGS-38021: Integration of the OpenStack CRs into the insights-operator Aug 6, 2024
@openshift-ci-robot openshift-ci-robot added the jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. label Aug 6, 2024
@openshift-ci-robot
Copy link
Contributor

@slawqo: This pull request references Jira Issue OCPBUGS-38021, which is invalid:

  • release note text must be set and not match the template OR release note type must be set to "Release Note Not Required"
  • expected dependent Jira Issue OCPBUGS-38022 to be in one of the following states: VERIFIED, RELEASE PENDING, CLOSED (ERRATA), CLOSED (CURRENT RELEASE), CLOSED (DONE), CLOSED (DONE-ERRATA), but it is ON_QA instead

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

In response to this:

Control Plan of the Red Hat OpenStack starting from the version 18 (RHOSO 18) will be deployed using OpenShift operators as a set of OpenShift applications. Compute nodes will be deployed still as RHEL nodes and will be outside of the OpenShift cluster but all will be mangaged by the Dataplane-operator and only interface for OpenStack operators will be though the CRs:

  • OpenStackControlPlane CR for the control plane,
  • OpenStackDataplaneNodeSet and OpenStackDataplaneDeployment CRs for the compute nodes.

This PR adds those OpenStack CRs to the insights-operator so that whole configuration of the OpenStack control plane may be gathered by the insights-operator and send to analysis to the Insights server.

Categories

  • Bugfix
  • Data Enhancement
  • Feature
  • Backporting
  • Others (CI, Infrastructure, Documentation)

Sample Archive

  • docs/insights-archive-sample/customresources/core.openstack.org/openstackcontrolplanes/openstack/openstack-galera-network-isolation.json
  • docs/insights-archive-sample/customresources/dataplane.openstack.org/openstackdataplanedeployments/openstack/edpm-deployment.json
  • docs/insights-archive-sample/customresources/dataplane.openstack.org/openstackdataplanenodesets/openstack/openstack-edpm.json

Unit Tests

  • pkg/gatherers/clusterconfig/gather_openstack_controlplanes_test.go
  • pkg/gatherers/clusterconfig/gather_openstack_dataplane_deployments_test.go
  • pkg/gatherers/clusterconfig/gather_openstack_dataplane_node_sets_test.go

Privacy

Anonymization functions are added and used while collecting OpenStack CRs. Anonymized data:

  • OpenstackControlplanes
  • Spec.dns.template.options
  • OpenstackDataplaneNodeSets
  • ansibleUser
  • edpm_sshd_allowed_ranges
  • dnsClusterAddresses
  • Status.allHostnames

Changelog

No

Breaking Changes

No

References

https://issues.redhat.com/browse/OSPRH-5904

Additional Info

To test what data are collected by the insights-operator with those changes, OpenStack has to be deployed on top of the OpenShift cluster. The easiest way to do this is to use CRC OpenShift and install_yamls tool. Please follow the guide to deploy OpenStack Control Plane and OpenStackDataPlane CRs

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@slawqo
Copy link
Contributor Author

slawqo commented Aug 9, 2024

/jira refresh

@openshift-ci-robot
Copy link
Contributor

@slawqo: This pull request references Jira Issue OCPBUGS-38021, which is invalid:

  • expected dependent Jira Issue OCPBUGS-38022 to be in one of the following states: VERIFIED, RELEASE PENDING, CLOSED (ERRATA), CLOSED (CURRENT RELEASE), CLOSED (DONE), CLOSED (DONE-ERRATA), but it is ON_QA instead

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@karelyatin
Copy link

/lgtm

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Aug 13, 2024
@tremes
Copy link
Contributor

tremes commented Aug 30, 2024

/jira refresh

@openshift-ci-robot openshift-ci-robot added jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. and removed jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels Aug 30, 2024
@openshift-ci-robot
Copy link
Contributor

@tremes: This pull request references Jira Issue OCPBUGS-38021, which is valid. The bug has been moved to the POST state.

7 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target version (4.16.z) matches configured target version for branch (4.16.z)
  • bug is in the state New, which is one of the valid states (NEW, ASSIGNED, POST)
  • release note text is set and does not match the template
  • dependent bug Jira Issue OCPBUGS-38022 is in the state Verified, which is one of the valid states (VERIFIED, RELEASE PENDING, CLOSED (ERRATA), CLOSED (CURRENT RELEASE), CLOSED (DONE), CLOSED (DONE-ERRATA))
  • dependent Jira Issue OCPBUGS-38022 targets the "4.17.0" version, which is one of the valid target versions: 4.17.0
  • bug has dependents

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@BaiyangZhou
Copy link

/label cherry-pick-approved

@openshift-ci openshift-ci bot added the cherry-pick-approved Indicates a cherry-pick PR into a release branch has been approved by the release branch manager. label Sep 2, 2024
@openshift-ci-robot
Copy link
Contributor

/retest-required

Remaining retests: 0 against base HEAD 3a723d3 and 2 for PR HEAD bb6452f in total

@tremes
Copy link
Contributor

tremes commented Sep 2, 2024

/test e2e-agnostic-upgrade

@tremes
Copy link
Contributor

tremes commented Sep 3, 2024

/retest

@openshift-ci-robot
Copy link
Contributor

/retest-required

Remaining retests: 0 against base HEAD 3a723d3 and 2 for PR HEAD bb6452f in total

2 similar comments
@openshift-ci-robot
Copy link
Contributor

/retest-required

Remaining retests: 0 against base HEAD 3a723d3 and 2 for PR HEAD bb6452f in total

@openshift-ci-robot
Copy link
Contributor

/retest-required

Remaining retests: 0 against base HEAD 3a723d3 and 2 for PR HEAD bb6452f in total

@karelyatin
Copy link

A test fails:-
ERROR tests/data_uploaded/test_sca.py::test_periodic_check_sca - TypeError: NamespacedResource.delete() got an unexpected keyword argument 'wait'

Seems real issue and iiuc ccx-testlib version needs to be updated for 4.16, @tremes @BaiyangZhou can you check/clear this

@tremes
Copy link
Contributor

tremes commented Sep 4, 2024

/test insights-operator-e2e-tests

@openshift-ci-robot
Copy link
Contributor

/retest-required

Remaining retests: 0 against base HEAD 3a723d3 and 2 for PR HEAD bb6452f in total

@tremes
Copy link
Contributor

tremes commented Sep 4, 2024

Test failures are not related to these changes.
/override ci/prow/insights-operator-e2e-tests

Copy link

openshift-ci bot commented Sep 4, 2024

@tremes: Overrode contexts on behalf of tremes: ci/prow/insights-operator-e2e-tests

In response to this:

Test failures are not related to these changes.
/override ci/prow/insights-operator-e2e-tests

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

Copy link

openshift-ci bot commented Sep 4, 2024

@slawqo: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/insights-operator-e2e-tests bb6452f link true /test insights-operator-e2e-tests

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@openshift-merge-bot openshift-merge-bot bot merged commit 0c42c04 into openshift:release-4.16 Sep 4, 2024
9 checks passed
@openshift-ci-robot
Copy link
Contributor

@slawqo: Jira Issue OCPBUGS-38021: Some pull requests linked via external trackers have merged:

The following pull requests linked via external trackers have not merged:

These pull request must merge or be unlinked from the Jira bug in order for it to move to the next state. Once unlinked, request a bug refresh with /jira refresh.

Jira Issue OCPBUGS-38021 has not been moved to the MODIFIED state.

In response to this:

Control Plan of the Red Hat OpenStack starting from the version 18 (RHOSO 18) will be deployed using OpenShift operators as a set of OpenShift applications. Compute nodes will be deployed still as RHEL nodes and will be outside of the OpenShift cluster but all will be mangaged by the Dataplane-operator and only interface for OpenStack operators will be though the CRs:

  • OpenStackControlPlane CR for the control plane,
  • OpenStackDataplaneNodeSet and OpenStackDataplaneDeployment CRs for the compute nodes.

This PR adds those OpenStack CRs to the insights-operator so that whole configuration of the OpenStack control plane may be gathered by the insights-operator and send to analysis to the Insights server.

Categories

  • Bugfix
  • Data Enhancement
  • Feature
  • Backporting
  • Others (CI, Infrastructure, Documentation)

Sample Archive

  • docs/insights-archive-sample/customresources/core.openstack.org/openstackcontrolplanes/openstack/openstack-galera-network-isolation.json
  • docs/insights-archive-sample/customresources/dataplane.openstack.org/openstackdataplanedeployments/openstack/edpm-deployment.json
  • docs/insights-archive-sample/customresources/dataplane.openstack.org/openstackdataplanenodesets/openstack/openstack-edpm.json

Unit Tests

  • pkg/gatherers/clusterconfig/gather_openstack_controlplanes_test.go
  • pkg/gatherers/clusterconfig/gather_openstack_dataplane_deployments_test.go
  • pkg/gatherers/clusterconfig/gather_openstack_dataplane_node_sets_test.go

Privacy

Anonymization functions are added and used while collecting OpenStack CRs. Anonymized data:

  • OpenstackControlplanes
  • Spec.dns.template.options
  • OpenstackDataplaneNodeSets
  • ansibleUser
  • edpm_sshd_allowed_ranges
  • dnsClusterAddresses
  • Status.allHostnames

Changelog

No

Breaking Changes

No

References

https://issues.redhat.com/browse/OSPRH-5904

Additional Info

To test what data are collected by the insights-operator with those changes, OpenStack has to be deployed on top of the OpenShift cluster. The easiest way to do this is to use CRC OpenShift and install_yamls tool. Please follow the guide to deploy OpenStack Control Plane and OpenStackDataPlane CRs

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-bot
Copy link
Contributor

[ART PR BUILD NOTIFIER]

Distgit: ose-insights-operator
This PR has been included in build ose-insights-operator-container-v4.16.0-202409041137.p0.g0c42c04.assembly.stream.el9.
All builds following this will include this PR.

This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. backport-risk-assessed Indicates a PR to a release branch has been evaluated and considered safe to accept. cherry-pick-approved Indicates a cherry-pick PR into a release branch has been approved by the release branch manager. jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants