Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NETOBSERV-1648: Make high cardinality check more visible #650

Merged
merged 6 commits into from
May 22, 2024

Conversation

jotak
Copy link
Member

@jotak jotak commented May 21, 2024

Description

  • Add Status/Conditions to FlowMetrics CRD
  • Add a Ready status
  • Add a Cardinality status
  • Add both of them as printColumns

Dependencies

n/a

Checklist

If you are not familiar with our processes or don't know what to answer in the list below, let us know in a comment: the maintainers will take care of that.

  • Is this PR backed with a JIRA ticket? If so, make sure it is written as a title prefix (in general, PRs affecting the NetObserv/Network Observability product should be backed with a JIRA ticket - especially if they bring user facing changes).
  • Does this PR require product documentation?
    • If so, make sure the JIRA epic is labelled with "documentation" and provides a description relevant for doc writers, such as use cases or scenarios. Any required step to activate or configure the feature should be documented there, such as new CRD knobs.
  • Does this PR require a product release notes entry?
    • If so, fill in "Release Note Text" in the JIRA.
  • Is there anything else the QE team should know before testing? E.g: configuration changes, environment setup, etc.
    • If so, make sure it is described in the JIRA ticket.
  • QE requirements (check 1 from the list):
    • Standard QE validation, with pre-merge tests unless stated otherwise.
    • Regression tests only (e.g. refactoring with no user-facing change).
    • No QE (e.g. trivial change with high reviewer's confidence, or per agreement with the QE team).

- Add Status/Conditions to FlowMetrics CRD
- Add a Ready status
- Add a Cardinality status
- Add both of them as printColumns
@openshift-ci-robot
Copy link
Collaborator

openshift-ci-robot commented May 21, 2024

@jotak: This pull request references NETOBSERV-1648 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.17.0" version, but no target version was set.

In response to this:

Description

  • Add Status/Conditions to FlowMetrics CRD
  • Add a Ready status
  • Add a Cardinality status
  • Add both of them as printColumns

Dependencies

n/a

Checklist

If you are not familiar with our processes or don't know what to answer in the list below, let us know in a comment: the maintainers will take care of that.

  • Is this PR backed with a JIRA ticket? If so, make sure it is written as a title prefix (in general, PRs affecting the NetObserv/Network Observability product should be backed with a JIRA ticket - especially if they bring user facing changes).
  • Does this PR require product documentation?
  • If so, make sure the JIRA epic is labelled with "documentation" and provides a description relevant for doc writers, such as use cases or scenarios. Any required step to activate or configure the feature should be documented there, such as new CRD knobs.
  • Does this PR require a product release notes entry?
  • If so, fill in "Release Note Text" in the JIRA.
  • Is there anything else the QE team should know before testing? E.g: configuration changes, environment setup, etc.
  • If so, make sure it is described in the JIRA ticket.
  • QE requirements (check 1 from the list):
  • Standard QE validation, with pre-merge tests unless stated otherwise.
  • Regression tests only (e.g. refactoring with no user-facing change).
  • No QE (e.g. trivial change with high reviewer's confidence, or per agreement with the QE team).

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

The "make manifests" target changed to only look in ./apis, hence missing
the kubebuilder:rbac annotations. Thus moving these annotations inside ./apis
@jotak
Copy link
Member Author

jotak commented May 21, 2024

With this PR, here's the output of oc get flowmetrics:

$ oc get flowmetrics
NAME                                          METRIC NAME                              STATUS   CARDINALITY
flowmetric-cluster-external-ingress-bytes     cluster_external_ingress_bytes_total     Ready    fine
flowmetric-cluster-external-ingress-flows     cluster_external_ingress_flows_total     Ready    careful
flowmetric-cluster-external-ingress-packets   cluster_external_ingress_packets_total   Ready    fine

@codecov-commenter
Copy link

codecov-commenter commented May 21, 2024

Codecov Report

Attention: Patch coverage is 73.78641% with 27 lines in your changes are missing coverage. Please review.

Project coverage is 67.18%. Comparing base (18f3da6) to head (a4509eb).

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #650      +/-   ##
==========================================
+ Coverage   67.00%   67.18%   +0.18%     
==========================================
  Files          68       70       +2     
  Lines        7804     7893      +89     
==========================================
+ Hits         5229     5303      +74     
- Misses       2197     2209      +12     
- Partials      378      381       +3     
Flag Coverage Δ
unittests 67.18% <73.78%> (+0.18%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Coverage Δ
apis/flowmetrics/v1alpha1/flowmetric_types.go 100.00% <ø> (ø)
apis/flowmetrics/v1alpha1/zz_generated.deepcopy.go 43.63% <100.00%> (+8.39%) ⬆️
controllers/consoleplugin/config/config.go 75.00% <ø> (ø)
controllers/flp/flp_controller.go 60.91% <100.00%> (+0.40%) ⬆️
pkg/helper/helpers.go 87.30% <100.00%> (+3.01%) ⬆️
pkg/manager/manager.go 66.66% <ø> (ø)
controllers/flp/flp_pipeline_builder.go 87.35% <40.00%> (-0.70%) ⬇️
pkg/helper/cardinality.go 88.88% <88.88%> (ø)
apis/flowmetrics/v1alpha1/flowmetric_webhook.go 52.00% <33.33%> (-0.09%) ⬇️
controllers/flp/fmstatus/flowmetrics_status.go 69.09% <69.09%> (ø)

@memodi
Copy link
Contributor

memodi commented May 21, 2024

/ok-to-test

@openshift-ci openshift-ci bot added the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label May 21, 2024
Copy link

New images:

  • quay.io/netobserv/network-observability-operator:a443382
  • quay.io/netobserv/network-observability-operator-bundle:v0.0.0-a443382
  • quay.io/netobserv/network-observability-operator-catalog:v0.0.0-a443382

They will expire after two weeks.

To deploy this build:

# Direct deployment, from operator repo
IMAGE=quay.io/netobserv/network-observability-operator:a443382 make deploy

# Or using operator-sdk
operator-sdk run bundle quay.io/netobserv/network-observability-operator-bundle:v0.0.0-a443382

Or as a Catalog Source:

apiVersion: operators.coreos.com/v1alpha1
kind: CatalogSource
metadata:
  name: netobserv-dev
  namespace: openshift-marketplace
spec:
  sourceType: grpc
  image: quay.io/netobserv/network-observability-operator-catalog:v0.0.0-a443382
  displayName: NetObserv development catalog
  publisher: Me
  updateStrategy:
    registryPoll:
      interval: 1m

@memodi
Copy link
Contributor

memodi commented May 21, 2024

thanks @jotak , this works as expected:

image
$ oc get flowmetrics
NAME           METRIC NAME           STATUS   CARDINALITY
addr-metrics   service_addr_total    Ready    avoid
port-metrics   service_ports_total   Ready    careful

/label qe-approved

@openshift-ci openshift-ci bot added the qe-approved QE has approved this pull request label May 21, 2024
@openshift-ci-robot
Copy link
Collaborator

openshift-ci-robot commented May 21, 2024

@jotak: This pull request references NETOBSERV-1648 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.17.0" version, but no target version was set.

In response to this:

Description

  • Add Status/Conditions to FlowMetrics CRD
  • Add a Ready status
  • Add a Cardinality status
  • Add both of them as printColumns

Dependencies

n/a

Checklist

If you are not familiar with our processes or don't know what to answer in the list below, let us know in a comment: the maintainers will take care of that.

  • Is this PR backed with a JIRA ticket? If so, make sure it is written as a title prefix (in general, PRs affecting the NetObserv/Network Observability product should be backed with a JIRA ticket - especially if they bring user facing changes).
  • Does this PR require product documentation?
  • If so, make sure the JIRA epic is labelled with "documentation" and provides a description relevant for doc writers, such as use cases or scenarios. Any required step to activate or configure the feature should be documented there, such as new CRD knobs.
  • Does this PR require a product release notes entry?
  • If so, fill in "Release Note Text" in the JIRA.
  • Is there anything else the QE team should know before testing? E.g: configuration changes, environment setup, etc.
  • If so, make sure it is described in the JIRA ticket.
  • QE requirements (check 1 from the list):
  • Standard QE validation, with pre-merge tests unless stated otherwise.
  • Regression tests only (e.g. refactoring with no user-facing change).
  • No QE (e.g. trivial change with high reviewer's confidence, or per agreement with the QE team).

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

nsname := types.NamespacedName{Name: fm.Name, Namespace: fm.Namespace}
mapStatuses[nsname] = &metav1.Condition{
Type: "Ready",
Reason: "Ready",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"Pending" ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SetPending is actually not used I'll remove it

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

if err != nil {
log.Error(err, "failed to update FlowMetrics status")
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it might be a good idea adding UT for this while u are here

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

//+kubebuilder:rbac:groups=config.openshift.io,resources=clusterversions;networks,verbs=get;list;watch
//+kubebuilder:rbac:groups=loki.grafana.com,resources=network,resourceNames=logs,verbs=get;create
//+kubebuilder:rbac:urls="/metrics",verbs=get

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we should change what the sdk operator generated template, now if we added new API or webhook we will have to move the newly generated ones to the permissions.go while this is none functional change but I prefer sticking with SDK generated fmt

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is because here you changed the controller-gen call to check only the ./apis path and not the whole repo, so that broke the rbac generation; we need either to move that to apis or to revert that change on controller-gen path, did you have a specific reason to change the path?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no we can revert the controller-gen path changes I was debugging something and left that change around

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, done

@@ -132,18 +132,11 @@ rules:
- flows.netobserv.io
resources:
- flowcollectors/status
- flowmetrics/status
verbs:
- get
- patch
- update
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

u dropped watch verb that was used with fm is that no longer needed ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no flowmetrics still has watch look at line 124, it's now having same perms as flowcollectors

@github-actions github-actions bot removed the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label May 22, 2024
@msherif1234
Copy link
Contributor

Thanks @jotak
/LGTM

@jotak
Copy link
Member Author

jotak commented May 22, 2024

/approve

Copy link

openshift-ci bot commented May 22, 2024

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: jotak

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-merge-bot openshift-merge-bot bot merged commit cc6dd13 into netobserv:main May 22, 2024
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants