[metrics 2/x] Configure Prometheus Operator #687

zeeke · 2024-04-23T17:00:42Z

Deploy the needed configuration to make the prometheus
operator to find and scrape the sriov-network-metrics-exporter
endpoints, including the ServiceMonitor, Role and RoleBinding

depends on:

[metrics 1/x] Support sriov-network-metrics-exporter #655

github-actions · 2024-04-23T17:00:53Z

Thanks for your PR,
To run vendors CIs use one of:

/test-all: To run all tests for all vendors.
/test-e2e-all: To run all E2E tests for all vendors.
/test-e2e-nvidia-all: To run all E2E tests for NVIDIA vendor.

To skip the vendors CIs use one of:

/skip-all: To skip all tests for all vendors.
/skip-e2e-all: To skip all E2E tests for all vendors.
/skip-e2e-nvidia-all: To skip all E2E tests for NVIDIA vendor.
Best regards.

github-actions · 2024-04-24T15:51:48Z

Thanks for your PR,
To run vendors CIs use one of:

/test-all: To run all tests for all vendors.
/test-e2e-all: To run all E2E tests for all vendors.
/test-e2e-nvidia-all: To run all E2E tests for NVIDIA vendor.

To skip the vendors CIs use one of:

/skip-all: To skip all tests for all vendors.
/skip-e2e-all: To skip all E2E tests for all vendors.
/skip-e2e-nvidia-all: To skip all E2E tests for NVIDIA vendor.
Best regards.

github-actions · 2024-04-24T16:51:37Z

Thanks for your PR,
To run vendors CIs use one of:

/test-all: To run all tests for all vendors.
/test-e2e-all: To run all E2E tests for all vendors.
/test-e2e-nvidia-all: To run all E2E tests for NVIDIA vendor.

To skip the vendors CIs use one of:

/skip-all: To skip all tests for all vendors.
/skip-e2e-all: To skip all E2E tests for all vendors.
/skip-e2e-nvidia-all: To skip all E2E tests for NVIDIA vendor.
Best regards.

coveralls · 2024-04-24T16:57:33Z

Pull Request Test Coverage Report for Build 9746872108

Details

14 of 41 (34.15%) changed or added relevant lines in 3 files are covered.
7 unchanged lines in 3 files lost coverage.
Overall coverage decreased (-0.08%) to 40.546%

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
controllers/sriovoperatorconfig_controller.go	9	12	75.0%
controllers/helper.go	5	10	50.0%
pkg/utils/cluster.go	0	19	0.0%

Files with Coverage Reduction	New Missed Lines	%
controllers/drain_controller.go	1	68.06%
controllers/sriovoperatorconfig_controller.go	1	63.95%
controllers/generic_network_controller.go	5	74.53%

Totals
Change from base Build 9745388570:	-0.08%
Covered Lines:	5520
Relevant Lines:	13614

💛 - Coveralls

github-actions · 2024-04-24T17:06:29Z

Thanks for your PR,
To run vendors CIs use one of:

/test-all: To run all tests for all vendors.
/test-e2e-all: To run all E2E tests for all vendors.
/test-e2e-nvidia-all: To run all E2E tests for NVIDIA vendor.

To skip the vendors CIs use one of:

/skip-all: To skip all tests for all vendors.
/skip-e2e-all: To skip all E2E tests for all vendors.
/skip-e2e-nvidia-all: To skip all E2E tests for NVIDIA vendor.
Best regards.

github-actions · 2024-05-17T16:31:58Z

Thanks for your PR,
To run vendors CIs use one of:

/test-all: To run all tests for all vendors.
/test-e2e-all: To run all E2E tests for all vendors.
/test-e2e-nvidia-all: To run all E2E tests for NVIDIA vendor.

To skip the vendors CIs use one of:

/skip-all: To skip all tests for all vendors.
/skip-e2e-all: To skip all E2E tests for all vendors.
/skip-e2e-nvidia-all: To skip all E2E tests for NVIDIA vendor.
Best regards.

github-actions · 2024-05-17T16:53:56Z

Thanks for your PR,
To run vendors CIs use one of:

/test-all: To run all tests for all vendors.
/test-e2e-all: To run all E2E tests for all vendors.
/test-e2e-nvidia-all: To run all E2E tests for NVIDIA vendor.

To skip the vendors CIs use one of:

/skip-all: To skip all tests for all vendors.
/skip-e2e-all: To skip all E2E tests for all vendors.
/skip-e2e-nvidia-all: To skip all E2E tests for NVIDIA vendor.
Best regards.

SchSeba · 2024-05-22T12:23:58Z

bindata/manifests/metrics-exporter/metrics-service.yaml

@@ -18,6 +18,7 @@ spec:
      name: sriov-network-metrics
      port: {{ .MetricsExporterPort }}
      targetPort: {{ .MetricsExporterPort }}
+{{ if .IsOpenshift }}


I think here we can also support k8s

lets check if the ServiceMonitor CRD exist in the cluster and deploy it instead of checking only for openshift WDYT?

good idea, working on that

Done, I used the unstructured client to check if the ServiceMonitore resource definition is available in the cluster. Does it sound good?

github-actions · 2024-05-28T07:23:06Z

Thanks for your PR,
To run vendors CIs use one of:

/test-all: To run all tests for all vendors.
/test-e2e-all: To run all E2E tests for all vendors.
/test-e2e-nvidia-all: To run all E2E tests for NVIDIA vendor.

To skip the vendors CIs use one of:

/skip-all: To skip all tests for all vendors.
/skip-e2e-all: To skip all E2E tests for all vendors.
/skip-e2e-nvidia-all: To skip all E2E tests for NVIDIA vendor.
Best regards.

github-actions · 2024-05-28T08:10:11Z

Thanks for your PR,
To run vendors CIs use one of:

/test-all: To run all tests for all vendors.
/test-e2e-all: To run all E2E tests for all vendors.
/test-e2e-nvidia-all: To run all E2E tests for NVIDIA vendor.

To skip the vendors CIs use one of:

/skip-all: To skip all tests for all vendors.
/skip-e2e-all: To skip all E2E tests for all vendors.
/skip-e2e-nvidia-all: To skip all E2E tests for NVIDIA vendor.
Best regards.

github-actions · 2024-05-28T08:12:43Z

Thanks for your PR,
To run vendors CIs use one of:

/test-all: To run all tests for all vendors.
/test-e2e-all: To run all E2E tests for all vendors.
/test-e2e-nvidia-all: To run all E2E tests for NVIDIA vendor.

To skip the vendors CIs use one of:

/skip-all: To skip all tests for all vendors.
/skip-e2e-all: To skip all E2E tests for all vendors.
/skip-e2e-nvidia-all: To skip all E2E tests for NVIDIA vendor.
Best regards.

github-actions · 2024-05-28T08:17:05Z

Thanks for your PR,
To run vendors CIs use one of:

/test-all: To run all tests for all vendors.
/test-e2e-all: To run all E2E tests for all vendors.
/test-e2e-nvidia-all: To run all E2E tests for NVIDIA vendor.

To skip the vendors CIs use one of:

/skip-all: To skip all tests for all vendors.
/skip-e2e-all: To skip all E2E tests for all vendors.
/skip-e2e-nvidia-all: To skip all E2E tests for NVIDIA vendor.
Best regards.

github-actions · 2024-05-28T09:29:40Z

Thanks for your PR,
To run vendors CIs use one of:

/test-all: To run all tests for all vendors.
/test-e2e-all: To run all E2E tests for all vendors.
/test-e2e-nvidia-all: To run all E2E tests for NVIDIA vendor.

To skip the vendors CIs use one of:

/skip-all: To skip all tests for all vendors.
/skip-e2e-all: To skip all E2E tests for all vendors.
/skip-e2e-nvidia-all: To skip all E2E tests for NVIDIA vendor.
Best regards.

SchSeba · 2024-05-29T10:49:21Z

deploy/role.yaml

@@ -32,6 +32,7 @@ rules:
  verbs:
  - get
  - create
+  - list


I think we need this one also under the config folder so it will be generated for the bundle

the contents of the config/ folder is currently out of sync with openshit/sriov-network-operator (e.g. see config/rbac/role.yaml here and here).

I would align that in a separated PR. Does it sound good?

SchSeba · 2024-05-29T10:50:47Z

controllers/helper.go

+
+func isPrometheusOperatorInstalled(ctx context.Context, client k8sclient.Reader) bool {
+	u := &uns.UnstructuredList{}
+	u.SetGroupVersionKind(schema.GroupVersionKind{


I was thinking (maybe that is not the right way) to do a kubectl get crd servicemonitor not to search if there is any server monitor object in the cluster :)

I think we can try getting the CRD (e.g. see this).

The drawback is that we have to add the permission (ClusterRole,ClusterRoleBinding,...) to make the operator read that CustomResourceDefinition resource, but it might end up cleaner.

I think it should be ok to add get for CRD that should not expose the operator to any security issues :)

I had to add the permission to the ClusterRole (instead of Role), as the CustomResourceDefinition is not namespaced.
For the same reason, I had to add a non-namespace client to the SriovOperatorConfigReconicler.
please, take a look

SchSeba · 2024-06-03T11:27:45Z

controllers/sriovoperatorconfig_controller.go

+		}
+
+		if r.PlatformHelper.IsOpenshiftCluster() {
+			err = utils.AddLabelToNamespace(ctx, vars.Namespace, "openshift.io/cluster-monitoring", "true", r.Client)


I think we need to put this in the namespace creation template and not let the operator have permission to upgrade namespace that sounds a security risk

Sounds good. Maybe we can leverage the operatorframework.io/cluster-monitoring annotation in the CSV, in the openshift fork

https://github.com/openshift/enhancements/blob/master/enhancements/olm/olm-managed-operator-metrics.md#fulfilling-namespace-and-rbac-requirements

adrianchiris · 2024-06-25T12:56:59Z

@zeeke can you rebase this one ?

github-actions · 2024-06-27T09:33:32Z

Thanks for your PR,
To run vendors CIs use one of:

/test-all: To run all tests for all vendors.
/test-e2e-all: To run all E2E tests for all vendors.
/test-e2e-nvidia-all: To run all E2E tests for NVIDIA vendor.

To skip the vendors CIs use one of:

/skip-all: To skip all tests for all vendors.
/skip-e2e-all: To skip all E2E tests for all vendors.
/skip-e2e-nvidia-all: To skip all E2E tests for NVIDIA vendor.
Best regards.

zeeke · 2024-07-01T16:32:07Z

/hold

github-actions · 2024-07-02T13:17:03Z

Thanks for your PR,
To run vendors CIs use one of:

/test-all: To run all tests for all vendors.
/test-e2e-all: To run all E2E tests for all vendors.
/test-e2e-nvidia-all: To run all E2E tests for NVIDIA vendor.

To skip the vendors CIs use one of:

/skip-all: To skip all tests for all vendors.
/skip-e2e-all: To skip all E2E tests for all vendors.
/skip-e2e-nvidia-all: To skip all E2E tests for NVIDIA vendor.
Best regards.

github-actions · 2024-07-03T09:16:00Z

Thanks for your PR,
To run vendors CIs use one of:

/test-all: To run all tests for all vendors.
/test-e2e-all: To run all E2E tests for all vendors.
/test-e2e-nvidia-all: To run all E2E tests for NVIDIA vendor.

To skip the vendors CIs use one of:

/skip-all: To skip all tests for all vendors.
/skip-e2e-all: To skip all E2E tests for all vendors.
/skip-e2e-nvidia-all: To skip all E2E tests for NVIDIA vendor.
Best regards.

Package `github.com/prometheus-operator/prometheus-operator/pkg/client` can be used for testing purpose. Signed-off-by: Andrea Panattoni <apanatto@redhat.com>

github-actions · 2024-07-03T09:20:32Z

Thanks for your PR,
To run vendors CIs use one of:

/test-all: To run all tests for all vendors.
/test-e2e-all: To run all E2E tests for all vendors.
/test-e2e-nvidia-all: To run all E2E tests for NVIDIA vendor.

To skip the vendors CIs use one of:

/skip-all: To skip all tests for all vendors.
/skip-e2e-all: To skip all E2E tests for all vendors.
/skip-e2e-nvidia-all: To skip all E2E tests for NVIDIA vendor.
Best regards.

github-actions · 2024-07-03T11:40:24Z

Thanks for your PR,
To run vendors CIs use one of:

/test-all: To run all tests for all vendors.
/test-e2e-all: To run all E2E tests for all vendors.
/test-e2e-nvidia-all: To run all E2E tests for NVIDIA vendor.

To skip the vendors CIs use one of:

/skip-all: To skip all tests for all vendors.
/skip-e2e-all: To skip all E2E tests for all vendors.
/skip-e2e-nvidia-all: To skip all E2E tests for NVIDIA vendor.
Best regards.

github-actions · 2024-07-03T15:15:28Z

Thanks for your PR,
To run vendors CIs use one of:

/test-all: To run all tests for all vendors.
/test-e2e-all: To run all E2E tests for all vendors.
/test-e2e-nvidia-all: To run all E2E tests for NVIDIA vendor.

To skip the vendors CIs use one of:

/skip-all: To skip all tests for all vendors.
/skip-e2e-all: To skip all E2E tests for all vendors.
/skip-e2e-nvidia-all: To skip all E2E tests for NVIDIA vendor.
Best regards.

github-actions · 2024-07-03T15:35:05Z

Thanks for your PR,
To run vendors CIs use one of:

/test-all: To run all tests for all vendors.
/test-e2e-all: To run all E2E tests for all vendors.
/test-e2e-nvidia-all: To run all E2E tests for NVIDIA vendor.

To skip the vendors CIs use one of:

/skip-all: To skip all tests for all vendors.
/skip-e2e-all: To skip all E2E tests for all vendors.
/skip-e2e-nvidia-all: To skip all E2E tests for NVIDIA vendor.
Best regards.

Signed-off-by: Andrea Panattoni <apanatto@redhat.com>

github-actions · 2024-07-03T16:04:17Z

Thanks for your PR,
To run vendors CIs use one of:

/test-all: To run all tests for all vendors.
/test-e2e-all: To run all E2E tests for all vendors.
/test-e2e-nvidia-all: To run all E2E tests for NVIDIA vendor.

To skip the vendors CIs use one of:

/skip-all: To skip all tests for all vendors.
/skip-e2e-all: To skip all E2E tests for all vendors.
/skip-e2e-nvidia-all: To skip all E2E tests for NVIDIA vendor.
Best regards.

github-actions · 2024-07-05T08:47:58Z

Thanks for your PR,
To run vendors CIs use one of:

/test-all: To run all tests for all vendors.
/test-e2e-all: To run all E2E tests for all vendors.
/test-e2e-nvidia-all: To run all E2E tests for NVIDIA vendor.

To skip the vendors CIs use one of:

/skip-all: To skip all tests for all vendors.
/skip-e2e-all: To skip all E2E tests for all vendors.
/skip-e2e-nvidia-all: To skip all E2E tests for NVIDIA vendor.
Best regards.

zeeke · 2024-07-05T08:48:16Z

@adrianchiris , @SchSeba please take another look

SchSeba

just some small comments :)

SchSeba · 2024-07-11T10:37:58Z

bindata/manifests/metrics-exporter/metrics-service.yaml

+subjects:
+- kind: ServiceAccount
+  name: prometheus-k8s
+  namespace: openshift-monitoring


this is good for openshift on vanilla k8s please add a variable to the helmchart something like
https://github.com/metallb/metallb/blob/21dd75560f3b8614c14b1bb55a79dbcc231e36a7/charts/metallb/templates/servicemonitor.yaml#L192

Added environment variables and Helm stuff to make the subject configurable.

SchSeba · 2024-07-11T10:43:14Z

deploy/clusterrole.yaml

@@ -36,6 +36,9 @@ rules:
 - apiGroups: ["config.openshift.io"]
  resources: ["infrastructures"]
  verbs: ["get", "list", "watch"]
+- apiGroups: ["apiextensions.k8s.io"]
+  resources: ["customresourcedefinitions"]


I think we should add this one also to the config folder this way when we generate the csv file for OLM it should be also in place

Added, though I think we still need some cleanup in that folder.

SchSeba · 2024-07-11T10:45:43Z

pkg/utils/cluster.go

@@ -161,3 +161,28 @@ func AnnotateNode(ctx context.Context, nodeName string, key, value string, c cli

 	return AnnotateObject(ctx, node, key, value, c)
 }
+
+func AddLabelToNamespace(ctx context.Context, namespaceName, key, value string, c client.Client) error {
+	ns := &corev1.Namespace{}


I was not able to find where we use this one.

in general I think we should document the need to add a label for monitoring on namespace creation and not add a rbac to allow the operator to update namespace object

I removed the function. I will re-add it in the e2e test PR.

About permissions, the operator already had the RBAC to write on namespaces (see deploy/clusterrole.yaml and openshift CSV). No permission has been added in this PR for namespaces.

I'll take care of documenting the namespace configuration in OpenShift

github-actions · 2024-07-12T06:38:34Z

Thanks for your PR,
To run vendors CIs use one of:

/test-all: To run all tests for all vendors.
/test-e2e-all: To run all E2E tests for all vendors.
/test-e2e-nvidia-all: To run all E2E tests for NVIDIA vendor.

To skip the vendors CIs use one of:

/skip-all: To skip all tests for all vendors.
/skip-e2e-all: To skip all E2E tests for all vendors.
/skip-e2e-nvidia-all: To skip all E2E tests for NVIDIA vendor.
Best regards.

Deploy the needed configuration to make the prometheus operator to find and scrape the sriov-network-metrics-exporter endpoints, including the ServiceMonitor, Role and RoleBinding. Resources are installed only if the Prometheus operator is installed. Signed-off-by: Andrea Panattoni <apanatto@redhat.com>

When useing `ServiceMonitors`, Prometheus Operator needs permissions to read Services,Endpoint and Pods in the monitored namespace (i.e. the SRIOV operator ns). Make the ServiceAccount subject configurable via environment variables. Signed-off-by: Andrea Panattoni <apanatto@redhat.com>

github-actions · 2024-07-12T06:55:13Z

Thanks for your PR,
To run vendors CIs use one of:

/test-all: To run all tests for all vendors.
/test-e2e-all: To run all E2E tests for all vendors.
/test-e2e-nvidia-all: To run all E2E tests for NVIDIA vendor.

To skip the vendors CIs use one of:

/skip-all: To skip all tests for all vendors.
/skip-e2e-all: To skip all E2E tests for all vendors.
/skip-e2e-nvidia-all: To skip all E2E tests for NVIDIA vendor.
Best regards.

zeeke · 2024-07-16T10:02:10Z

@adrianchiris , @SchSeba please take another look

zeeke changed the title ~~[metrics 2/3] Configure Prometheus Operator~~ [metrics 2/x] Configure Prometheus Operator Apr 23, 2024

zeeke force-pushed the metrics-exporter-prometheus branch from 06e880c to 75e8305 Compare April 24, 2024 15:51

zeeke force-pushed the metrics-exporter-prometheus branch from 75e8305 to 06c86e1 Compare April 24, 2024 16:51

zeeke force-pushed the metrics-exporter-prometheus branch from 06c86e1 to 37fbdf4 Compare April 24, 2024 17:06

zeeke force-pushed the metrics-exporter-prometheus branch from 37fbdf4 to 7999f7e Compare May 17, 2024 16:31

zeeke force-pushed the metrics-exporter-prometheus branch from 7999f7e to 58b9fb3 Compare May 17, 2024 16:53

SchSeba reviewed May 22, 2024

View reviewed changes

zeeke mentioned this pull request May 22, 2024

[metrics 3/x] end-to-end tests #702

Open

zeeke force-pushed the metrics-exporter-prometheus branch from 58b9fb3 to 9b5e4cf Compare May 28, 2024 07:22

zeeke force-pushed the metrics-exporter-prometheus branch from 9b5e4cf to 2bddf42 Compare May 28, 2024 08:10

zeeke force-pushed the metrics-exporter-prometheus branch from 2bddf42 to cec70bd Compare May 28, 2024 08:12

zeeke force-pushed the metrics-exporter-prometheus branch from cec70bd to 6da499a Compare May 28, 2024 08:16

zeeke force-pushed the metrics-exporter-prometheus branch from 6da499a to e87a82d Compare May 28, 2024 09:29

SchSeba reviewed May 29, 2024

View reviewed changes

SchSeba reviewed Jun 3, 2024

View reviewed changes

zeeke force-pushed the metrics-exporter-prometheus branch from e87a82d to 17dabd0 Compare June 27, 2024 09:33

github-actions bot added the hold label Jul 1, 2024

zeeke force-pushed the metrics-exporter-prometheus branch from 391a8ca to 686a48c Compare July 2, 2024 13:16

zeeke force-pushed the metrics-exporter-prometheus branch from 686a48c to 63f4fe3 Compare July 3, 2024 09:15

Add prometheus-operator/pkg/client dependency

9c1eb96

Package `github.com/prometheus-operator/prometheus-operator/pkg/client` can be used for testing purpose. Signed-off-by: Andrea Panattoni <apanatto@redhat.com>

zeeke force-pushed the metrics-exporter-prometheus branch from 63f4fe3 to ed8f86f Compare July 3, 2024 09:20

zeeke force-pushed the metrics-exporter-prometheus branch from ed8f86f to de99a66 Compare July 3, 2024 11:40

zeeke force-pushed the metrics-exporter-prometheus branch from de99a66 to 7aacaae Compare July 3, 2024 15:15

zeeke force-pushed the metrics-exporter-prometheus branch from 7aacaae to 2faab1d Compare July 3, 2024 15:34

Add a non namespaced client to SriovOperatorConfigReconciler

11d6708

Signed-off-by: Andrea Panattoni <apanatto@redhat.com>

zeeke force-pushed the metrics-exporter-prometheus branch from 2faab1d to 19e5be7 Compare July 3, 2024 16:04

zeeke removed the hold label Jul 5, 2024

zeeke force-pushed the metrics-exporter-prometheus branch from 19e5be7 to 41106a5 Compare July 5, 2024 08:47

zeeke mentioned this pull request Jul 5, 2024

CNF-12007: d/s: CSV adjustment for sriov-network-metrics-exporter openshift/sriov-network-operator#951

Open

SchSeba reviewed Jul 11, 2024

View reviewed changes

zeeke added 2 commits July 12, 2024 08:48

zeeke force-pushed the metrics-exporter-prometheus branch from a39a440 to 108ef15 Compare July 12, 2024 06:55

@@ @@ -32,6 +32,7 @@ rules: @@
                 verbs:
                 - get
                 - create
+                - list

[metrics 2/x] Configure Prometheus Operator #687

Are you sure you want to change the base?

[metrics 2/x] Configure Prometheus Operator #687

Conversation

zeeke commented Apr 23, 2024 • edited Loading

github-actions bot commented Apr 23, 2024

github-actions bot commented Apr 24, 2024

github-actions bot commented Apr 24, 2024

coveralls commented Apr 24, 2024 • edited Loading

Pull Request Test Coverage Report for Build 9746872108

Details

💛 - Coveralls

github-actions bot commented Apr 24, 2024

github-actions bot commented May 17, 2024

github-actions bot commented May 17, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

github-actions bot commented May 28, 2024

github-actions bot commented May 28, 2024

github-actions bot commented May 28, 2024

github-actions bot commented May 28, 2024

github-actions bot commented May 28, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

adrianchiris commented Jun 25, 2024

github-actions bot commented Jun 27, 2024

zeeke commented Jul 1, 2024

github-actions bot commented Jul 2, 2024

github-actions bot commented Jul 3, 2024

github-actions bot commented Jul 3, 2024

github-actions bot commented Jul 3, 2024

github-actions bot commented Jul 3, 2024

github-actions bot commented Jul 3, 2024

github-actions bot commented Jul 3, 2024

github-actions bot commented Jul 5, 2024

zeeke commented Jul 5, 2024

SchSeba left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

github-actions bot commented Jul 12, 2024

github-actions bot commented Jul 12, 2024

zeeke commented Jul 16, 2024

zeeke commented Apr 23, 2024 •

edited

Loading

coveralls commented Apr 24, 2024 •

edited

Loading