From df5df5b2b6ba8f61ff652aa9657bee9587d775e8 Mon Sep 17 00:00:00 2001 From: Yiming Bao Date: Fri, 19 Aug 2022 14:07:06 +0800 Subject: [PATCH] PowerScale Obs doc update for CSM Observability 1.3.0 --- content/docs/observability/_index.md | 37 ++++----- .../docs/observability/deployment/_index.md | 78 +++++++++++++++---- content/docs/observability/deployment/helm.md | 46 ++++++++++- .../docs/observability/deployment/offline.md | 44 ++++++++--- .../docs/observability/deployment/online.md | 5 +- content/docs/observability/design/_index.md | 5 +- .../docs/observability/metrics/powerscale.md | 45 +++++++++++ content/docs/observability/release/_index.md | 11 ++- .../observability/troubleshooting/_index.md | 2 +- content/docs/observability/upgrade/_index.md | 41 +++++++++- 10 files changed, 257 insertions(+), 57 deletions(-) create mode 100644 content/docs/observability/metrics/powerscale.md diff --git a/content/docs/observability/_index.md b/content/docs/observability/_index.md index 8f9f05fc63..a014c860eb 100644 --- a/content/docs/observability/_index.md +++ b/content/docs/observability/_index.md @@ -14,13 +14,14 @@ Description: > Metrics data is collected and pushed to the [OpenTelemetry Collector](https://github.com/open-telemetry/opentelemetry-collector), so it can be processed, and exported in a format consumable by Prometheus. SSL certificates for TLS between nodes are handled by [cert-manager](https://github.com/jetstack/cert-manager). -CSM for Observability is composed of several services, each living in its own GitHub repository, that can be installed following one of the three deployments we support [here](deployment). Contributions can be made to this repository or any of the CSM for Observability repositories listed below. +CSM for Observability is composed of several services, each living in its own GitHub repository, that can be installed following one of the four deployments we support [here](deployment). Contributions can be made to this repository or any of the CSM for Observability repositories listed below. {{}} | Name | Repository | Description | | ---- | --------- | ----------- | -| Performance Metrics for PowerFlex | [CSM Metrics for PowerFlex](https://github.com/dell/karavi-metrics-powerflex) | Performance Metrics for PowerFlex captures telemetry data about Kubernetes storage usage and performance obtained through the CSI (Container Storage Interface) Driver for Dell PowerFlex. The metrics service pushes it to the OpenTelemetry Collector, so it can be processed, and exported in a format consumable by Prometheus. Prometheus can then be configured to scrape the OpenTelemetry Collector exporter endpoint to provide metrics so they can be visualized in Grafana. Please visit the repository for more information. | -| Performance Metrics for PowerStore | [CSM Metrics for PowerStore](https://github.com/dell/csm-metrics-powerstore) | Performance Metrics for PowerStore captures telemetry data about Kubernetes storage usage and performance obtained through the CSI (Container Storage Interface) Driver for Dell PowerStore. The metrics service pushes it to the OpenTelemetry Collector, so it can be processed, and exported in a format consumable by Prometheus. Prometheus can then be configured to scrape the OpenTelemetry Collector exporter endpoint to provide metrics so they can be visualized in Grafana. Please visit the repository for more information. | +| Metrics for PowerFlex | [CSM Metrics for PowerFlex](https://github.com/dell/karavi-metrics-powerflex) | Metrics for PowerFlex captures telemetry data about Kubernetes storage usage and performance obtained through the CSI (Container Storage Interface) Driver for Dell PowerFlex. The metrics service pushes it to the OpenTelemetry Collector, so it can be processed, and exported in a format consumable by Prometheus. Prometheus can then be configured to scrape the OpenTelemetry Collector exporter endpoint to provide metrics, so they can be visualized in Grafana. Please visit the repository for more information. | +| Metrics for PowerStore | [CSM Metrics for PowerStore](https://github.com/dell/csm-metrics-powerstore) | Metrics for PowerStore captures telemetry data about Kubernetes storage usage and performance obtained through the CSI (Container Storage Interface) Driver for Dell PowerStore. The metrics service pushes it to the OpenTelemetry Collector, so it can be processed, and exported in a format consumable by Prometheus. Prometheus can then be configured to scrape the OpenTelemetry Collector exporter endpoint to provide metrics, so they can be visualized in Grafana. Please visit the repository for more information. | +| Metrics for PowerScale | [CSM Metrics for PowerScale](https://github.com/dell/csm-metrics-powerscale) | Metrics for PowerScale captures telemetry data about Kubernetes storage usage and performance obtained through the CSI (Container Storage Interface) Driver for Dell PowerScale. The metrics service pushes it to the OpenTelemetry Collector, so it can be processed, and exported in a format consumable by Prometheus. Prometheus can then be configured to scrape the OpenTelemetry Collector exporter endpoint to provide metrics, so they can be visualized in Grafana. Please visit the repository for more information. | | Volume Topology | [CSM Topology](https://github.com/dell/karavi-topology) | Topology provides Kubernetes administrators with the topology data related to containerized storage that is provisioned by a CSI (Container Storage Interface) Driver for Dell storage products. The Topology service is enabled by default as part of the CSM for Observability Helm Chart [values file](https://github.com/dell/helm-charts/blob/main/charts/karavi-observability/values.yaml). Please visit the repository for more information. | {{
}} @@ -31,14 +32,14 @@ CSM for Observability provides the following capabilities: {{}} | Capability | PowerMax | PowerFlex | Unity XT | PowerScale | PowerStore | | - | :-: | :-: | :-: | :-: | :-: | -| Collect and expose Volume Metrics via the OpenTelemetry Collector | no | yes | no | no | yes | +| Collect and expose Volume Metrics via the OpenTelemetry Collector | no | yes | no | yes | yes | | Collect and expose File System Metrics via the OpenTelemetry Collector | no | no | no | no | yes | | Collect and expose export (k8s) node metrics via the OpenTelemetry Collector | no | yes | no | no | no | -| Collect and expose filesystem capacity metrics via the OpenTelemetry Collector | no | no | no | no | yes | -| Collect and expose block storage capacity metrics via the OpenTelemetry Collector | no | yes | no | no | yes | -| Non-disruptive config changes | no | yes | no | no | yes | -| Non-disruptive log level changes | no | yes | no | no | yes | -| Grafana Dashboards for displaying metrics and topology data | no | yes | no | no | yes | +| Collect and expose block storage metrics via the OpenTelemetry Collector | no | yes | no | no | yes | +| Collect and expose file storage metrics via the OpenTelemetry Collector | no | no | no | yes | yes | +| Non-disruptive config changes | no | yes | no | yes | yes | +| Non-disruptive log level changes | no | yes | no | yes | yes | +| Grafana Dashboards for displaying metrics and topology data | no | yes | no | yes | yes | {{
}} ## Supported Operating Systems/Container Orchestrator Platforms @@ -56,9 +57,9 @@ CSM for Observability provides the following capabilities: ## Supported Storage Platforms {{}} -| | PowerFlex | PowerStore | -|---------------|:-------------------:|:----------------:| -| Storage Array | 3.5.x, 3.6.x | 1.0.x, 2.0.x, 2.1.x | +| | PowerFlex | PowerStore | PowerScale | +|---------------|:-------------------:|:----------------:|:----------------:| +| Storage Array | 3.5.x, 3.6.x | 1.0.x, 2.0.x, 2.1.x | 8.x, 9.x | {{
}} ## Supported CSI Drivers @@ -69,6 +70,7 @@ CSM for Observability supports the following CSI drivers and versions. | ------------- | ---------- | ------------------ | | CSI Driver for Dell PowerFlex | [csi-powerflex](https://github.com/dell/csi-powerflex) | v2.0 + | | CSI Driver for Dell PowerStore | [csi-powerstore](https://github.com/dell/csi-powerstore) | v2.0 + | +| CSI Driver for Dell PowerScale | [csi-powerscale](https://github.com/dell/csi-powerscale) | v2.0 + | {{}} ## Topology Data @@ -78,17 +80,16 @@ CSM for Observability provides Kubernetes administrators with the topology data | Field | Description | | -------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------- | | Namespace | The namespace associated with the persistent volume claim | +| Persistent Volume Claim | The name of the persistent volume claim associated with the persistent volume | | Persistent Volume | The name of the persistent volume | +| Storage Class | The storage class associated with the persistent volume | +| Provisioned Size | The provisioned size of the persistent volume | | Status | The status of the persistent volume. "Released" indicates the persistent volume does not have a claim. "Bound" indicates the persistent volume has a claim | -| Persistent Volume Claim | The name of the persistent volume claim associated with the persistent volume | -| CSI Driver | The name of the CSI driver that was responsible for provisioning the volume on the storage system | | Created | The date the persistent volume was created | -| Provisioned Size | The provisioned size of the persistent volume | -| Storage Class | The storage class associated with the persistent volume | -| Storage System Volume Name | The name of the volume on the storage system that is associated with the persistent volume | -| Storage Pool | The storage pool name the volume/storage class is associated with | | Storage System | The storage system ID or IP address the volume is associated with | | Protocol | The storage system protocol type the volume/storage class is associated with | +| Storage Pool | The storage pool name the volume/storage class is associated with | +| Storage System Volume Name | The name of the volume on the storage system that is associated with the persistent volume | {{}} ## TLS Encryption diff --git a/content/docs/observability/deployment/_index.md b/content/docs/observability/deployment/_index.md index 50efaa2c3f..62b10741bb 100644 --- a/content/docs/observability/deployment/_index.md +++ b/content/docs/observability/deployment/_index.md @@ -239,8 +239,8 @@ Below are the steps to deploy a new Grafana instance into your Kubernetes cluste dashboards: enabled: true - ## Additional grafana server CofigMap mounts - ## Defines additional mounts with CofigMap. CofigMap must be manually created in the namespace. + ## Additional grafana server ConfigMap mounts + ## Defines additional mounts with ConfigMap. ConfigMap must be manually created in the namespace. extraConfigmapMounts: [] # If you created a ConfigMap on the previous step, delete [] and uncomment the lines below # - name: certs-configmap # mountPath: /etc/ssl/certs/ca-certificates.crt @@ -275,23 +275,29 @@ Below are the steps to deploy a new Grafana instance into your Kubernetes cluste Once Grafana is properly configured, you can import the pre-built observability dashboards. Log into Grafana and click the + icon in the side menu. Then click Import. From here you can upload the JSON files or paste the JSON text directly into the text area. Below are the locations of the dashboards that can be imported: -| Dashboard | Description | -| ------------------- | --------------------------------- | -| [PowerFlex: I/O Performance by Kubernetes Node](https://github.com/dell/karavi-observability/blob/main/grafana/dashboards/powerflex/sdc_io_metrics.json) | Provides visibility into the I/O performance metrics (IOPS, bandwidth, latency) by Kubernetes node | -| [PowerFlex: I/O Performance by Provisioned Volume](https://github.com/dell/karavi-observability/blob/main/grafana/dashboards/powerflex/volume_io_metrics.json) | Provides visibility into the I/O performance metrics (IOPS, bandwidth, latency) by volume | -| [PowerFlex: Storage Pool Consumption By CSI Driver](https://github.com/dell/karavi-observability/blob/main/grafana/dashboards/powerflex/storage_consumption.json) | Provides visibility into the total, used, and available capacity for a storage class and associated underlying storage construct. | -| [PowerStore: I/O Performance by Provisioned Volume](https://github.com/dell/karavi-observability/blob/main/grafana/dashboards/powerstore/volume_io_metrics.json) | *As of Release 0.4.0:* Provides visibility into the I/O performance metrics (IOPS, bandwidth, latency) by volume | -| [CSI Driver Provisioned Volume Topology](https://github.com/dell/karavi-observability/blob/main/grafana/dashboards/topology/topology.json) | Provides visibility into Dell CSI (Container Storage Interface) driver provisioned volume characteristics in Kubernetes correlated with volumes on the storage system. | +| Dashboard | Description | +|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| [PowerFlex: I/O Performance by Kubernetes Node](https://github.com/dell/karavi-observability/blob/main/grafana/dashboards/powerflex/sdc_io_metrics.json) | Provides visibility into the I/O performance metrics (IOPS, bandwidth, latency) by Kubernetes node | +| [PowerFlex: I/O Performance by Provisioned Volume](https://github.com/dell/karavi-observability/blob/main/grafana/dashboards/powerflex/volume_io_metrics.json) | Provides visibility into the I/O performance metrics (IOPS, bandwidth, latency) by volume | +| [PowerFlex: Storage Pool Consumption By CSI Driver](https://github.com/dell/karavi-observability/blob/main/grafana/dashboards/powerflex/storage_consumption.json) | Provides visibility into the total, used and available capacity for a storage class and associated underlying storage construct | +| [PowerStore: I/O Performance by Provisioned Volume](https://github.com/dell/karavi-observability/blob/main/grafana/dashboards/powerstore/volume_io_metrics.json) | Provides visibility into the I/O performance metrics (IOPS, bandwidth, latency) by volume | +| [PowerStore: I/O Performance by File System](https://github.com/dell/karavi-observability/blob/main/grafana/dashboards/powerstore/filesystem_io_metrics.json) | Provides visibility into the I/O performance metrics (IOPS, bandwidth, latency) by filesystem | +| [PowerStore: Array and Storage Class Consumption By CSI Driver](https://github.com/dell/karavi-observability/blob/main/grafana/dashboards/powerstore/storage_consumption.json) | Provides visibility into the total, used and available capacity for a storage class and associated underlying storage construct | +| [PowerScale: I/O Performance by Cluster](https://github.com/dell/karavi-observability/blob/main/grafana/dashboards/powerscale/cluster_io_metrics.json) | Provides visibility into the I/O performance metrics (IOPS, bandwidth) by cluster | +| [PowerScale: Capacity by Cluster](https://github.com/dell/karavi-observability/blob/main/grafana/dashboards/powerscale/cluster_capacity.json) | Provides visibility into the total, used, available capacity and directory quota capacity by cluster | +| [PowerScale: Capacity by Quota](https://github.com/dell/karavi-observability/blob/main/grafana/dashboards/powerscale/volume_capacity.json) | Provides visibility into the subscribed, remaining capacity and usage by quota | +| [CSI Driver Provisioned Volume Topology](https://github.com/dell/karavi-observability/blob/main/grafana/dashboards/topology/topology.json) | Provides visibility into Dell CSI (Container Storage Interface) driver provisioned volume characteristics in Kubernetes correlated with volumes on the storage system. | ## Dynamic Configuration Some parameters can be configured/updated during runtime without restarting the CSM for Observability services. These parameters will be stored in ConfigMaps that can be updated on the Kubernetes cluster. This will automatically change the settings on the services. -| ConfigMap | Observability Service | Parameters | -| - | - | - | -| karavi-metrics-powerflex-configmap | karavi-metrics-powerflex | | -| karavi-metrics-powerstore-configmap | karavi-metrics-powerstore | | -| karavi-topology-configmap | karavi-topology | | +| ConfigMap | Observability Service | Parameters | +|-------------------------------------|---------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| karavi-metrics-powerflex-configmap | karavi-metrics-powerflex | | +| karavi-metrics-powerstore-configmap | karavi-metrics-powerstore | | +| karavi-metrics-powerscale-configmap | karavi-metrics-powerscale | | +| karavi-topology-configmap | karavi-topology | | To update any of these settings, run the following command on the Kubernetes cluster then save the updated ConfigMap data. @@ -387,29 +393,57 @@ In this case, all storage system requests made by CSM for Observability will be #### Update the Authorization Module Token +##### CSI Driver for Dell PowerFlex + 1. Delete the current `proxy-authz-tokens` Secret from the CSM namespace. ```console $ kubectl delete secret proxy-authz-tokens -n [CSM_NAMESPACE] ``` -2. Copy the `proxy-authz-tokens` Secret from a CSI Driver to the CSM namespace. +2. Copy the `proxy-authz-tokens` Secret from the CSI Driver for Dell PowerFlex to the CSM namespace. ```console $ kubectl get secret proxy-authz-tokens -n [CSI_DRIVER_NAMESPACE] -o yaml | sed 's/namespace: [CSM_CSI_DRIVER_NAMESPACE]/namespace: [CSM_NAMESPACE]/' | kubectl create -f - ``` +##### CSI Driver for Dell PowerScale + +1. Delete the current `isilon-proxy-authz-tokens` Secret from the CSM namespace. + ```console + $ kubectl delete secret isilon-proxy-authz-tokens -n [CSM_NAMESPACE] + ``` + +2. Copy the `isilon-proxy-authz-tokens` Secret from the CSI Driver for Dell PowerScale namespace to the CSM namespace. + ```console + $ kubectl get secret proxy-authz-tokens -n [CSI_DRIVER_NAMESPACE] -o yaml | sed 's/namespace: [CSM_CSI_DRIVER_NAMESPACE]/namespace: [CSM_NAMESPACE]/'| sed 's/name: proxy-authz-tokens/name: isilon-proxy-authz-tokens/' | kubectl create -f + ``` + #### Update Storage Systems If the list of storage systems managed by a Dell CSI Driver have changed, the following steps can be performed to update CSM for Observability to reference the updated systems: +##### CSI Driver for Dell PowerFlex + 1. Delete the current `karavi-authorization-config` Secret from the CSM namespace. ```console $ kubectl delete secret proxy-authz-tokens -n [CSM_NAMESPACE] ``` -2. Copy the `karavi-authorization-config` Secret from the CSI Driver namespace to CSM for Observability namespace. +2. Copy the `karavi-authorization-config` Secret from the CSI Driver for Dell PowerFlex namespace to CSM for Observability namespace. ```console $ kubectl get secret karavi-authorization-config -n [CSI_DRIVER_NAMESPACE] -o yaml | sed 's/namespace: [CSM_CSI_DRIVER_NAMESPACE]/namespace: [CSM_NAMESPACE]/' | kubectl create -f - ``` +##### CSI Driver for Dell PowerScale + +1. Delete the current `isilon-karavi-authorization-config` Secret from the CSM namespace. + ```console + $ kubectl delete secret isilon-karavi-authorization-config -n [CSM_NAMESPACE] + ``` + +2. Copy the isilon-karavi-authorization-config Secret from the CSI Driver for Dell PowerScale namespace to CSM for Observability namespace. + ```console + $ kubectl get secret karavi-authorization-config -n [CSI_DRIVER_NAMESPACE] -o yaml | sed 's/namespace: [CSM_CSI_DRIVER_NAMESPACE]/namespace: [CSM_NAMESPACE]/' | sed 's/name: karavi-authorization-config/name: isilon-karavi-authorization-config/' | kubectl create -f + ``` + ### When CSM for Observability does not use the Authorization module In this case all storage system requests made by CSM for Observability will not be routed through the Authorization module. The following must be performed: @@ -437,3 +471,15 @@ In this case all storage system requests made by CSM for Observability will not ```console $ kubectl get secret powerstore-config -n [CSI_DRIVER_NAMESPACE] -o yaml | sed 's/namespace: [CSI_DRIVER_NAMESPACE]/namespace: [CSM_NAMESPACE]/' | kubectl create -f - ``` + +### CSI Driver for Dell PowerScale + +1. Delete the current `isilon-creds` Secret from the CSM namespace. + ```console + $ kubectl delete secret isilon-creds -n [CSM_NAMESPACE] + ``` + +2. Copy the `isilon-creds` Secret from the CSI Driver for Dell PowerScale namespace to the CSM namespace. + ```console + $ kubectl get secret isilon-creds -n [CSI_DRIVER_NAMESPACE] -o yaml | sed 's/namespace: [CSI_DRIVER_NAMESPACE]/namespace: [CSM_NAMESPACE]/' | kubectl create -f - + ``` \ No newline at end of file diff --git a/content/docs/observability/deployment/helm.md b/content/docs/observability/deployment/helm.md index 02feb6186f..55ecead109 100644 --- a/content/docs/observability/deployment/helm.md +++ b/content/docs/observability/deployment/helm.md @@ -22,7 +22,8 @@ The Container Storage Modules (CSM) for Observability Helm chart bootstraps an O 3. Add the Dell Helm Charts repo `helm repo add dell https://dell.github.io/helm-charts` 4. Copy only the deployed CSI driver entities to the Observability namespace - #### PowerFlex + + ### PowerFlex 1. Copy the config Secret from the CSI PowerFlex namespace into the CSM for Observability namespace: @@ -38,12 +39,30 @@ The Container Storage Modules (CSM) for Observability Helm chart bootstraps an O `kubectl get secret karavi-authorization-config proxy-server-root-certificate proxy-authz-tokens -n [CSI_DRIVER_NAMESPACE] -o yaml | sed 's/namespace: [CSI_DRIVER_NAMESPACE]/namespace: [CSM_NAMESPACE]/' | kubectl create -f -` - #### PowerStore + ### PowerStore 1. Copy the config Secret from the CSI PowerStore namespace into the CSM for Observability namespace: `kubectl get secret powerstore-config -n [CSI_DRIVER_NAMESPACE] -o yaml | sed 's/namespace: [CSI_DRIVER_NAMESPACE]/namespace: [CSM_NAMESPACE]/' | kubectl create -f -` + ### PowerScale + + 1. Copy the config Secret from the CSI PowerScale namespace into the CSM for Observability namespace: + + `kubectl get secret isilon-creds -n [CSI_DRIVER_NAMESPACE] -o yaml | sed 's/namespace: [CSI_DRIVER_NAMESPACE]/namespace: [CSM_NAMESPACE]/' | kubectl create -f -` + + If [CSM for Authorization is enabled](../../../authorization/deployment/#configuring-a-dell-csi-driver-with-csm-for-authorization) for CSI PowerScale, perform the following steps: + + 2. Copy the driver configuration parameters ConfigMap from the CSI PowerScale namespace into the CSM for Observability namespace: + + `kubectl get configmap isilon-config-params -n [CSI_DRIVER_NAMESPACE] -o yaml | sed 's/namespace: [CSI_DRIVER_NAMESPACE]/namespace: [CSM_NAMESPACE]/' | kubectl create -f -` + + 3. Copy the `karavi-authorization-config`, `proxy-server-root-certificate`, `proxy-authz-tokens` Secret from the CSI PowerScale namespace into the CSM for Observability namespace: + + `kubectl get secret karavi-authorization-config proxy-server-root-certificate proxy-authz-tokens -n [CSI_DRIVER_NAMESPACE] -o yaml | sed 's/namespace: [CSI_DRIVER_NAMESPACE]/namespace: [CSM_NAMESPACE]/' | sed 's/name: karavi-authorization-config/name: isilon-karavi-authorization-config/' | sed 's/name: proxy-server-root-certificate/name: isilon-proxy-server-root-certificate/' | sed 's/name: proxy-authz-tokens/name: isilon-proxy-authz-tokens/' | kubectl create -f -` + + + 5. Configure the [parameters](#configuration) and install the CSM for Observability Helm Chart A default values.yaml file is located [here](https://github.com/dell/helm-charts/blob/main/charts/karavi-observability/values.yaml) that can be used for installation. This can be copied into a file named `myvalues.yaml` and either used as is or modified accordingly. @@ -51,6 +70,7 @@ The Container Storage Modules (CSM) for Observability Helm chart bootstraps an O __Note:__ - The default `values.yaml` is configured to deploy the CSM for Observability Topology service on install. - If CSM for Authorization is enabled for CSI PowerFlex, the `karaviMetricsPowerflex.authorization` parameters must be properly configured in your values file for CSM Observability. + - If CSM for Authorization is enabled for CSI PowerScale, the `karaviMetricsPowerscale.authorization` parameters must be properly configured in your values file for CSM Observability. ```console $ helm install karavi-observability dell/karavi-observability -n [CSM_NAMESPACE] -f myvalues.yaml @@ -106,7 +126,7 @@ The following table lists the configurable parameters of the CSM for Observabili | `karaviMetricsPowerstore.collectorAddr` | Metrics Collector accessible from the Kubernetes cluster | `otel-collector:55680` | | `karaviMetricsPowerstore.provisionerNames` | Provisioner Names used to filter for determining PowerStore volumes (must be a Comma-separated list) | `csi-powerstore.dellemc.com` | | `karaviMetricsPowerstore.volumePollFrequencySeconds` | The polling frequency (in seconds) to gather volume metrics | `10` | -| `karaviMetricsPowerstore.concurrentPowerflexQueries` | The number of simultaneous metrics queries to make to PowerStore (must be less than 10; otherwise, several request errors from PowerStore will ensue.) | `10` | +| `karaviMetricsPowerstore.concurrentPowerstoreQueries` | The number of simultaneous metrics queries to make to PowerStore (must be less than 10; otherwise, several request errors from PowerStore will ensue.) | `10` | | `karaviMetricsPowerstore.volumeMetricsEnabled` | Enable PowerStore Volume Metrics Collection | `true` | | `karaviMetricsPowerstore.endpoint` | Endpoint for pod leader election | `karavi-metrics-powerstore` | | `karaviMetricsPowerstore.service.type` | Kubernetes service type | `ClusterIP` | @@ -115,3 +135,23 @@ The following table lists the configurable parameters of the CSM for Observabili | `karaviMetricsPowerstore.zipkin.uri` | URI of a Zipkin instance where tracing data can be forwarded | | | `karaviMetricsPowerstore.zipkin.serviceName` | Service name used for Zipkin tracing data | `metrics-powerstore`| | `karaviMetricsPowerstore.zipkin.probability` | Percentage of trace information to send to Zipkin (Valid range: 0.0 to 1.0) | `0` | +| `karaviMetricsPowerscale.image` | CSM Metrics for PowerScale Service image | `dellemc/csm-metrics-powerscale:v1.0`| +| `karaviMetricsPowerscale.enabled` | Enable CSM Metrics for PowerScale service | `true` | +| `karaviMetricsPowerscale.collectorAddr` | Metrics Collector accessible from the Kubernetes cluster | `otel-collector:55680` | +| `karaviMetricsPowerscale.provisionerNames` | Provisioner Names used to filter for determining PowerScale volumes (must be a Comma-separated list) | `csi-isilon.dellemc.com` | +| `karaviMetricsPowerscale.capacityMetricsEnabled` | Enable PowerScale capacity metric Collection | `true` | +| `karaviMetricsPowerscale.performanceMetricsEnabled` | Enable PowerScale performance metric Collection | `true` | +| `karaviMetricsPowerscale.clusterCapacityPollFrequencySeconds` | The polling frequency (in seconds) to gather cluster capacity metrics | `30` | +| `karaviMetricsPowerscale.clusterPerformancePollFrequencySeconds` | The polling frequency (in seconds) to gather cluster performance metrics | `20` | +| `karaviMetricsPowerscale.quotaCapacityPollFrequencySeconds` | The polling frequency (in seconds) to gather volume capacity metrics | `30` | +| `karaviMetricsPowerscale.concurrentPowerscaleQueries` | The number of simultaneous metrics queries to make to PowerScale(MUST be less than 10; otherwise, several request errors from PowerScale will ensue. | `10` | +| `karaviMetricsPowerscale.endpoint` | Endpoint for pod leader election | `karavi-metrics-powerscale` | +| `karaviMetricsPowerscale.service.type` | Kubernetes service type | `ClusterIP` | +| `karaviMetricsPowerscale.logLevel` | Output logs that are at or above the given log level severity (Valid values: TRACE, DEBUG, INFO, WARN, ERROR, FATAL, PANIC) | `INFO`| +| `karaviMetricsPowerscale.logFormat` | Output logs in the specified format (Valid values: text, json) | `text` | +| `karaviMetricsPowerscale.isiClientOptions.isiSkipCertificateValidation` | Skip OneFS API server's certificates | `true` | +| `karaviMetricsPowerscale.isiClientOptions.isiAuthType` | 0 to enable session-based Authentication; 1 to enables basic Authentication | `1` | +| `karaviMetricsPowerscale.isiClientOptions.isiLogVerbose` | Decide High/Medium/Low content of the OneFS REST API message | `0` | +| `karaviMetricsPowerscale.authorization.enabled` | [Authorization](../../../authorization) is an optional feature to apply credential shielding of the backend PowerScale. | `false` | +| `karaviMetricsPowerscale.authorization.proxyHost` | Hostname of the csm-authorization server. | | +| `karaviMetricsPowerscale.authorization.skipCertificateValidation` | A boolean that enables/disables certificate validation of the csm-authorization server. | | diff --git a/content/docs/observability/deployment/offline.md b/content/docs/observability/deployment/offline.md index b4c5ccd9d6..16b93f1bac 100644 --- a/content/docs/observability/deployment/offline.md +++ b/content/docs/observability/deployment/offline.md @@ -24,9 +24,9 @@ If one Linux system has both internet access and access to an internal registry, Preparing an offline bundle requires the following utilities: -| Dependency | Usage | -| --------------------- | ----- | -| `docker` | `docker` will be used to pull images from public image registries, tag them, and push them to a private registry.
Required on both the system building the offline bundle as well as the system preparing for installation.
Tested version is `docker` 18.09 +| Dependency | Usage | +|------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `docker` | `docker` will be used to pull images from public image registries, tag them, and push them to a private registry.
Required on both the system building the offline bundle as well as the system preparing for installation.
Tested version is `docker` 18.09+ | ### Executing the Installer @@ -72,10 +72,12 @@ To perform an offline installation of a Helm chart, the following steps should b * * Downloading and saving Docker images - dellemc/csm-topology:v0.3.0 - dellemc/csm-metrics-powerflex:v0.3.0 - otel/opentelemetry-collector:0.9.0 - nginxinc/nginx-unprivileged:1.18 + dellemc/csm-topology:v1.3.0 + dellemc/csm-metrics-powerflex:v1.3.0 + dellemc/csm-metrics-powerstore:v1.3.0 + dellemc/csm-metrics-powerscale:v1.3.0 + otel/opentelemetry-collector:0.42.0 + nginxinc/nginx-unprivileged:1.20 * * Compressing offline-karavi-observability-bundle.tar.gz @@ -103,10 +105,12 @@ To perform an offline installation of a Helm chart, the following steps should b * * Loading, tagging, and pushing Docker images to registry :5000/ - dellemc/csm-topology:v0.3.0 -> :5000/csm-topology:v0.3.0 - dellemc/csm-metrics-powerflex:v0.3.0 -> :5000/csm-metrics-powerflex:v0.3.0 - otel/opentelemetry-collector:0.9.0 -> :5000/opentelemetry-collector:0.9.0 - nginxinc/nginx-unprivileged:1.18 -> :5000/nginx-unprivileged:1.18 + dellemc/csm-topology:v1.3.0 -> :5000/csm-topology:v1.3.0 + dellemc/csm-metrics-powerflex:v1.3.0 -> :5000/csm-metrics-powerflex:v1.3.0 + dellemc/csm-metrics-powerstore:v1.3.0 -> :5000/csm-metrics-powerstore:v1.3.0 + dellemc/csm-metrics-powerscale:v1.3.0 -> :5000/csm-metrics-powerscale:v1.3.0 + otel/opentelemetry-collector:0.42.0 -> :5000/opentelemetry-collector:0.42.0 + nginxinc/nginx-unprivileged:1.20 -> :5000/nginx-unprivileged:1.20 ``` ### Perform Helm installation @@ -145,12 +149,28 @@ To perform an offline installation of a Helm chart, the following steps should b [user@anothersystem /home/user/offline-karavi-observability-bundle/helm]# kubectl get secret powerstore-config -n [CSI_DRIVER_NAMESPACE] -o yaml | sed 's/namespace: [CSI_DRIVER_NAMESPACE]/namespace: [CSM_NAMESPACE]/' | kubectl create -f - ``` -4. Now that the required images have been made available and the Helm chart's configuration updated with references to the internal registry location, installation can proceed by following the instructions that are documented within the Helm chart's repository. + CSI Driver for PowerScale: + ``` + [user@anothersystem /home/user/offline-karavi-observability-bundle/helm]# kubectl get secret isilon-creds -n [CSI_DRIVER_NAMESPACE] -o yaml | sed 's/namespace: [CSI_DRIVER_NAMESPACE]/namespace: [CSM_NAMESPACE]/' | kubectl create -f - + ``` + + If [CSM for Authorization is enabled](../../../authorization/deployment/#configuring-a-dell-csi-driver-with-csm-for-authorization) for CSI PowerScale, perform these steps: + + ``` + [user@anothersystem /home/user/offline-karavi-observability-bundle/helm]# kubectl get configmap isilon-config-params -n [CSI_DRIVER_NAMESPACE] -o yaml | sed 's/namespace: [CSI_DRIVER_NAMESPACE]/namespace: [CSM_NAMESPACE]/' | kubectl create -f - + ``` + + ``` + [user@anothersystem /home/user/offline-karavi-observability-bundle/helm]# kubectl get secret karavi-authorization-config proxy-server-root-certificate proxy-authz-tokens -n [CSI_DRIVER_NAMESPACE] -o yaml | sed 's/namespace: [CSI_DRIVER_NAMESPACE]/namespace: [CSM_NAMESPACE]/' | sed 's/name: karavi-authorization-config/name: isilon-karavi-authorization-config/' | sed 's/name: proxy-server-root-certificate/name: isilon-proxy-server-root-certificate/' | sed 's/name: proxy-authz-tokens/name: isilon-proxy-authz-tokens/' | kubectl create -f - + ``` + +4. Now that the required images have been made available and the Helm chart's configuration updated with references to the internal registry location, installation can proceed by following the instructions that are documented within the Helm chart's repository. **Note:** - Optionally, you could provide your own [configurations](../helm/#configuration). A sample values.yaml file is located [here](https://github.com/dell/helm-charts/blob/main/charts/karavi-observability/values.yaml). - The default `values.yaml` is configured to deploy the CSM for Observability Topology service on install. - If CSM for Authorization is enabled for CSI PowerFlex, the `karaviMetricsPowerflex.authorization` parameters must be properly configured. + - If CSM for Authorization is enabled for CSI PowerScale, the `karaviMetricsPowerscale.authorization` parameters must be properly configured. ``` [user@anothersystem /home/user/offline-karavi-observability-bundle/helm]# helm install -n install-namespace app-name karavi-observability diff --git a/content/docs/observability/deployment/online.md b/content/docs/observability/deployment/online.md index 60e83ef3a9..82524a658c 100644 --- a/content/docs/observability/deployment/online.md +++ b/content/docs/observability/deployment/online.md @@ -69,6 +69,8 @@ Options: --namespace[=] Namespace where Karavi Observability will be installed Optional --csi-powerflex-namespace[=] Namespace where CSI PowerFlex is installed, default is 'vxflexos' + --csi-powerstore-namespace[=] Namespace where CSI PowerStore is installed, default is 'csi-powerstore' + --csi-powerscale-namespace[=] Namespace where CSI PowerScale is installed, default is 'isilon' --set-file Set values from files used during helm installation (can be specified multiple times) --skip-verify Skip verification of the environment --values[=] Values file, which defines configuration values @@ -77,7 +79,7 @@ Options: --help Help ``` -__Note:__ CSM for Authorization currently does not support the Observability module for PowerStore. Therefore setting `enable-authorization` is not supported in this case. +__Note:__ CSM for Authorization currently does not support the Observability module for PowerStore. Therefore setting `enable-authorization` is not supported in this case. ### Executing the Installer @@ -101,6 +103,7 @@ To perform an online installation of CSM for Observability, the following steps __Note:__ - The default `values.yaml` is configured to deploy the CSM for Observability Topology service on install. - If CSM for Authorization is enabled for CSI PowerFlex, the `karaviMetricsPowerflex.authorization` parameters must be properly configured in `myvalues.yaml` for CSM Observability. + - If CSM for Authorization is enabled for CSI PowerScale, the `karaviMetricsPowerscale.authorization` parameters must be properly configured in `myvalues.yaml` for CSM Observability. ``` [user@system /home/user/karavi-observability/installer]# ./karavi-observability-install.sh install --namespace [CSM_NAMESPACE] --values myvalues.yaml diff --git a/content/docs/observability/design/_index.md b/content/docs/observability/design/_index.md index e6dc2b93c1..adb56abcb8 100644 --- a/content/docs/observability/design/_index.md +++ b/content/docs/observability/design/_index.md @@ -19,7 +19,10 @@ The following prerequisites must be deployed into the namespace where CSM for Ob - Prometheus for scraping the metrics from the OTEL collector. - Grafana for visualizing the metrics from Prometheus and Topology services using custom dashboards. -- CSM for Observability will use secrets to get details about the storage systems used by the CSI drivers. These secrets should be copied from the namespaces where the drivers are deployed. CSI Powerflex driver uses the 'vxflexos-config' secret and CSI PowerStore uses the 'powerstore-config' secret. +- CSM for Observability will use secrets to get details about the storage systems used by the CSI drivers. These secrets should be copied from the namespaces where the drivers are deployed. + - CSI PowerFlex driver uses the 'vxflexos-config' secret. + - CSI PowerStore driver uses the 'powerstore-config' secret. + - CSI PowerScale driver uses the 'isilon-creds' secret. ## Deployment Architectures diff --git a/content/docs/observability/metrics/powerscale.md b/content/docs/observability/metrics/powerscale.md new file mode 100644 index 0000000000..d06d168902 --- /dev/null +++ b/content/docs/observability/metrics/powerscale.md @@ -0,0 +1,45 @@ +--- +title: PowerScale Metrics +linktitle: PowerScale Metrics +weight: 1 +description: > + Dell Container Storage Modules (CSM) for Observability PowerScale Metrics +--- + +This section outlines the metrics collected by the Container Storage Modules (CSM) Observability module for PowerScale. The [Grafana reference dashboards](https://github.com/dell/karavi-observability/blob/main/grafana/dashboards/powerscale) for PowerScale metrics can be uploaded to your Grafana instance. + +## I/O Performance Metrics + +Storage system I/O performance metrics (IOPS, bandwidth) are available by default and broken down by cluster and quota. + +To disable these metrics, set the ```performanceMetricsEnabled``` field to false in helm/values.yaml. + +The following I/O performance metrics are available from the OpenTelemetry collector endpoint. Please see the [CSM for Observability](../../) for more information on deploying and configuring the OpenTelemetry collector. + +| Metric | Description | +|--------------------------------------------------------------------|-------------------------------------------------------------------------------------| +| powerscale_cluster_cpu_use_rate | Average CPU usage for all nodes in the monitored cluster | +| powerscale_cluster_disk_read_operation_rate | Average rate at which the disks in the cluster servicing data read change requests | +| powerscale_cluster_disk_write_operation_rate | Average rate at which the disks in the cluster servicing data write change requests | +| powerscale_cluster_disk_throughput_read_rate_megabytes_per_second | Throughput rate of data being read from the disks in the cluster | +| powerscale_cluster_disk_throughput_write_rate_megabytes_per_second | Throughput rate of data being written to the disks in the cluster | + +## Storage Capacity Metrics + +Provides visibility into the total, used, and available capacity for PowerScale cluster and quotas. + +To disable these metrics, set the ```capacityMetricsEnabled``` field to false in helm/values.yaml. + +The following storage capacity metrics are available from the OpenTelemetry collector endpoint. Please see the [CSM for Observability](../../) for more information on deploying and configuring the OpenTelemetry collector. + +| Metric | Description | +|---------------------------------------------------|------------------------------------------------------------------| +| powerscale_cluster_total_capacity_terabytes | Total cluster capacity (TB) | +| powerscale_cluster_remaining_capacity_terabytes | Total unused cluster capacity (TB) | +| powerscale_cluster_used_capacity_percentage | Percent of total cluster capacity that has been used | +| powerscale_cluster_total_hard_quota_gigabytes | Amount of total capacity allocated in all directory hard quotas | +| powerscale_cluster_total_hard_quota_percentage | Percent of total capacity allocated in all directory hard quotas | +| powerscale_volume_quota_subscribed_gigabytes | Space used of Quota for a directory (GB) | +| powerscale_volume_hard_quota_remaining_gigabytes | Unused spaced below the hard limit for a directory (GB) | +| powerscale_volume_quota_subscribed_percentage | Percentage of space used in hard limit for a directory | +| powerscale_volume_hard_quota_remaining_percentage | Percentage of the remaining space in hard limit for a directory | diff --git a/content/docs/observability/release/_index.md b/content/docs/observability/release/_index.md index 84a9c87ea2..5b78088474 100644 --- a/content/docs/observability/release/_index.md +++ b/content/docs/observability/release/_index.md @@ -6,14 +6,17 @@ Description: > Dell Container Storage Modules (CSM) release notes for observability --- -## Release Notes - CSM Observability 1.2.0 +## Release Notes - CSM Observability 1.3.0 ### New Features/Changes +- Support PowerScale in CSM Observability +- Add PV/PVC namespace mapping for CSM Observability metrics +- Rearrange columns for topology dashboard +- Add offline upgrade +- Upgrade Otel to 1.7.0 ### Fixed Issues -- [PowerStore Grafana dashboard does not populate correctly ](https://github.com/dell/csm/issues/279) -- [Grafana installation script - prometheus address is incorrect](https://github.com/dell/csm/issues/278) -- [prometheus-values.yaml error in json](https://github.com/dell/csm/issues/259) +- [Observability Topology: nil pointer error](https://github.com/dell/csm/issues/430) ### Known Issues \ No newline at end of file diff --git a/content/docs/observability/troubleshooting/_index.md b/content/docs/observability/troubleshooting/_index.md index 4c094c212d..7a5fbac6d7 100644 --- a/content/docs/observability/troubleshooting/_index.md +++ b/content/docs/observability/troubleshooting/_index.md @@ -171,7 +171,7 @@ sidecar: enabled: true ## Additional grafana server ConfigMap mounts -## Defines additional mounts with ConfigMap. CofigMap must be manually created in the namespace. +## Defines additional mounts with ConfigMap. ConfigMap must be manually created in the namespace. extraConfigmapMounts: [] ``` diff --git a/content/docs/observability/upgrade/_index.md b/content/docs/observability/upgrade/_index.md index a44d38c615..82145b14c6 100644 --- a/content/docs/observability/upgrade/_index.md +++ b/content/docs/observability/upgrade/_index.md @@ -55,7 +55,7 @@ CSM for Observability online installer upgrade can be used if the initial deploy ``` 2. Update `values.yaml` file as needed. Configuration options are outlined in the [Helm chart deployment section](../deployment/helm#configuration). -2. Execute the `./karavi-observability-install.sh` script: +3. Execute the `./karavi-observability-install.sh` script: ``` [user@system /home/user/karavi-observability/installer]# ./karavi-observability-install.sh upgrade --namespace $namespace --values myvalues.yaml --version $latest_chart_version --------------------------------------------------------------------------------- @@ -80,3 +80,42 @@ CSM for Observability online installer upgrade can be used if the initial deploy | |- Waiting for pods in namespace karavi to be ready Success ``` + +## Offline Installer Upgrade + +Assuming that you have already installed the Karavi Observability Helm Chart by offline installer and meet its installation requirement. +The following instructions can be followed when a Helm chart was installed and will be upgraded in an environment that does not have an internet connection and will be unable to download the Helm chart and related Docker images. + +1. Build the Offline Bundle + Follow [Offline Karavi Observability Helm Chart Installer](../deployment/offline) to build the latest bundle. + +2. Unpack the Offline Bundle + Follow [Offline Karavi Observability Helm Chart Installer](../deployment/offline), copy and unpack the Offline Bundle to another Linux system, and push Docker images to the internal Docker registry. + +3. Perform Helm upgrade + 1. Change directory to `helm` which contains the updated Helm chart directory: + ``` + [user@anothersystem /home/user/offline-karavi-observability-bundle]# cd helm + ``` + 2. Install necessary cert-manager CustomResourceDefinitions provided. + ``` + [user@anothersystem /home/user/offline-karavi-observability-bundle/helm]# kubectl apply --validate=false -f cert-manager.crds.yaml + ``` + 3. (Optional) Enable Karavi Observability for PowerFlex/PowerScale to use an existing instance of Karavi Authorization for accessing the REST API for the given storage systems. + **Note**: Assuming that if the Karavi Observability's Authorization has been enabled in the phase of [Offline Karavi Observability Helm Chart Installer](../deployment/offline), the Authorization Secrets/Configmap have been copied to the Karavi Observability namespace. + A sample configuration values.yaml file is located [here](https://github.com/dell/helm-charts/blob/main/charts/karavi-observability/values.yaml). + In your own configuration values.yaml, you need to enable PowerFlex/PowerScale Authorization, and provide the location of the sidecar-proxy Docker image and URL of the Karavi Authorization proxyHost address. + + 4. Now that the required images have been made available and the Helm chart's configuration updated with references to the internal registry location, installation can proceed by following the instructions that are documented within the Helm chart's repository. + **Note**: Assuming that Your Secrets from CSI Drivers have been copied to the Karavi Observability namespace in the phase of [Offline Karavi Observability Helm Chart Installer](../deployment/offline) + Optionally, you could provide your own [configurations](../deployment/helm/#configuration). A sample values.yaml file is located [here](https://github.com/dell/helm-charts/blob/main/charts/karavi-observability/values.yaml). + ``` + [user@anothersystem /home/user/offline-karavi-observability-bundle/helm]# helm upgrade -n install-namespace app-name karavi-observability + NAME: app-name + LAST DEPLOYED: Wed Aug 17 14:44:04 2022 + NAMESPACE: install-namespace + STATUS: deployed + REVISION: 1 + TEST SUITE: None + ``` + \ No newline at end of file