From c1166ff5cbc19ee19cb4f4ef5cd29dce2d923c18 Mon Sep 17 00:00:00 2001 From: Meg McRoberts Date: Wed, 11 Oct 2023 00:32:55 -0700 Subject: [PATCH] docs: how to migrate quality gates to Keptn Analysis feature (#2251) Signed-off-by: Meg McRoberts Co-authored-by: Florian Bacher Co-authored-by: odubajDT <93584209+odubajDT@users.noreply.github.com> --- .../en/docs/migrate/metrics-observe/_index.md | 334 ++++++++++++++++-- .../en/docs/migrate/strategy/_index.md | 52 ++- docs/content/en/docs/yaml-crd-ref/analysis.md | 5 + .../docs/yaml-crd-ref/analysisdefinition.md | 5 + .../yaml-crd-ref/analysisvaluetemplate.md | 5 + 5 files changed, 362 insertions(+), 39 deletions(-) create mode 100644 docs/content/en/docs/yaml-crd-ref/analysis.md create mode 100644 docs/content/en/docs/yaml-crd-ref/analysisdefinition.md create mode 100644 docs/content/en/docs/yaml-crd-ref/analysisvaluetemplate.md diff --git a/docs/content/en/docs/migrate/metrics-observe/_index.md b/docs/content/en/docs/migrate/metrics-observe/_index.md index ae92c71717..dc4fce4fd5 100644 --- a/docs/content/en/docs/migrate/metrics-observe/_index.md +++ b/docs/content/en/docs/migrate/metrics-observe/_index.md @@ -2,11 +2,31 @@ title: Migrate Quality Gates description: How to translate Quality Gates into Keptn Metrics and evaluations weight: 40 -hidechildren: false # this flag hides all sub-pages in the sidebar-multicard.html --- The SLIs and SLOs used for Keptn v1 quality gates can be ported to -Keptn KeptnMetrics and KeptnEvaluationDefintions. +appropriate Keptn facilities: + +* [Keptn Metrics](../../implementing/evaluatemetrics.md/) + allow you to define and view metrics + from multiple data sources in your Kubernetes cluster. +* Use + [Keptn Evaluations](../../implementing/evaluations.md) + to do a simple evaluation of the metrics data you capture. + To implement this, transfer the information from the Keptn v1 + [sli.yaml](https://keptn.sh/docs/1.0.x/reference/files/sli/) + and + [slo.yaml](https://keptn.sh/docs/1.0.x/reference/files/slo/) + files into + [KeptnEvaluationDefinition](../../yaml-crd-ref/evaluationdefinition.md) + resources. + +* Keptn v1 calculations that use weighting and scoring + can be converted to use the + [Keptn Analysis](../../implementing/slo) + feature. + Tools are provided to help with this conversion; + see below. By default, Keptn includes additional observability features that are not included in Keptn v1: @@ -14,39 +34,301 @@ that are not included in Keptn v1: * [DORA metrics](../../implementing/dora) * [OpenTelemetry observability](../../implementing/otel.md) -Keptn v1 Quality Gates can be migrated into Keptn metrics -and Keptn evaluations. +## Paradigm changes -> **Note** -The full SLO capabilities -provided by Keptn v1 such as weighting and scoring -have a first implementation in the [Analysis](../../implementing/slo/). - -Notice the paradigm differences when implementing Keptn evaluations: +The Keptn paradigm for evaluations and analyses +differs from that of Keptn v1 quality gates: * Data providers are installed and configured into your Kubernetes cluster using Helm charts and standard practices. +* Keptn supports multiple instances of multiple data providers. * You must populate a [KeptnMetricsProvider](../../yaml-crd-ref/metricsprovider.md) resource for each instance of each data provider. This resource specifies the URL and namespace for the data provider - and gives it a unique `name` that can be referenced by other resources. - This is the only Keptn specific configuration that is required. -* Each query that is specified in your Keptn v1 - [slo.yaml](https://keptn.sh/docs/1.0.x/reference/files/sli/) file - should be converted into its own - [KeptnMetric](../../yaml-crd-ref/metric.md) resource - if you are using it for an evaluation. - Note that Keptn supports using multiple instances of multiple data providers. -* Simple Keptn v1 comparisons that are defined in + and gives it a unique `name` that can be referenced + for Keptn Metrics (which are also used for Evaluations) and Analysis. +* Queries and objectives that are specified in your Keptn v1 + [sli.yaml](https://keptn.sh/docs/1.0.x/reference/files/sli/) + and [slo.yaml](https://keptn.sh/docs/1.0.x/reference/files/slo/) - files can be converted to + files are transferred/converted into Keptn resources. + +## Transfer Keptn v1 SLIs/SLOs to evaluation resources + +Simple comparisons of data can be implemented as +[Keptn Evaluations](../../implementing/evaluations.md). +To implement this: + +* Transfer the information from the Keptn v1 + [sli.yaml](https://keptn.sh/docs/1.0.x/reference/files/sli/) + files into + [KeptnMetric](../../yaml-crd-ref/metric.md) resources +* Transfer the information from the Keptn v1 + [slo.yaml](https://keptn.sh/docs/1.0.x/reference/files/slo/) + files into [KeptnEvaluationDefinition](../../yaml-crd-ref/evaluationdefinition.md) resources. -* Keptn v1 calculations that use weighting and scoring - cannot currently be implemented for Keptn - but this functionality is under development. -For more information about working with Keptn metrics, see the -[Keptn Metrics](../../implementing/evaluatemetrics.md) -page. +## Convert Keptn v1 SLIs/SLOs to Analysis resources + +The Keptn Analysis feature provides capabilities +similar to those of the Keptn v1 +[Quality Gates](https://keptn.sh/docs/1.0.x/define/quality-gates/) +feature +but it uses Kubernetes resources to define the analysis to be done +rather than the configuration files used for Keptn v1. +Tools are provided to convert Keptn v1 SLI and SLO definitions +to Keptn Analysis resources. + +These instructions assume that the same SLO file +has been used for all services in the project, +so only one `AnalysisDefinition` resource +(named `my-project-ad`) is created. +If your Keptn v1 project has multiple SLOs, +you need to create a separate `AnalysisDefinition` +with a unique name for each one. + +The process is: + +1. Convert the SLIs to `AnalysisValueTemplates` resources + + The following command sequence converts a Keptn v1 + [sli.yaml](https://keptn.sh/docs/1.0.x/reference/files/sli/) + file to a Keptn + [AnalysisValueTemplate](../../yaml-crd-ref/analysisvaluetemplate.md) + resource: + + ```shell + METRICS_OPERATOR_IMAGE=ghcr.io/keptn/metrics-operator:v0.8.2 + PATH_TO_SLI=sli.yaml + KEPTN_PROVIDER_NAME=my-prometheus-provider + KEPTN_PROVIDER_NAMESPACE=keptn-lifecycle-poc + + docker run -v .:/mydata $METRICS_OPERATOR_IMAGE \ + --convert-sli=mydata/$PATH_TO_SLI \ + --keptn-provider-name=$KEPTN_PROVIDER_NAME \ + --keptn-provider-namespace=$KEPTN_PROVIDER_NAMESPACE > analysis-value-template.yaml + ``` + + This command creates an `AnalysisValueTemplate` resource + for each SLI that is defined in the `sli.yaml` file. + + > **Note** + The `name` of each `AnalysisValueTemplate` resource + must adhere to the + [Kubernetes resource naming rules](https://kubernetes.io/docs/concepts/overview/working-with-objects/names/). + The conversion tools preserve the name of each SLI + but modify the names to match the Kubernetes requirements. + For example, the `Response_Time` SLI + generates an `AnalysisValueTemplate` that is named `response-time`. + > + + All the SLIs for a particular `sli.yaml` file + are defined in a file called `analysis-value-template.yaml`. + Apply this file to your cluster with a command like the following. + Be sure to specify the namespace; + if you omit it, the yaml file is applied to the default namespace. + + ```shell + kubectl apply -f analysis-value-template.yaml -n keptn-lifecycle-poc + ``` + +1. Convert the SLO to an `AnalysisDefinition` resource: + + The process of converting the Keptn v1 + [slo.yaml](https://keptn.sh/docs/1.0.x/reference/files/slo/) + files to + [AnalysisDefinition](../../yaml-crd-ref/analysisdefinition.md) + resources is similar to the process of converting the SLIs. + Use the following command sequence: + + ```shell + METRICS_OPERATOR_IMAGE=ghcr.io/keptn/metrics-operator:v0.8.2 + PATH_TO_SLO=slo.yaml + ANALYSIS_VALUE_TEMPLATE_NAMESPACE=keptn-lifecycle-poc + ANALYSIS_DEFINITION_NAME=my-project-ad + + docker run -v $(pwd):/mydata $METRICS_OPERATOR_IMAGE \ + --convert-slo=/mydata/$PATH_TO_SLO \ + --analysis-value-template-namespace=$ANALYSIS_VALUE_TEMPLATE_NAMESPACE \ + --analysis-definition-name=$ANALYSIS_DEFINITION_NAME > analysis-definition.yaml + ``` + + The result of this command yields an `AnalysisDefinition` resource + that is defined in a file called `analysis-definition.yaml`. + Apply this to your Keptn cluster with the following command. + Be sure to add the namespace; + if you omit it, the yaml file is applied to the default namespace. + + ```shell + kubectl apply -f analysis-definition.yaml -n keptn-lifecycle-poc + ``` + +1. Create a `KeptnMetricsProvider` resource + + A [KeptnMetricsProvider](../../yaml-crd-ref/metricsprovider.md) + resource configures the data provider from which the values + for the `AnalysisValueTemplate` resource are fetched. + This same resource is used for any metrics and evaluations you are using. + Note that Keptn supports multiple instances of multiple data providers + and each `AnalysisValueTemplate` resource + can reference a different provider, + + The `KeptnMetricsProvider` resource defines: + + * The URL of the server used for your data provider + * The namespace where the data provider is located + * (optional) Secret information to access a protected data source + + See the `prometheus-provider` file for an example. + +1. Define the Analysis resource to run + + Create a yaml file (such as `analysis-instance.yaml`) + to populate the + [Analysis](../../yaml-crd-ref/analysis.md) + resource that defines the specific analysis you want to run. + Specify the following: + + * Name of the `AnalysisDefinition` resource that defines the goals + * Timeframe for which to retrieve the metrics. + The `from`/`to` timestamps are added automatically + to the query defined in the `AnalysisValueTemplate` resource. + * (optional) List of `args` that specify values for this analysis + that replace variables used in the queries + in the `AnalysisValueTemplate` resources. + See + [Passing contextual arguments to the Analysis](#passing-contextual-arguments-to-the-analysis) + for more information. + +1. Run the analysis + + To perform an Analysis (or "trigger an evaluation" in Keptn v1 jargon), + apply the `analysis-instance.yaml` file: + + ```shell + kubectl apply -f analysis-instance.yaml -n keptn-lifecycle-poc + ``` + + Retrieve the current status of the Analysis with the following command: + + ```shell + kubectl get analysis -n keptn-lifecycle-poc + ``` + + This yields an output that looks like the following: + + ```shell + NAME ANALYSISDEFINITION WARNING PASS + analysis-sample-1 my-project-ad true + ``` + + This shows that the analysis passed successfully. + + To get the detailed result of the evaluation, + use the `-oyaml` argument to inspect the full state of the analysis: + + This displays the `Analysis` resource + with the definition of the analysis + as well as the `status` (results) of the analysis; for example: + + ```shell + kubectl get analysis -n keptn-lifecycle-poc -oyaml + ``` + + ```yaml + apiVersion: v1 + items: + - apiVersion: metrics.keptn.sh/v1alpha3 + kind: Analysis + metadata: + creationTimestamp: "2023-09-14T11:00:01Z" + generation: 4 + name: analysis-sample-1 + namespace: keptn-lifecycle-poc + resourceVersion: "71327" + uid: 1c5e043d-ed5e-42f8-ba32-b7af54b55c35 + spec: + analysisDefinition: + name: my-project-ad + namespace: keptn-lifecycle-poc + args: + ns: keptn-lifecycle-toolkit-system + project: my-project + timeframe: + from: "2023-09-14T11:20:19Z" + to: "2023-09-14T11:22:19Z" + status: + pass: true + raw: '{"objectiveResults":[{"result":{"failResult":{"operator":{"greaterThan":{"fixedValue":"50"}}},"warnResult":{"operator":{"greaterThan":{"fixedValue":"50"}}},"pass":true},"value":7,"score":1}],"totalScore":1,"maximumScore":1,"pass":true}' + kind: List + metadata: + resourceVersion: "" + ``` + + As can be seen in the yaml above, + the `status.raw` property contains the detailed breakdown + of the analysis goals, and whether or not they passed. + + The result of this analysis stays in the cluster + until the `Analysis` is deleted. + That also means that, if another analysis should be performed, + the new analysis must be given a new, unique name within the namespace. + +### Passing contextual arguments to the Analysis + +In some cases, especially when migrating from Keptn v1, +you may want to do an analysis for different services within a project +and adapt the query for fetching the metric values, +based on which service/stage you are evaluating. +Keptn enables you to do this by using variables in the query defined in the +`AnalysisValueTemplates` resource +and arguments that are defined in the `Analysis` resource +for the specific analysis being run. + +For example, you may have a query +for retrieving the response time of a service, +which is identified by its label in Prometheus. +In this case, use the go templating syntax +to insert a variable as a placeholder +(for example, in this case, `{{.service}}`) +for the service identifier in the prometheus query: + +```yaml +apiVersion: metrics.keptn.sh/v1alpha3 +kind: AnalysisValueTemplate +metadata: + creationTimestamp: null + name: response-time +spec: + provider: + name: my-prometheus-provider + namespace: keptn-lifecycle-poc + query: response_time{label="{{.service}}"} +``` + +Then, if an analysis for that particular service should be performed, +the name of the service can be passed to the analysis +using the `spec.args` property from the `Analysis` resource: + +```yaml +apiVersion: metrics.keptn.sh/v1alpha3 +kind: Analysis +metadata: + name: analysis-sample-1 + namespace: keptn-lifecycle-poc +spec: + timeframe: + from: 2023-09-14T11:20:19Z + to: 2023-09-14T11:22:19Z + args: + "service": "my-service" + analysisDefinition: + name: my-project-ad + namespace: keptn-lifecycle-poc +``` + +This way, you can use the same `AnalysisDefinition` +and `AnalysisValueTemplates` for multiple services within the same project. +~ diff --git a/docs/content/en/docs/migrate/strategy/_index.md b/docs/content/en/docs/migrate/strategy/_index.md index 3530488e26..2a8b1f496b 100644 --- a/docs/content/en/docs/migrate/strategy/_index.md +++ b/docs/content/en/docs/migrate/strategy/_index.md @@ -277,23 +277,49 @@ and multiple instances of each data provider for your SLIs whereas Keptn v1 only allows you to use one SLI per project. The queries defined for the Keptn v1 SLIs -should be translated into -[KeptnMetric](../../yaml-crd-ref/metric.md) -resources. +should be translated into an appropriate Keptn resource: + +* [KeptnMetric](../../yaml-crd-ref/metric.md) +resources + to do + [evaluations](../../implementing/evaluations.md) + with + [KeptnEvaluationDefinition](../../yaml-crd-ref/evaluationdefinition.md) + resources. +* [AnalysisValueTemplate](../../yaml-crd-ref/analysisvaluetemplate.md) + resources to do + [analyses](../../implementing/slo) + with + [AnalysisDefinition](../../yaml-crd-ref/analysisdefinition.md) + resources. + Tools are provided to convert Keptn v1 SLIs and SLOs + to Keptn resources; see + [Convert Keptn v1 SLIs/SLOs to Analysis resources](../metrics-observe/#convert-keptn-v1-slisslos-to-analysis-resources) + for more information. ### SLOs -Keptn at this time does not support the full range -of Quality Gates evaluations that are represented by +Keptn v1 [SLOs](https://keptn.sh/docs/1.0.x/reference/files/slo/). -Facilities such as weighting of SLIs and scoring of the evaluation -do not currently exist. -This functionality is under development; see -[Epic 1646](https://github.com/keptn/lifecycle-toolkit/issues/1646). - -However, simple evaluations of an SLI can be defined as -[KeptnEvaluationDefinition](../../yaml-crd-ref/evaluationdefinition.md) -resources. +can be implemented on Keptn as evaluations or analyses: + +* Simple evaluations of an SLI can be implemented as + [Evaluations](../../implementing/evaluations.md) + which are defined as + [KeptnEvaluationDefinition](../../yaml-crd-ref/evaluationdefinition.md) + resources. + +* Complex analyses that use weighting and scoring + and analyze the value over a specified time frame + can be implemented as + [Analyses](../../implementing/slo) + that are defined in + [AnalysisDefinition](../../yaml-crd-ref/analysisdefinition.md) + resources. + Tools are provided to convert Keptn v1 SLIs and SLOs + to Keptn resources; see + [Convert Keptn v1 SLIs/SLOs to Analysis resources](../metrics-observe/#convert-keptn-v1-slisslos-to-analysis-resources) + for more information. ### Remediation diff --git a/docs/content/en/docs/yaml-crd-ref/analysis.md b/docs/content/en/docs/yaml-crd-ref/analysis.md new file mode 100644 index 0000000000..e6916b6789 --- /dev/null +++ b/docs/content/en/docs/yaml-crd-ref/analysis.md @@ -0,0 +1,5 @@ +--- +title: Analysis +description: Define specific configurations and the Analysis to report +weight: 4 +--- diff --git a/docs/content/en/docs/yaml-crd-ref/analysisdefinition.md b/docs/content/en/docs/yaml-crd-ref/analysisdefinition.md new file mode 100644 index 0000000000..92ae2cf1f5 --- /dev/null +++ b/docs/content/en/docs/yaml-crd-ref/analysisdefinition.md @@ -0,0 +1,5 @@ +--- +title: AnalysisDefinition +description: Define SLOs for an Analysis +weight: 6 +--- diff --git a/docs/content/en/docs/yaml-crd-ref/analysisvaluetemplate.md b/docs/content/en/docs/yaml-crd-ref/analysisvaluetemplate.md new file mode 100644 index 0000000000..62b960e669 --- /dev/null +++ b/docs/content/en/docs/yaml-crd-ref/analysisvaluetemplate.md @@ -0,0 +1,5 @@ +--- +title: AnalysisValueTemplate +description: Define the data source and query for each SLI +weight: 8 +---