Skip to content

Commit

Permalink
revise based on comments
Browse files Browse the repository at this point in the history
  • Loading branch information
caozhuozi committed Mar 15, 2024
1 parent c12ffb6 commit 131b8da
Show file tree
Hide file tree
Showing 5 changed files with 118 additions and 53 deletions.
77 changes: 77 additions & 0 deletions site/content/en/docs/user/cel-expressions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
---
title: "CEL Expressions in `kwok`"
---

# Notes on CEL Expressions in `kwok`

The page provides a concise note on writing CEL expressions in `kwok` CRs.

Below is the list of all CRs in `kwok` that contains CEL based fields.
* [Metric]
* [ResourceUsage]
* [ClusterResourceUsage]


You must follow [the CEL language specification] when writing the expressions.
For predefined functions of CEL, please refer to [CEL predefined functions].

Besides the built-in functions, `kwok` also provides some customized extension functions.
An exhaustive list of all the extension functions with their usages is given below.

* `Now()`: takes no parameters and returns the current timestamp.
* `Rand()`: takes no parameters and returns a random `float64` value.
* `SinceSecond()` returns the seconds elapsed since a given resource (`pod` or `node`) was created.
For example: `SinceSecond(pod)`, `node.SinceSecond(node)`.
* `UnixSecond()` returns the Unix time of a given time of type `time.Time`.
For example: , `UnixSecond(Now())`, `UnixSecond(node.metadata.creationTimestamp)`.
* `Quantity()` returns a float64 value of a given Quantity value. For example: `Quantity("100m")`, `Quantity("10Mi")`.
* `Usage()` returns the current instantaneous resource usage with the simulation data in [ResourceUsage (ClusterResourceUsage)].
For example: `Usage(pod, "memory")`, `Usage(node, "memory")`, `Usage(pod, "memory", container.name)` return the
current working set of a resource (pod, node or container) in bytes.
* `CumulativeUsage()` returns the cumulative resource usage in seconds with the simulation data given in [ResourceUsage (ClusterResourceUsage)].
For example: `CumulativeUsage(pod, "cpu")`, `CumulativeUsage(node, "cpu")`, `CumulativeUsage(pod, "cpu", container.name)`
return a cumulative cpu time consumed by a resource (pod, node or container) in core-seconds.

Additionally, `kwok` provides three special CEL variables `node`, `pod`, and `container` that could be used
in the expressions.
The three variables are set to the corresponding node, pod, container resource object respectively and users can
reference any nested fields of the resource objects simply via the CEL field selection expression (`e.f` format).
For example, you could use expression `node.metadata.name` to obtain the node name.

{{< hint "info" >}}

The functions with at least one parameter can be called in a receiver call-style.
That is, a function call like `f(e1, e2)` can also be called in style `e1.f(e2)`. For example, you can use `pod.Usage("memory")`
as an alternative to `Usage(pod, "memory")`.

{{< /hint >}}


It is worth noting that the use of some extension functions is restricted to specific CRs and contexts in the sense
that they are not generic but designed for special evaluating tasks.
The detailed limitations are described below.

## Functions Limitation

Function `Usage()` and `CumulativeUsage()` can only be used in the Metric resource.
For other functions listed above, users are also allowed to use them in ResourceUsage and ClusterResourceUsage
to build dynamic resource usage patterns.

The reason behind is that when `kwok` evaluates functions `Usage()` or `CumulativeUsage()`,
it actually takes the simulation data given in ResourceUsage and ClusterResourceUsage to obtain metric values.
Therefore, please ensure that the associated ResourceUsage or ClusterResourceUsage with the needed resource types
(cpu or memory) are also provided when using function `Usage()` and `CumulativeUsage()`.

## Variables Limitation

When using the three special CEL variables `node`, `pod`, and `container` in Metric resource, you should follow the below rules.
* When `dimension` is `node`: only `node` variable can be used.
* When `dimension` is `pod`: only `node`, `pod` can be used.
* When `dimension` is `container`: `node`, `pod`, `container` all can be used.


[Metric]: {{< relref "/docs/generated/apis" >}}#kwok.x-k8s.io/v1alpha1.Metric
[ResourceUsage]: {{< relref "/docs/generated/apis" >}}#kwok.x-k8s.io/v1alpha1.ResourceUsage
[ClusterResourceUsage]: {{< relref "/docs/generated/apis" >}}#kwok.x-k8s.io/v1alpha1.ClusterResourceUsage
[the CEL language specification]: https://github.com/google/cel-spec/blob/master/doc/langdef.md
[CEL predefined functions]: https://github.com/google/cel-spec/blob/master/doc/langdef.md#list-of-standard-definitions
26 changes: 26 additions & 0 deletions site/content/en/docs/user/go-template.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
---
title: "Go Template in `kwok`"
---

# Notes on Go Template in `kwok`


The page provides a concise note on writing go templates in kwok CRs.


Currently, only `Stage` CR has go template based fields (`Spec.Next.StatusTemplate`).


You must follow [the go text template syntax] when writing the templates.
For predefined functions of go text template, please refer to [go text template functions].
Besides the built-in functions, `kwok` also supports [sprig template functions].

It is worth noting that the "context" (which is denoted by the period character `.` ) to a template in `kwok` is set to the
referenced Kubernetes resource.
For example, you can use `.metadata.name` in a template to obtain the corresponding Kubernetes resource name.



[the go text template syntax]: https://pkg.go.dev/text/template
[go text template functions]: https://pkg.go.dev/text/template#hdr-Functions
[sprig template functions]: https://masterminds.github.io/sprig/
53 changes: 6 additions & 47 deletions site/content/en/docs/user/metrics-configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,10 @@ With `{nodeName}`, a single `path` is able to differentiate the metric data from


The `metrics` field are used to customize the return body of the installed metrics endpoint.

The descriptions of each sub-field are available at [Metric API][Metric].
For readers' convenience, we also mirror the documents here with some additional notes.

`metrics` is a list of specific configuration items, with each corresponding to a Prometheus style metric:
* `name` defines the metric name.
* `labels` defines the metric labels, with each item corresponding to a specific metric label.
Expand All @@ -73,7 +77,7 @@ The `metrics` field are used to customize the return body of the installed metri
* `kind` defines the type of the metric: `counter`, `guage` or `histogram`.
* `dimension` defines where the data comes from. It could be `node`, `pod`, or `container`.
* `value` is a CEL expression that defines the metric value if `kind` is `counter` or `guage`.
Please refer to [built-in CEL extension functions] for an exhausted list that you can use to simulate the metric value.
Please refer to [CEL expressions in `kwok`] for more detailed instructions that might be helpful to simulate the metric value.
* `buckets` is exclusively for customizing the data of the metric of kind `histogram`.
- `le`, which defines the histogram bucket’s upper threshold, has the same meaning as the one of Prometheus histogram bucket.
That is, each bucket contains values less than or equal to `le`.
Expand All @@ -84,51 +88,6 @@ The `metrics` field are used to customize the return body of the installed metri
Please refer to [Metrics for kubelet's "metrics/resource" endpoint][metrics resource endpoint] for a detailed example.


## built-in CEL extension functions

As `kwok` doesn't actually run containers, to simulate the resource usage data, `kwok` provides the following CEL
extension functions you can use in a CEL expression to help calculate the simulation data flexibly and dynamically:
* `Now()`: returns the current timestamp.
* `Rand()`: returns a random `float64` value.
* `SinceSecond()` returns the seconds elapsed since a resource (pod or node) was created.
For example: `pod.SinceSecond()`, `node.SinceSecond()`.
* `UnixSecond()` returns the Unix time of a given time of type `time.Time`.
For example: , `UnixSecond(Now())`, `UnixSecond(node.metadata.creationTimestamp)`.
* `Quantity()` returns a float64 value of a given Quantity value. For example: `Quantity("100m")`, `Quantity("10Mi")`.
* `Usage()` returns the current instantaneous resource usage with the simulation data in [ResourceUsage (ClusterResourceUsage)].
For example: `Usage(pod, "memory")`, `Usage(node, "memory")`, `Usage(pod, "memory", container.name)` return the
current working set of a resource (pod, node or container) in bytes.
* `CumulativeUsage()` returns the cumulative resource usage in seconds with the simulation data given in [ResourceUsage (ClusterResourceUsage)].
For example: `CumulativeUsage(pod, "cpu")`, `CumulativeUsage(node, "cpu")`, `CumulativeUsage(pod, "cpu", container.name)`
return a cumulative cpu time consumed by a resource (pod, node or container) in core-seconds.


`node`, `pod`, `container` are three special parameters that can be used only when `dimension` is set to the corresponding value.
You should follow the below rules to use the three parameters:
* When dimension is `node`: only `node` parameter can be used.
* When dimension is `pod`: only `node`, `pod` can be used.
* When dimension is `container`: `node`, `pod`, `container` all can be used.


The functions that have more one parameter can be called in a receiver call-style.
That is, a function call like `f(e1, e2)` can also be called in style `e1.f(e2)`. For example, you can use `pod.Usage("memory")`
as an alternative to `Usage(pod, "memory")`.


{{< hint "warning" >}}

Function `Usage()` and `CumulativeUsage()` can only be used in the Metric resource.
For other functions listed above, users are also allowed to use them in ResourceUsage and ClusterResourceUsage
to build dynamic resource usage patterns.

The reason behind is that when `kwok` evaluates functions `Usage()` or `CumulativeUsage()`,
it actually takes the simulation data given in ResourceUsage and ClusterResourceUsage to obtain metric values.
Therefore, please ensure that the associated ResourceUsage or ClusterResourceUsage with the needed resource types
(cpu or memory) are also provided when using function `Usage()` and `CumulativeUsage()`.

{{< /hint >}}


## Out-of-box Metric Config

`kwok` currently provides the [Metrics config][metrics resource endpoint] that is capable of
Expand All @@ -140,6 +99,6 @@ nodes managed by `kwok`.

[configuration]: {{< relref "/docs/user/configuration" >}}
[Metrics]: {{< relref "/docs/generated/apis" >}}#kwok.x-k8s.io/v1alpha1.Metrics
[built-in CEL extension functions]: {{< relref "/docs/user/metrics-configuration" >}}#built-in-cel-extension-functions
[CEL expressions in `kwok`]: {{< relref "/docs/user/cel-expression" >}}
[metrics resource endpoint]: https://github.com/kubernetes-sigs/kwok/blob/main/kustomize/metrics/resource
[ResourceUsage (ClusterResourceUsage)]: {{< relref "/docs/user/resource-usage-configuration" >}}
9 changes: 5 additions & 4 deletions site/content/en/docs/user/resource-usage-configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,8 @@ spec:
expression: <string>
```
To associate a ResourceUsage with a certain pod to be simulated, users must ensure `metadata.name` and `metadata.namespace` are inconsistent with the name and namespace of the target pod.
To associate a ResourceUsage with a certain pod to be simulated, users must ensure `metadata.name` and `metadata.namespace`
are inconsistent with the name and namespace of the target pod.

The resource usages of a pod are specified via `usages` field.
The `usages` field are organized by groups, with each corresponding to a collection of containers that shares a same resource usage simulation setting.
Expand Down Expand Up @@ -66,14 +67,14 @@ yields memory usage that grows linearly with time.
```yaml
expression: (pod.SinceSecond() / 60.0) * Quantity("1Mi")
```
Please refer to [built-in CEL extension functions] for an exhausted list that may be helpful to configure dynamic resource usage.
Please refer to [CEL expressions in `kwok`] for an exhausted list that may be helpful to configure dynamic resource usage.


### ClusterResourceUsage

In addition to simulating a single pod, users can also simulate the resource usage for multiple pods via [ClusterResourceUsage].

A ClusterResourceUsage resource has the following fields:
The YAML below shows all the fields of a ClusterResourceUsage resource:

``` yaml
kind: ClusterResourceUsage
Expand Down Expand Up @@ -131,4 +132,4 @@ ResourceUsage or ClusterResourceUsage only takes effect when the [Metric] featur
[Metric]: {{< relref "/docs/user/metrics-configuration" >}}
[the default Metric resource]: https://github.com/kubernetes-sigs/kwok/blob/main/kustomize/metrics/resource
[pod resource usage from annotation]: https://github.com/kubernetes-sigs/kwok/blob/main/kustomize/metrics/usage/usage-from-annotation.yaml
[built-in CEL extension functions]: {{< relref "/docs/user/metrics-configuration" >}}#built-in-cel-extension-functions
[CEL expressions in `kwok`]: {{< relref "/docs/user/cel-expressions" >}}
6 changes: 4 additions & 2 deletions site/content/en/docs/user/stages-configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,8 +65,9 @@ users can specify the conditions that need to be met for the stage to be applied
and the changes that will be made to the resource when the stage is applied.
The `next` field allows users to define the new status of the resource using the `statusTemplate` field, and even `delete` the resource.

`statusTemplate` and `delete` are the two fundamental fields in `next` that respectively represent the two basic phases
of resource lifecycle simulation: status update and resource deletion.
`statusTemplate` and `delete` are the two fundamental fields in `next` that respectively represent the two basic phases of resource lifecycle simulation: status update and resource deletion.
`statusTemplate` provides a way to define resource status based on go template rendering. Please see
[go template in `kwok`] for more detailed instructions.
`delete: true` has higher priority than a non-empty `statusTemplate`, which means `kwok` will delete the resource
rather than update its status if both are set.

Expand Down Expand Up @@ -257,3 +258,4 @@ This example shows how to configure the simplest and fastest stages of Pod resou
[Stage API]: {{< relref "/docs/generated/apis" >}}#kwok.x-k8s.io/v1alpha1.Stage
[Resource Lifecycle Simulation Controller]: {{< relref "/docs/design/architecture" >}}
[How Delay is Calculated]: {{< relref "/docs/user/stages-configuration#how-delay-is-calculated" >}}
[go template in `kwok`]: {{< relref "/docs/user/go-template" >}}

0 comments on commit 131b8da

Please sign in to comment.