revise based on comments

demo/resource-usage.demo

@@ -4,24 +4,20 @@ kwokctl create cluster --enable-metrics-server -c ./kustomize/metrics/resource/m
 kwokctl scale node --replicas 2
 kwokctl scale pod --replicas 8
 
-# Wait for a while to let the metrics server collect the metrics
-sleep 30
+# Wait for a while to let the metrics server collect the metrics.
+sleep 45
 
-# Now we can check the metrics
+# Now we can check the metrics.
 kubectl top node
 kubectl top pod
 
-# I don't know why the first result is always empty
-# let's try again
-kubectl top pod
-
-# Let's add some usage metrics to the pods
-kubectl patch pod pod-000000 --type=json -p='[{"op": "add", "path": "/metadata/annotations", "value": {"kwok.x-k8s.io/usage-cpu": "10000m", "kwok.x-k8s.io/usage-memory": "10000Mi"}}]'
+# Let's add some usage metrics to the pods.
+kubectl patch pod pod-000000 --type=json -p='[{"op":"add","path":"/metadata/annotations","value":{"kwok.x-k8s.io/usage-cpu":"10000m","kwok.x-k8s.io/usage-memory":"10000Mi"}}]'
 
-# Wait for a while to let the metrics server collect the metrics
-sleep 30
+# Wait for a while to let the metrics server collect the metrics.
+sleep 15
 
-# Now we can check the metrics
+# Now we can check the metrics again.
 kubectl top node
 kubectl top pod
 

site/content/en/docs/user/cel-expressions.md

@@ -0,0 +1,77 @@
+---
+title: "CEL Expressions in `kwok`"
+---
+
+# Notes on CEL Expressions in `kwok`
+
+The page provides a concise note on writing CEL expressions in `kwok` CRs.  
+
+Below is the list of all CRs in `kwok` that contains CEL based fields.
+* [Metric]
+* [ResourceUsage]
+* [ClusterResourceUsage]
+
+
+You must follow [the CEL language specification] when writing the expressions.
+For predefined functions of CEL, please refer to [CEL predefined functions].
+
+Besides the built-in functions, `kwok` also provides some customized extension functions.
+An exhaustive list of all the extension functions with their usages is given below.
+
+* `Now()`: takes no parameters and returns the current timestamp.
+* `Rand()`: takes no parameters and returns a random `float64` value.
+* `SinceSecond()` returns the seconds elapsed since a given resource (`pod` or `node`) was created.
+  For example: `SinceSecond(pod)`, `node.SinceSecond(node)`.
+* `UnixSecond()` returns the Unix time of a given time of type `time.Time`.
+  For example: , `UnixSecond(Now())`, `UnixSecond(node.metadata.creationTimestamp)`.
+* `Quantity()` returns a float64 value of a given Quantity value. For example: `Quantity("100m")`, `Quantity("10Mi")`.
+* `Usage()` returns the current instantaneous resource usage with the simulation data in [ResourceUsage (ClusterResourceUsage)].
+  For example: `Usage(pod, "memory")`, `Usage(node, "memory")`, `Usage(pod, "memory", container.name)` return the
+  current working set of a resource (pod, node or container) in bytes.
+* `CumulativeUsage()` returns the cumulative resource usage in seconds with the simulation data given in [ResourceUsage (ClusterResourceUsage)].
+  For example: `CumulativeUsage(pod, "cpu")`, `CumulativeUsage(node, "cpu")`, `CumulativeUsage(pod, "cpu", container.name)`
+  return a cumulative cpu time consumed by a resource (pod, node or container) in core-seconds.
+
+Additionally, `kwok` provides three special CEL variables `node`, `pod`, and `container` that could be used 
+in the expressions.
+The three variables are set to the corresponding node, pod, container resource object respectively and users can
+reference any nested fields of the resource objects simply via the CEL field selection expression (`e.f` format). 
+For example, you could use expression `node.metadata.name` to obtain the node name. 
+
+{{< hint "info" >}}
+
+The functions with at least one parameter can be called in a receiver call-style.
+That is, a function call like `f(e1, e2)` can also be called in style `e1.f(e2)`. For example, you can use `pod.Usage("memory")`
+as an alternative to `Usage(pod, "memory")`.
+
+{{< /hint >}}
+
+
+It is worth noting that the use of some extension functions is restricted to specific CRs and contexts in the sense
+that they are not generic but designed for special evaluating tasks.
+The detailed limitations are described below.
+
+## Functions Limitation
+
+Function `Usage()` and `CumulativeUsage()` can only be used in the Metric resource.
+For other functions listed above, users are also allowed to use them in ResourceUsage and ClusterResourceUsage
+to build dynamic resource usage patterns.
+
+The reason behind is that when `kwok` evaluates functions `Usage()` or `CumulativeUsage()`,
+it actually takes the simulation data given in ResourceUsage and ClusterResourceUsage to obtain metric values.
+Therefore, please ensure that the associated ResourceUsage or ClusterResourceUsage with the needed resource types
+(cpu or memory) are also provided when using function `Usage()` and `CumulativeUsage()`.
+
+## Variables Limitation
+
+When using the three special CEL variables `node`, `pod`, and `container` in Metric resource, you should follow the below rules.
+* When `dimension` is `node`: only `node` variable can be used.
+* When `dimension` is `pod`: only `node`, `pod` can be used.
+* When `dimension` is `container`: `node`, `pod`, `container` all can be used.
+
+
+[Metric]: {{< relref "/docs/generated/apis" >}}#kwok.x-k8s.io/v1alpha1.Metric
+[ResourceUsage]: {{< relref "/docs/generated/apis" >}}#kwok.x-k8s.io/v1alpha1.ResourceUsage
+[ClusterResourceUsage]: {{< relref "/docs/generated/apis" >}}#kwok.x-k8s.io/v1alpha1.ClusterResourceUsage
+[the CEL language specification]: https://github.com/google/cel-spec/blob/master/doc/langdef.md
+[CEL predefined functions]: https://github.com/google/cel-spec/blob/master/doc/langdef.md#list-of-standard-definitions

site/content/en/docs/user/go-template.md

@@ -0,0 +1,26 @@
+---
+title: "Go Template in `kwok`"
+---
+
+# Notes on Go Template in `kwok`
+
+
+The page provides a concise note on writing go templates in kwok CRs.
+
+
+Currently, only `Stage` CR has go template based fields (`Spec.Next.StatusTemplate`).
+
+
+You must follow [the go text template syntax] when writing the templates.
+For predefined functions of go text template, please refer to [go text template functions].
+Besides the built-in functions, `kwok` also supports [sprig template functions].
+
+It is worth noting that the "context" (which is denoted by the period character `.` ) to a template in `kwok` is set to the 
+referenced Kubernetes resource. 
+For example, you can use `.metadata.name` in a template to obtain the corresponding Kubernetes resource name.
+
+
+
+[the go text template syntax]: https://pkg.go.dev/text/template
+[go text template functions]:  https://pkg.go.dev/text/template#hdr-Functions
+[sprig template functions]: https://masterminds.github.io/sprig/

site/content/en/docs/user/kwok-in-cluster.md

@@ -43,6 +43,13 @@ This allows to simulate the resource usage of nodes, pods and containers.
 kubectl apply -f "https://github.com/${KWOK_REPO}/releases/download/${KWOK_LATEST_RELEASE}/metrics-usage.yaml"
 ```
 
+The above configuration sets the CPU and memory usage of all the containers managed by `kwok` to `1m` and to `1Mi` respectively.
+To override the defaults, you can add annotation `"kwok.x-k8s.io/usage-cpu"` (for cpu usage) and
+`"kwok.x-k8s.io/usage-memory"` (for memory usage) with any quantity value you want to the fake pods.
+
+The resource usage simulation used above is annotation-based and the configuration is available at [here][resource usage from annotation].
+For the explanation of how it works and more complex resource usage simulation methods, please refer to [ResourceUsage configuration].
+
 ## Old way to deploy kwok
 
 Old way to deploy kwok is [here][kwok in cluster old].
@@ -53,3 +60,5 @@ Now, you can use `kwok` to [manage nodes and pods] in the Kubernetes cluster.
 
 [manage nodes and pods]: {{< relref "/docs/user/kwok-manage-nodes-and-pods" >}}
 [kwok in cluster old]: {{< relref "/docs/user/kwok-in-cluster-old" >}}
+[resource usage from annotation]: https://github.com/kubernetes-sigs/kwok/tree/main/kustomize/metrics/usage
+[ResourceUsage configuration]: {{< relref "/docs/user/resource-usage-configuration" >}}

site/content/en/docs/user/metrics-configuration.md

@@ -63,6 +63,10 @@ With `{nodeName}`, a single `path` is able to differentiate the metric data from
 
 
 The `metrics` field are used to customize the return body of the installed metrics endpoint.
+
+The descriptions of each sub-field are available at [Metric API][Metric]. 
+For readers' convenience, we also mirror the documents here with some additional notes.
+
 `metrics` is a list of specific configuration items, with each corresponding to a Prometheus style metric:
 * `name` defines the metric name.
 * `labels` defines the metric labels, with each item corresponding to a specific metric label.
@@ -73,7 +77,7 @@ The `metrics` field are used to customize the return body of the installed metri
 * `kind` defines the type of the metric: `counter`, `guage` or `histogram`.
 * `dimension` defines where the data comes from. It could be `node`, `pod`, or `container`.
 * `value` is a CEL expression that defines the metric value if `kind` is `counter` or `guage`.
-  Please refer to [built-in CEL extension functions] for an exhausted list that you can use to simulate the metric value.
+  Please refer to [CEL expressions in `kwok`] for more detailed instructions that might be helpful to simulate the metric value.
 * `buckets` is exclusively for customizing the data of the metric of kind `histogram`.
   - `le`, which defines the histogram bucket’s upper threshold, has the same meaning as the one of Prometheus histogram bucket.
     That is, each bucket contains values less than or equal to `le`.
@@ -84,51 +88,6 @@ The `metrics` field are used to customize the return body of the installed metri
 Please refer to [Metrics for kubelet's "metrics/resource" endpoint][metrics resource endpoint] for a detailed example.
 
 
-## built-in CEL extension functions
-
-As `kwok` doesn't actually run containers, to simulate the resource usage data, `kwok` provides the following CEL
-extension functions you can use in a CEL expression to help calculate the simulation data flexibly and dynamically:
-* `Now()`: returns the current timestamp.
-* `Rand()`: returns a random `float64` value.
-* `SinceSecond()` returns the seconds elapsed since a resource (pod or node) was created.
-  For example: `pod.SinceSecond()`, `node.SinceSecond()`.
-* `UnixSecond()` returns the Unix time of a given time of type `time.Time`.
-  For example: , `UnixSecond(Now())`, `UnixSecond(node.metadata.creationTimestamp)`.
-* `Quantity()` returns a float64 value of a given Quantity value. For example: `Quantity("100m")`, `Quantity("10Mi")`.
-* `Usage()` returns the current instantaneous resource usage with the simulation data in [ResourceUsage (ClusterResourceUsage)].
-  For example: `Usage(pod, "memory")`, `Usage(node, "memory")`, `Usage(pod, "memory", container.name)` return the
-  current working set of a resource (pod, node or container) in bytes.
-* `CumulativeUsage()` returns the cumulative resource usage in seconds with the simulation data given in [ResourceUsage (ClusterResourceUsage)].
-  For example: `CumulativeUsage(pod, "cpu")`, `CumulativeUsage(node, "cpu")`, `CumulativeUsage(pod, "cpu", container.name)`
-  return a cumulative cpu time consumed by a resource (pod, node or container) in core-seconds.
-
-
-`node`, `pod`, `container` are three special parameters that can be used only when `dimension` is set to the corresponding value.
-You should follow the below rules to use the three parameters:
-* When dimension is `node`: only `node` parameter can be used.
-* When dimension is `pod`: only `node`, `pod` can be used.
-* When dimension is `container`: `node`, `pod`, `container` all can be used.
-
-
-The functions that have more one parameter can be called in a receiver call-style.
-That is, a function call like `f(e1, e2)` can also be called in style `e1.f(e2)`. For example, you can use `pod.Usage("memory")`
-as an alternative to `Usage(pod, "memory")`.
-
-
-{{< hint "warning" >}}
-
-Function `Usage()` and `CumulativeUsage()` can only be used in the Metric resource.
-For other functions listed above, users are also allowed to use them in ResourceUsage and ClusterResourceUsage
-to build dynamic resource usage patterns.
-
-The reason behind is that when `kwok` evaluates functions `Usage()` or `CumulativeUsage()`,
-it actually takes the simulation data given in ResourceUsage and ClusterResourceUsage to obtain metric values.
-Therefore, please ensure that the associated ResourceUsage or ClusterResourceUsage with the needed resource types
-(cpu or memory) are also provided when using function `Usage()` and `CumulativeUsage()`.
-
-{{< /hint >}}
-
-
 ## Out-of-box Metric Config
 
 `kwok` currently provides the [Metrics config][metrics resource endpoint] that is capable of
@@ -140,6 +99,6 @@ nodes managed by `kwok`.
 
 [configuration]: {{< relref "/docs/user/configuration" >}}
 [Metrics]: {{< relref "/docs/generated/apis" >}}#kwok.x-k8s.io/v1alpha1.Metrics
-[built-in CEL extension functions]: {{< relref "/docs/user/metrics-configuration" >}}#built-in-cel-extension-functions
+[CEL expressions in `kwok`]: {{< relref "/docs/user/cel-expression" >}}
 [metrics resource endpoint]: https://github.com/kubernetes-sigs/kwok/blob/main/kustomize/metrics/resource
 [ResourceUsage (ClusterResourceUsage)]: {{< relref "/docs/user/resource-usage-configuration" >}}

site/content/en/docs/user/resource-usage-configuration.md

@@ -35,7 +35,8 @@ spec:
         expression: <string>
 ```
 
-To associate a ResourceUsage with a certain pod to be simulated, users must ensure `metadata.name` and `metadata.namespace` are inconsistent with the name and namespace of the target pod.
+To associate a ResourceUsage with a certain pod to be simulated, users must ensure `metadata.name` and `metadata.namespace` 
+are inconsistent with the name and namespace of the target pod.
 
 The resource usages of a pod are specified via `usages` field.
 The `usages` field are organized by groups, with each corresponding to a collection of containers that shares a same resource usage simulation setting.
@@ -66,14 +67,14 @@ yields memory usage that grows linearly with time.
 ```yaml
 expression: (pod.SinceSecond() / 60.0) * Quantity("1Mi")
 ```
-Please refer to [built-in CEL extension functions] for an exhausted list that may be helpful to configure dynamic resource usage.
+Please refer to [CEL expressions in `kwok`] for an exhausted list that may be helpful to configure dynamic resource usage.
 
 
 ### ClusterResourceUsage
 
 In addition to simulating a single pod, users can also simulate the resource usage for multiple pods via [ClusterResourceUsage].
 
-A ClusterResourceUsage resource has the following fields:
+The YAML below shows all the fields of a ClusterResourceUsage resource:
 
 ``` yaml
 kind: ClusterResourceUsage
@@ -131,4 +132,4 @@ ResourceUsage or ClusterResourceUsage only takes effect when the [Metric] featur
 [Metric]: {{< relref "/docs/user/metrics-configuration" >}}
 [the default Metric resource]:  https://github.com/kubernetes-sigs/kwok/blob/main/kustomize/metrics/resource
 [pod resource usage from annotation]: https://github.com/kubernetes-sigs/kwok/blob/main/kustomize/metrics/usage/usage-from-annotation.yaml
-[built-in CEL extension functions]: {{< relref "/docs/user/metrics-configuration" >}}#built-in-cel-extension-functions
+[CEL expressions in `kwok`]: {{< relref "/docs/user/cel-expressions" >}}

site/content/en/docs/user/stages-configuration.md

@@ -65,8 +65,9 @@ users can specify the conditions that need to be met for the stage to be applied
 and the changes that will be made to the resource when the stage is applied.
 The `next` field allows users to define the new status of the resource using the `statusTemplate` field, and even `delete` the resource.
 
-`statusTemplate` and `delete` are the two fundamental fields in `next` that respectively represent the two basic phases
-of resource lifecycle simulation: status update and resource deletion. 
+`statusTemplate` and `delete` are the two fundamental fields in `next` that respectively represent the two basic phases of resource lifecycle simulation: status update and resource deletion.
+`statusTemplate` provides a way to define resource status based on go template rendering. Please see
+[go template in `kwok`] for more detailed instructions.
 `delete: true` has higher priority than a non-empty `statusTemplate`, which means `kwok` will delete the resource
 rather than update its status if both are set.
 
@@ -257,3 +258,4 @@ This example shows how to configure the simplest and fastest stages of Pod resou
 [Stage API]: {{< relref "/docs/generated/apis" >}}#kwok.x-k8s.io/v1alpha1.Stage
 [Resource Lifecycle Simulation Controller]: {{< relref "/docs/design/architecture" >}}
 [How Delay is Calculated]: {{< relref "/docs/user/stages-configuration#how-delay-is-calculated" >}}
+[go template in `kwok`]: {{< relref "/docs/user/go-template" >}}