Implement When Expressions

Adding `WhenExpressions` used to efficiently specify guarded execution
of `Tasks`, without spinning up new pods. We use `WhenExpressions` to
avoid adding an opinionated and complex expression language to the Tekton
API to ensure Tekton can be supported by as many systems as possible.
Further details about the design are in [Conditions Beta TEP](https://github.com/tektoncd/community/blob/master/teps/0007-conditions-beta.md).

The components of `WhenExpressions` are `Input`, `Operator`
and `Values`:
- `Input` is the input for the `Guard` checking which can be static
inputs or variables, such as `Parameters` or `Results`.
- `Operator` represents an `Input`'s relationship to a set of `Values`.
`Operators` we will use in `WhenExpressions` are `In` and `NotIn`.
- `Values` is an array of string values. The `Values` array must be
non-empty. It can contain static values or variables (`Parameters` or
`Results`).

The declared `WhenExpressions` are evaluated before the `Task` is run.
If all the `WhenExpressions` evaluate to `True`, the `Task` is run.
If any of the `WhenExpressions` evaluate to `False`, the `Task` is
skipped. When a `Task` is skipped, it's included in the `Skipped Tasks`
section of the `PipelineRunStatus`.

docs/conditions.md

@@ -6,6 +6,8 @@ weight: 11
 -->
 # Conditions
 
+**Note:** `Conditions` are deprecated, use [WhenExpressions](pipelines.md#guard-task-execution-using-whenexpressions) instead. 
+
 - [Overview](#overview)
 - [Configuring a `Condition`](#configuring-a-condition)
   - [Specifying the condition `check`](#specifying-the-condition-check)

docs/pipelineruns.md

@@ -457,6 +457,26 @@ False|PipelineRunTimeout|Yes|The `PipelineRun` timed out.
 
 When a `PipelineRun` changes status, [events](events.md#pipelineruns) are triggered accordingly.
 
+When a `PipelineRun` has `Tasks` with [WhenExpressions](pipelines.md#guard-task-execution-using-whenexpressions):
+- If the `WhenExpressions` evaluate to `true`, the `Task` is executed and the `TaskRun` will be listed in the `Task Runs` section of the `status` of the `PipelineRun`.
+- If the `WhenExpressions` evaluate to `false`, the `Task` is skipped and it is listed in the `Skipped Tasks` section of the `status` of the `PipelineRun`. 
+
+```yaml
+Conditions:
+  Last Transition Time:  2020-08-27T15:07:34Z
+  Message:               Tasks Completed: 1 (Failed: 0, Cancelled 0), Skipped: 1
+  Reason:                Completed
+  Status:                True
+  Type:                  Succeeded
+Skipped Tasks:
+  Name:       skip-this-task
+Task Runs:
+  pipelinerun-to-skip-task-run-this-task-r2djj:
+    Pipeline Task Name:  run-this-task
+    Status:
+      ...
+```
+
 ## Cancelling a `PipelineRun`
 
 To cancel a `PipelineRun` that's currently executing, update its definition

docs/pipelines.md

@@ -15,6 +15,7 @@ weight: 3
     - [Using the `from` parameter](#using-the-from-parameter)
     - [Using the `runAfter` parameter](#using-the-runafter-parameter)
     - [Using the `retries` parameter](#using-the-retries-parameter)
+    - [Guard `Task` execution using `When Expressions`](#guard-task-execution-using-whenexpressions)
     - [Guard `Task` execution using `Conditions`](#guard-task-execution-using-conditions)
     - [Configuring the failure timeout](#configuring-the-failure-timeout)
   - [Using `Results`](#using-results)
@@ -316,8 +317,56 @@ tasks:
       name: build-push
 ```
 
+### Guard `Task` execution using `WhenExpressions`
+
+To run a `Task` only when certain conditions are met, it is possible to _guard_ task execution using the `when` field. The `when` field allows you to list a series of references to `WhenExpressions`.
+
+The components of `WhenExpressions` are `Input`, `Operator` and `Values`:
+- `Input` is the input for the `WhenExpression` which can be static inputs or variables ([`Parameters`](#specifying-parameters) or [`Results`](#using-results)). If the `Input` is not provided, it defaults to an empty string.
+- `Operator` represents an `Input`'s relationship to a set of `Values`. A valid `Operator` must be provided, which can be either `in` or `notin`.
+- `Values` is an array of string values. The `Values` array must be provided and be non-empty. It can contain static values or variables ([`Parameters`](#specifying-parameters) or [`Results`](#using-results)).
+
+The [`Parameters`](#specifying-parameters) are read from the `Pipeline` and [`Results`](#using-results) are read directly from previous [`Tasks`](#adding-tasks-to-the-pipeline). Using [`Results`](#using-results) in a `WhenExpression` in a guarded `Task` introduces a resource dependency on the previous `Task` that produced the `Result`. 
+
+The declared `WhenExpressions` are evaluated before the `Task` is run. If all the `WhenExpressions` evaluate to `True`, the `Task` is run. If any of the `WhenExpressions` evaluate to `False`, the `Task` is not run and the `Task` is listed in the [`Skipped Tasks` section of the `PipelineRunStatus`](pipelineruns.md#monitoring-execution-status). 
+
+In these examples, `create-readme-file` task will only be executed if the `path` parameter is `README.md` and `echo-file-exists` task will only be executed if the `status` result from `check-file` task is `exists`. 
+
+```yaml
+tasks:
+  - name: first-create-file
+    when:
+      - input: "$(params.path)"
+        operator: in
+        values: ["README.md"]
+    taskRef:
+      name: create-readme-file
+---
+tasks:
+  - name: echo-file-exists
+    when:
+      - input: "$(tasks.check-file.results.status)"
+        operator: in
+        values: ["exists"]
+    taskRef:
+        name: echo-file-exists
+```
+
+For an end-to-end example, see [PipelineRun with WhenExpressions](../examples/v1beta1/pipelineruns/pipelinerun-with-when-expressions.yaml).
+
+When `WhenExpressions` are specified in a `Task`, [`Conditions`](#guard-task-execution-using-conditions) should not be specified in the same `Task`. The `Pipeline` will be rejected as invalid if both `WhenExpressions` and `Conditions` are included.
+
+There are a lot of scenarios where `WhenExpressions` can be really useful. Some of these are:
+- Checking if the name of a git branch matches
+- Checking if the `Result` of a previous `Task` is as expected
+- Checking if a git file has changed in the previous commits
+- Checking if an image exists in the registry
+- Checking if the name of a CI job matches
+
 ### Guard `Task` execution using `Conditions`
 
+**Note:** `Conditions` are deprecated, use [`WhenExpressions`](#guard-task-execution-using-whenexpressions) instead. 
+
 To run a `Task` only when certain conditions are met, it is possible to _guard_ task execution using
 the `conditions` field. The `conditions` field allows you to list a series of references to
 [`Condition`](./conditions.md) resources. The declared `Conditions` are run before the `Task` is run.

examples/v1beta1/pipelineruns/conditional-pipelinerun-with-optional-resources.yaml

@@ -1,3 +1,5 @@
+# `Conditions` are deprecated, use `WhenExpressions` instead
+# https://github.com/tektoncd/pipeline/blob/master/docs/pipelines.md#guard-task-execution-using-whenexpressions
 apiVersion: tekton.dev/v1alpha1
 kind: Condition
 metadata:

examples/v1beta1/pipelineruns/conditional-pipelinerun-with-same-condition-refer.yaml

@@ -1,3 +1,5 @@
+# `Conditions` are deprecated, use `WhenExpressions` instead
+# https://github.com/tektoncd/pipeline/blob/master/docs/pipelines.md#guard-task-execution-using-whenexpressions
 apiVersion: tekton.dev/v1alpha1
 kind: Condition
 metadata:

examples/v1beta1/pipelineruns/conditional-pipelinerun.yaml

@@ -1,3 +1,5 @@
+# `Conditions` are deprecated, use `WhenExpressions` instead
+# https://github.com/tektoncd/pipeline/blob/master/docs/pipelines.md#guard-task-execution-using-whenexpressions
 apiVersion: tekton.dev/v1alpha1
 kind: Condition
 metadata:

examples/v1beta1/pipelineruns/demo-optional-resources.yaml

@@ -1,3 +1,5 @@
+# `Conditions` are deprecated, use `WhenExpressions` instead
+# https://github.com/tektoncd/pipeline/blob/master/docs/pipelines.md#guard-task-execution-using-whenexpressions
 apiVersion: tekton.dev/v1alpha1
 kind: Condition
 metadata:
@@ -13,7 +15,8 @@ spec:
     image: alpine
     script: 'test -f $(resources.git-repo.path)/$(params.path)'
 ---
-
+# `Conditions` are deprecated, use `WhenExpressions` instead
+# https://github.com/tektoncd/pipeline/blob/master/docs/pipelines.md#guard-task-execution-using-whenexpressions
 apiVersion: tekton.dev/v1alpha1
 kind: Condition
 metadata:

examples/v1beta1/pipelineruns/pipeline-result-conditions.yaml

@@ -1,3 +1,5 @@
+# `Conditions` are deprecated, use `WhenExpressions` instead
+# https://github.com/tektoncd/pipeline/blob/master/docs/pipelines.md#guard-task-execution-using-whenexpressions
 apiVersion: tekton.dev/v1alpha1
 kind: Condition
 metadata:

examples/v1beta1/pipelineruns/pipelinerun-with-resourcespec.yaml

@@ -20,7 +20,8 @@ spec:
         - |
           ls -al $(resources.inputs.pipeline-git.path)
 ---
-
+# `Conditions` are deprecated, use `WhenExpressions` instead
+# https://github.com/tektoncd/pipeline/blob/master/docs/pipelines.md#guard-task-execution-using-whenexpressions
 apiVersion: tekton.dev/v1alpha1
 kind: Condition
 metadata:

examples/v1beta1/pipelineruns/pipelinerun-with-when-expressions.yaml

@@ -0,0 +1,99 @@
+apiVersion: tekton.dev/v1beta1
+kind: Pipeline
+metadata:
+  name: guarded-pipeline
+spec:
+  params:
+    - name: path
+      type: string
+      description: The path of the file to be created
+  workspaces:
+    - name: source
+      description: |
+        This workspace will receive the cloned git repo and be passed
+        to the next Task to create a file
+  tasks:
+    - name: create-file
+      when:
+        - input: "$(params.path)"
+          operator: in
+          values: ["README.md"]
+      workspaces:
+        - name: source
+          workspace: source
+      taskSpec:
+        workspaces:
+          - name: source
+            description: The workspace to create the readme file in
+        steps:
+          - name: write-new-stuff
+            image: ubuntu
+            script: 'touch $(workspaces.source.path)/README.md'
+    - name: check-file
+      params:
+        - name: path
+          value: "$(params.path)"
+      workspaces:
+        - name: source
+          workspace: source
+      runAfter:
+        - create-file
+      taskSpec:
+        params:
+          - name: path
+        workspaces:
+          - name: source
+            description: The workspace to check for the file
+        results:
+          - name: status
+            description: indicates whether the file exists or is missing
+        steps:
+          - name: check-file
+            image: alpine
+            script: |
+              if test -f $(workspaces.source.path)/$(params.path); then
+                printf exists | tee /tekton/results/status
+              else
+                printf missing | tee /tekton/results/status
+              fi
+    - name: echo-file-exists
+      when:
+        - input: "$(tasks.check-file.results.status)"
+          operator: in
+          values: ["exists"]
+      taskSpec:
+        steps:
+          - name: echo
+            image: ubuntu
+            script: 'echo file exists'
+    - name: task-should-be-skipped
+      when:
+        - input: "$(params.path)"
+          operator: notin
+          values: ["README.md"]
+      taskSpec:
+        steps:
+          - name: echo
+            image: ubuntu
+            script: exit 1
+---
+apiVersion: tekton.dev/v1beta1
+kind: PipelineRun
+metadata:
+  name: guarded-pr
+spec:
+  serviceAccountName: 'default'
+  pipelineRef:
+    name: guarded-pipeline
+  params:
+    - name: path
+      value: README.md
+  workspaces:
+    - name: source
+      volumeClaimTemplate:
+        spec:
+          accessModes:
+          - ReadWriteOnce
+          resources:
+            requests:
+              storage: 1Gi

internal/builder/v1beta1/pipeline.go

@@ -24,6 +24,7 @@ import (
 	resource "github.com/tektoncd/pipeline/pkg/apis/resource/v1alpha1"
 	corev1 "k8s.io/api/core/v1"
 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
+	"k8s.io/apimachinery/pkg/selection"
 	"knative.dev/pkg/apis"
 )
 
@@ -332,6 +333,18 @@ func PipelineTaskConditionResource(name, resource string, from ...string) Pipeli
 	}
 }
 
+// PipelineTaskWhenExpression adds a WhenExpression with the specified input, operator and values
+// which are used to determine whether the PipelineTask should be executed or skipped.
+func PipelineTaskWhenExpression(input string, operator selection.Operator, values []string) PipelineTaskOp {
+	return func(pt *v1beta1.PipelineTask) {
+		pt.WhenExpressions = append(pt.WhenExpressions, v1beta1.WhenExpression{
+			Input:    input,
+			Operator: operator,
+			Values:   values,
+		})
+	}
+}
+
 // PipelineTaskWorkspaceBinding adds a workspace with the specified name, workspace and subpath on a PipelineTask.
 func PipelineTaskWorkspaceBinding(name, workspace, subPath string) PipelineTaskOp {
 	return func(pt *v1beta1.PipelineTask) {

internal/builder/v1beta1/pipeline_test.go

@@ -23,6 +23,7 @@ import (
 	"github.com/google/go-cmp/cmp"
 	corev1 "k8s.io/api/core/v1"
 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
+	"k8s.io/apimachinery/pkg/selection"
 	"knative.dev/pkg/apis"
 	duckv1beta1 "knative.dev/pkg/apis/duck/v1beta1"
 
@@ -54,6 +55,7 @@ func TestPipeline(t *testing.T) {
 			tb.PipelineTaskOutputResource("some-image", "my-only-image-resource"),
 		),
 		tb.PipelineTask("never-gonna", "give-you-up",
+			tb.PipelineTaskWhenExpression("foo", selection.In, []string{"foo", "bar"}),
 			tb.RunAfter("foo"),
 			tb.PipelineTaskTimeout(5*time.Second),
 		),
@@ -133,10 +135,11 @@ func TestPipeline(t *testing.T) {
 					}},
 				},
 			}, {
-				Name:     "never-gonna",
-				TaskRef:  &v1beta1.TaskRef{Name: "give-you-up"},
-				RunAfter: []string{"foo"},
-				Timeout:  &metav1.Duration{Duration: 5 * time.Second},
+				Name:            "never-gonna",
+				TaskRef:         &v1beta1.TaskRef{Name: "give-you-up"},
+				WhenExpressions: []v1beta1.WhenExpression{{Input: "foo", Operator: selection.In, Values: []string{"foo", "bar"}}},
+				RunAfter:        []string{"foo"},
+				Timeout:         &metav1.Duration{Duration: 5 * time.Second},
 			}, {
 				Name: "foo",
 				TaskSpec: &v1beta1.EmbeddedTask{

pkg/apis/pipeline/v1beta1/pipeline_types.go

@@ -124,9 +124,14 @@ type PipelineTask struct {
 	TaskSpec *EmbeddedTask `json:"taskSpec,inline,omitempty"`
 
 	// Conditions is a list of conditions that need to be true for the task to run
+	// Conditions are deprecated, use WhenExpressions instead
 	// +optional
 	Conditions []PipelineTaskCondition `json:"conditions,omitempty"`
 
+	// WhenExpressions is a list of when expressions that need to be true for the task to run
+	// +optional
+	WhenExpressions WhenExpressions `json:"when,omitempty"`
+
 	// Retries represents how many times this task should be retried in case of task failure: ConditionSucceeded set to False
 	// +optional
 	Retries int `json:"retries,omitempty"`
@@ -197,6 +202,16 @@ func (pt PipelineTask) Deps() []string {
 			}
 		}
 	}
+	// Add any dependents from when expressions
+	for _, whenExpression := range pt.WhenExpressions {
+		expressions, ok := whenExpression.GetVarSubstitutionExpressions()
+		if ok {
+			resultRefs := NewResultRefs(expressions)
+			for _, resultRef := range resultRefs {
+				deps = append(deps, resultRef.PipelineTask)
+			}
+		}
+	}
 	return deps
 }
 

pkg/apis/pipeline/v1beta1/pipeline_validation.go

@@ -212,6 +212,10 @@ func (ps *PipelineSpec) Validate(ctx context.Context) *apis.FieldError {
 		return err
 	}
 
+	if err := validateWhenExpressions(ps.Tasks); err != nil {
+		return err
+	}
+
 	return nil
 }
 
@@ -364,6 +368,9 @@ func validatePipelineParametersVariables(tasks []PipelineTask, prefix string, pa
 		if err := validatePipelineParametersVariablesInTaskParameters(task.Params, prefix, paramNames, arrayParamNames); err != nil {
 			return err
 		}
+		if err := task.WhenExpressions.validatePipelineParametersVariables(prefix, paramNames, arrayParamNames); err != nil {
+			return err
+		}
 	}
 	return nil
 }
@@ -460,6 +467,9 @@ func validateFinalTasks(finalTasks []PipelineTask) *apis.FieldError {
 		if len(f.Conditions) != 0 {
 			return apis.ErrInvalidValue(fmt.Sprintf("no conditions allowed under spec.finally, final task %s has conditions specified", f.Name), "spec.finally")
 		}
+		if len(f.WhenExpressions) != 0 {
+			return apis.ErrInvalidValue(fmt.Sprintf("no when expressions allowed under spec.finally, final task %s has when expressions specified", f.Name), "spec.finally")
+		}
 	}
 
 	if err := validateTaskResultReferenceNotUsed(finalTasks); err != nil {
@@ -503,3 +513,23 @@ func validateTasksInputFrom(tasks []PipelineTask) *apis.FieldError {
 	}
 	return nil
 }
+
+func validateWhenExpressions(tasks []PipelineTask) *apis.FieldError {
+	for i, t := range tasks {
+		if err := validateOneOfWhenExpressionsOrConditions(i, t); err != nil {
+			return err
+		}
+		if err := t.WhenExpressions.validate(); err != nil {
+			return err
+		}
+	}
+	return nil
+}
+
+func validateOneOfWhenExpressionsOrConditions(i int, t PipelineTask) *apis.FieldError {
+	prefix := "spec.tasks"
+	if t.WhenExpressions != nil && t.Conditions != nil {
+		return apis.ErrMultipleOneOf(fmt.Sprintf(fmt.Sprintf(prefix+"[%d].when", i), fmt.Sprintf(prefix+"[%d].conditions", i)))
+	}
+	return nil
+}