Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: add info to implementing/tasks about sequential execution #1950

Merged
merged 8 commits into from
Sep 4, 2023
128 changes: 120 additions & 8 deletions docs/content/en/docs/implementing/tasks/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,35 +7,45 @@ hidechildren: false # this flag hides all sub-pages in the sidebar-multicard.htm

A
[KeptnTaskDefinition](../../yaml-crd-ref/taskdefinition.md/)
resource defines tasks that the Keptn Lifecycle Toolkit runs
resource defines one or more "executables"
(functions, programs, scripts, etc)
that the Keptn Lifecycle Toolkit runs
as part of the pre- and post-deployment phases of a
[KeptnApp](../../yaml-crd-ref/app.md) or
[KeptnWorkload](../../crd-ref/lifecycle/v1alpha3/#keptnworkload).

A Keptn task executes as a runner in an application
A
[KeptnTask](../../crd-ref/lifecycle/v1alpha3/#keptntask)
executes as a runner in an application
[container](https://kubernetes.io/docs/concepts/containers/),
which runs as part of a Kubernetes
[job](https://kubernetes.io/docs/concepts/workloads/controllers/job/).
A `KeptnTaskDefinition` includes a function
that defines the action taken by that task.
A `KeptnTaskDefinition` includes calls to executables to be run.

To implement a Keptn task:
To implement a `KeptnTask`:

- Define a
[KeptnTaskDefinition](../../yaml-crd-ref/taskdefinition.md)
resource that defines the runner to use for the container
and the executables to be run
pre- and post-deployment
- Apply [basic-annotations](../integrate/#basic-annotations)
to your workloads to integrate your task with Kubernetes
- Add your task to the [KeptnApp](../../yaml-crd-ref/app.md)
resource that associates your `KeptnTaskDefinition`
with the pre- and post-deployment tasks that should run in it;
- Annotate the appropriate
[KeptnApp](../../yaml-crd-ref/app.md)
resource to associate your `KeptnTaskDefinition`
with the pre- and post-deployment tasks that should run it;
see
[Pre- and post-deployment tasks and checks](../integrate/#pre--and-post-deployment-checks)
for more information.

This page provides information to help you create your tasks:

- Code your task in an appropriate [runner](#runners-and-containers)
- How to control the
[execution order](#executing-sequential-tasks)
of functions, programs, and scripts
since all `KeptnTask` resources at the same level run in parallel
- Understand how to use [Context](#context)
that contains a Kubernetes cluster, a user, a namespace,
the application name, workload name, and version.
Expand All @@ -53,6 +63,9 @@ to define the task.
The `spec` section of the `KeptnTaskDefinition`
defines the runner to use for the container:

KLT provides a general Kubernetes that you can configure
to do almost anything you want:

- The `container-runtime` runner provides
a pure custom Kubernetes application container
that you define to includes a runtime, an application
Expand Down Expand Up @@ -84,6 +97,105 @@ See the
[KeptnTaskDefinition](../../yaml-crd-ref/taskdefinition.md)
reference page for the synopsis and examples for each runner.

## Executing sequential tasks

All `KeptnTask` resources that are defined by
`KeptnTaskDefinition` resources at the same level
(either pre-deployment or post-deployment) execute in parallel.
This is by design, because Keptn is not a pipeline engine.
**Task sequences that are not part of the lifecycle workflow
should not be handled by KLT**
but should instead be handled by the pipeline engine tools being used
such as Jenkins, Argo Workflows, Flux, and Tekton.

If your lifecycle workflow includes
a sequence of executables that need to be run in order,
you can put them all in one `KeptnTaskDefinition` resource,
which can execute a virtually unlimited number
of programs, scripts, and functions,
as long as they all need the same runner, such as Python.
StackScribe marked this conversation as resolved.
Show resolved Hide resolved

Another option is to encode all your steps in the language of your choice
and build a container that Keptn executes.
This is often the best solution if you need to execute complex sequences.

If you use either the `deno-runtime` or `python-runtime` runner,
you can specify the actions to take by coding the actual calls
inline in the manifest,
by calling scripts from a remote webserver,
or by calling other `KeptnTaskDefinition` resources you have defined.
This provides great flexibility in
how you define your `KeptnTaskDefinition` resources,
allowing you to define the ideal mix of executables that run sequentially
and executables (or sets of executables) that run in parallel.

As an example, let's say you need to run a set of integration tests,
a set of performance tests, and a set of regression tests.

- You can create one `KeptnTaskDefinition` that calls all the tasks,
in order, either by putting the actual calls
in the `KeptnTaskDefinition` resource (`inline` syntax)
or by calling scripts from a remote webserver (`httpRef` syntax), or by providing a container to run.

- You can create separate `KeptnTaskDefinition` resources
for integration tests, performance tests, and regression tests.

- If you annotate the `KeptnApp` resource to call
each of these `KeptnTask` resources,
the three sets of tests run in parallel.

- You can create a "parent" `KeptnTaskDefinition` resource
that uses the `functionref` syntax
to call the `KeptnTaskDefinition` resources
for integration tests, performance tests, and regression tests.
If you annotate the `KeptnApp` resource
to run this parent `KeptnTask` resource,
all tests run sequentially.

This approach also allows you to run the test sequence if, for example,
the integration tests require the `deno-runtime` runner
but the performance and regression tests
require the `python-runtime` runner.
The parent `KeptnTaskDefinition` can run `KeptnTask` resources
that use different runners,
although the "parent" definition runtime is used
for the container that runs all the tests.
In other words, the parent `KeptnTaskDefinition` resources
is not a merge of other `KeptnTaskDefinition` resources
but rather the code/container of the parent runner.

- If you need to test your deployment for different platforms
(such as Linux, MacOS, and Windows)
or for different software versions
(such as Java 11, Java 17, and Java 21),
parallel testing can improve performance.
In this case, you can construct different `KeptnTaskDefinition` resources
for each platform,
perhaps defining different input parameters
(such as different secrets or environment variables).

- Define one `KeptnTaskDefinition` resource that runs
integration tests, then regression test, then performance tests
in order.

- You could use the `functionRef` syntax
and code the calling sequences for all tests
inside your `KeptnTaskDefinition` resource
or you could use the `httpRef` syntax
to call scripts from an external webserver.
- You could code separate `KeptnTaskDefinition` resources
for integration tests, regression tests, and performance tests.
These three test sets could then run in parallel.
- You could create a `KeptnTaskDefinition` resource
that uses the `functionRef` syntax to call the
`KeptnTaskDefinition` resources for each type of testing.
Executing that resource would execute the three types of tests
in sequential order.

If you define a `container-runtime` runner container
for your `KeptnTaskDefinition`,
you can do anything allowed by the configuration you define for that container.

## Context

A Kubernetes context is a set of access parameters
Expand Down