diff --git a/README.md b/README.md
index 4aae6eb4..6fb5377a 100644
--- a/README.md
+++ b/README.md
@@ -201,6 +201,7 @@ The following templating functions are available:
- `blocked`: boolean (default: false): no tasks can be created from this template
- `hidden`: boolean (default: false): the template is not listed on the API, it is concealed to regular users
- `retry_max`: int (default: 100): maximum amount of consecutive executions of a task based on this template, before being blocked for manual review
+- `tags`: templateable map, used to filter tasks (see [tags](#tags))
### Inputs
@@ -229,6 +230,25 @@ See the example template above to see variables in action. The expression in a v
The JavaScript evaluation is done using [otto](https://github.com/robertkrimen/otto).
+### Tags
+
+Tags are a map of strings property of a task. They will be used in the task listing to search for some tasks using filters. With tags, uTask can be used as a task backend by others APIs.
+
+Tags values are expected to be a `string`: it support all uTask templating on values. To remove a tag from a task, use the empty value `""`.
+
+```yaml
+ tags:
+ customer: "{{.input.customer_id}}"
+ type: "billing"
+```
+
+In this example, tag `customer` will be templated from the task inputs, and allow others APIs to search all the tasks for a given customer.
+
+Tags can be added to a task:
+- from the template definition of the task
+- while creating a task, requester can input custom tags
+- during the execution, using the [`tag` builtin plugin](./pkg/plugins/builtin/tag/README.md)
+
### Steps
A step is the smallest unit of work that can be performed within a task. At is's heart, a step defines an **action**: several types of actions are available, and each type requires a different configuration, provided as part of the step definition. The state of a step will change during a task's resolution process, and determine which steps become elligible for execution. Custom states can be defined for a step, to fine-tune execution flow (see below).
@@ -297,14 +317,20 @@ steps:
- `name`: a unique identifier
- `description`: a human readable sentence to convey the step's intent
+- `action`: see [Action](#step-action)
+- `foreach`: see [Loops](#step-foreach)
- `dependencies`: a list of step names on which this step waits before running
+- `idempotent`: a boolean indicating if this step is safe to be replayed in case of uTask instance crash
+- `json_schema`: a JSON-Schema object to validate the step output
+- `resources`: a list of resources that will be used by this step to apply some rate-limiting (see [resources](#resources))
+- `custom_states`: a list of user-defined states for this step, to be used inside conditions
- `retry_pattern`: (`seconds`, `minutes`, `hours`) define on what temporal order of magnitude the re-runs of this step should be spread (default = `seconds`)
-#### Action
+#### Action
The `action` field of a step defines the actual workload to be performed. It consists of at least a `type` chosen among the registered action plugins, and a `configuration` fitting that plugin. See below for a detailed description of builtin plugins. For information on how to develop your own action plugins, refer to [this section](#plugins).
@@ -408,8 +434,9 @@ Browse [builtin actions](./pkg/plugins/builtin)
|**`email`** | Send an email | [Access plugin doc](./pkg/plugins/builtin/email/README.md)
|**`ping`** | Send a ping to an hostname *Warn: This plugin will keep running until the count is done* | [Access plugin doc](./pkg/plugins/builtin/ping/README.md)
|**`script`** | Execute a script under `scripts` folder | [Access plugin doc](./pkg/plugins/builtin/script/README.md)
+|**`tag`** | Add tags to the current running task | [Access plugin doc](./pkg/plugins/builtin/tag/README.md)
-#### Loops
+#### Loops
A step can be configured to take a json-formatted collection as input, in its `foreach` property. It will be executed once for each element in the collection, and its result will be a collection of each iteration. This scheme makes it possible to chain several steps with the `foreach` property.
@@ -442,6 +469,43 @@ This output can be then passed to another step in json format:
foreach: '{{.step.prefixStrings.children | toJson}}'
```
+### Resources
+
+Resources can be declared to throttle the number of parallel access to a 'resource'. Each resources are labels that can correspond to a physical or logical device/object that will be used inside an action, on which you want to limit the number of parallel accesses. Those labels can be very specific (example: an IP/port combination `172.17.0.1:5432`) or not (example: `all-databases`).
+
+Resources are configured in the `utask-cfg` configuration, indicating a name and a number of maximum parallel accesses.
+
+```json
+{
+ "resource_limits": {
+ "redis-foobar": 2,
+ "internet-gateway": 1000,
+ "database": 14
+ }
+}
+```
+
+Note: resources maximum parallel accesses are defined **per instances**. In this example, if you have 3 instances, then, `redis-foobar` can have up to 6 parallel accesses.
+
+Resources are available to be used inside steps, ensuring that the declared resources won't be accessed more than expected.
+
+```yaml
+steps:
+ getUser:
+ description: Get user
+ resources: ["redis-foobar", "internal-gateway"]
+ action:
+ type: http
+ configuration:
+ url: http://example.org/addToCache
+ method: POST
+ body: '{"cache_method":"redis", "data":"hello"}'
+```
+
+If a resource is already accessed at maximum capacity in others tasks/steps, the step execution will wait until a slot is available.
+
+If a resource declared in a step doesn't exist in the configuration of current uTask instance, then no restriction is applied and the resource can be accessed freely, without limitation.
+
### Task templates validation
A JSON-schema file is available to validate the syntax of task templates, it's available in `hack/template-schema.json`.
diff --git a/examples/templates/hello-world-now.yaml b/examples/templates/hello-world-now.yaml
index 7e9eae5f..251012e9 100644
--- a/examples/templates/hello-world-now.yaml
+++ b/examples/templates/hello-world-now.yaml
@@ -1,44 +1,59 @@
-name: hello-world-now
-description: Say hello to the world, now!
+name: hello-world-now
+description: Say hello to the world, now!
long_description: This task prints out a greeting to the entire world, after retrieving the current UTC time from an external API
-doc_link: https://en.wikipedia.org/wiki/%22Hello,_World!%22_program
+doc_link: https://en.wikipedia.org/wiki/%22Hello,_World!%22_program
-title_format: Say hello in {{.input.language}}
+title_format: Say hello in {{.input.language}}
result_format:
- echo_message: '{{.step.sayHello.output.message}}'
- echo_when: '{{.step.sayHello.output.when}}'
+ echo_message: "{{.step.sayHello.output.message}}"
+ echo_when: "{{.step.sayHello.output.when}}"
-allowed_resolver_usernames: []
+allowed_resolver_usernames: []
allow_all_resolver_usernames: true
auto_runnable: true
-blocked: false
-hidden: false
+blocked: false
+hidden: false
variables:
-- name: english-message
- value: Hello World!
-- name: spanish-message
- expression: |-
- // a short javascript snippet
- var h = 'Hola';
- var m = 'mundo';
- h + ' ' + m + '!';
+ - name: english-message
+ value: Hello World!
+ - name: spanish-message
+ expression: |-
+ // a short javascript snippet
+ var h = 'Hola';
+ var m = 'mundo';
+ h + ' ' + m + '!';
inputs:
-- name: language
- description: The language in which you wish to greet the world
- legal_values: [english, spanish]
- optional: true
- default: english
+ - name: language
+ description: The language in which you wish to greet the world
+ legal_values: [english, spanish]
+ optional: true
+ default: english
steps:
getTime:
description: Get UTC time
+ idempotent: true
+ resources: ["worldclockapi"]
+ retry_pattern: minutes
action:
type: http
configuration:
url: http://worldclockapi.com/api/json/utc/now
method: GET
+ json_schema:
+ "$schema": "http://json-schema.org/draft-07/schema#"
+ type: object
+ required: [currentDateTime, isDayLightSavingsTime, dayOfTheWeek]
+ properties:
+ currentDateTime:
+ type: string
+ pattern: '^\d+-\d+-\d+T\d+:\d+Z$'
+ isDayLightSavingsTime:
+ type: boolean
+ dayOfTheWeek:
+ type: string
sayHello:
description: Echo a greeting in your language of choice
dependencies: [getTime]
@@ -49,5 +64,4 @@ steps:
message: >-
{{if (eq .input.language "english")}}{{eval "english-message"}}
{{else if (eq .input.language "spanish")}}{{eval "spanish-message"}}{{end}}
- when: '{{.step.getTime.output.currentDateTime}}'
-
+ when: "{{.step.getTime.output.currentDateTime}}"
diff --git a/hack/template-schema.json b/hack/template-schema.json
index 81eb17c7..23bdecd3 100644
--- a/hack/template-schema.json
+++ b/hack/template-schema.json
@@ -27,6 +27,15 @@
"action"
],
"properties": {
+ "name": {
+ "type": "string",
+ "title": "Step unique name",
+ "description": "Will identify the step uniquely",
+ "default": "",
+ "examples": [
+ "get_UTC_time"
+ ]
+ },
"description": {
"type": "string",
"title": "Step description",
@@ -70,6 +79,13 @@
"type": "string"
}
},
+ "resources": {
+ "type": "array",
+ "description": "Declares resources that will be used during this step",
+ "items": {
+ "type": "string"
+ }
+ },
"foreach": {
"type": "string",
"description": "Elements on which the step will loop"
@@ -77,6 +93,10 @@
"json_schema": {
"type": "object",
"description": "Elements on which the step will loop"
+ },
+ "idempotent": {
+ "type": "boolean",
+ "description": "Indicates if this step is safe to be retried when uTask crashes"
}
}
},
@@ -109,6 +129,9 @@
{
"$ref": "#/definitions/ActionAPIOVH"
},
+ {
+ "$ref": "#/definitions/ActionTag"
+ },
{
"type": "object",
"title": "Generic action",
@@ -433,6 +456,20 @@
},
"resolver_usernames": {
"type": "string"
+ },
+ "watcher_usernames": {
+ "type": "string"
+ },
+ "delay": {
+ "type": "string"
+ },
+ "tags": {
+ "type": "object",
+ "patternProperties": {
+ ".*": {
+ "type": "string"
+ }
+ }
}
}
},
@@ -447,6 +484,40 @@
"additionalProperties": false,
"description": "Subtask action will spawn a subtask of the specified template, and wait for resolution"
},
+ "ActionTag": {
+ "type": "object",
+ "properties": {
+ "type": {
+ "const": "tag"
+ },
+ "configuration": {
+ "type": "object",
+ "additionalProperties": false,
+ "required": [
+ "tags"
+ ],
+ "properties": {
+ "tags": {
+ "type": "object",
+ "patternProperties": {
+ ".*": {
+ "type": "string"
+ }
+ }
+ }
+ }
+ },
+ "base_configuration": {
+ "type": "string"
+ },
+ "base_output": {
+ "type": "object"
+ }
+ },
+ "title": "Tag Action",
+ "additionalProperties": false,
+ "description": "Tag action will add some tags to the task"
+ },
"ActionPing": {
"type": "object",
"properties": {
@@ -947,6 +1018,15 @@
}
}
},
+ "tags": {
+ "type": "object",
+ "description": "Defines this template tags",
+ "patternProperties": {
+ ".*": {
+ "type": "string"
+ }
+ }
+ },
"base_configurations": {
"type": "object",
"description": "Global configurations that will be used as base for steps inside this template"
diff --git a/pkg/plugins/builtin/tag/README.md b/pkg/plugins/builtin/tag/README.md
index 7c3194ab..517c653d 100644
--- a/pkg/plugins/builtin/tag/README.md
+++ b/pkg/plugins/builtin/tag/README.md
@@ -5,7 +5,6 @@ This plugin updates the tags of the current task. Existing tags are overwritten
## Configuration
|Fields|Description
-| --- | --- |
| ------ | --------------- |
| `tags` | key/values tags |
@@ -20,5 +19,4 @@ action:
tags:
foo: bar
bar: # deleted
-
```