Skip to content

Commit

Permalink
INGEST: Document Pipeline Processor (#33418)
Browse files Browse the repository at this point in the history
* Added documentation for Pipeline Processor
* Relates #33188
  • Loading branch information
original-brownbear authored and jakelandis committed Oct 24, 2018
1 parent aed947f commit 27ba45e
Showing 1 changed file with 114 additions and 0 deletions.
114 changes: 114 additions & 0 deletions docs/reference/ingest/ingest-node.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -2091,6 +2091,120 @@ Converts a string to its lowercase equivalent.
--------------------------------------------------
// NOTCONSOLE

[[pipeline-processor]]
=== Pipeline Processor
Executes another pipeline.

[[pipeline-options]]
.Pipeline Options
[options="header"]
|======
| Name | Required | Default | Description
| `name` | yes | - | The name of the pipeline to execute
|======

[source,js]
--------------------------------------------------
{
"pipeline": {
"name": "inner-pipeline"
}
}
--------------------------------------------------
// NOTCONSOLE

An example of using this processor for nesting pipelines would be:

Define an inner pipeline:

[source,js]
--------------------------------------------------
PUT _ingest/pipeline/pipelineA
{
"description" : "inner pipeline",
"processors" : [
{
"set" : {
"field": "inner_pipeline_set",
"value": "inner"
}
}
]
}
--------------------------------------------------
// CONSOLE

Define another pipeline that uses the previously defined inner pipeline:

[source,js]
--------------------------------------------------
PUT _ingest/pipeline/pipelineB
{
"description" : "outer pipeline",
"processors" : [
{
"pipeline" : {
"name": "pipelineA"
}
},
{
"set" : {
"field": "outer_pipeline_set",
"value": "outer"
}
}
]
}
--------------------------------------------------
// CONSOLE
// TEST[continued]

Now indexing a document while applying the outer pipeline will see the inner pipeline executed
from the outer pipeline:

[source,js]
--------------------------------------------------
PUT /myindex/_doc/1?pipeline=pipelineB
{
"field": "value"
}
--------------------------------------------------
// CONSOLE
// TEST[continued]

Response from the index request:

[source,js]
--------------------------------------------------
{
"_index": "myindex",
"_type": "_doc",
"_id": "1",
"_version": 1,
"result": "created",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"_seq_no": 0,
"_primary_term": 1,
}
--------------------------------------------------
// TESTRESPONSE

Indexed document:

[source,js]
--------------------------------------------------
{
"field": "value",
"inner_pipeline_set": "inner",
"outer_pipeline_set": "outer"
}
--------------------------------------------------
// NOTCONSOLE

[[remove-processor]]
=== Remove Processor
Removes existing fields. If one field doesn't exist, an exception will be thrown.
Expand Down

0 comments on commit 27ba45e

Please sign in to comment.