ingest: Documentation improvements for ingest node

[jakelandis]

• document the use of tag [1] (@jakelandis) ingest: add common options to each processor's documentation #35091
• call out which field/processors support templating [2] ingest: document fields that support templating #34536 (@jakelandis)
• raise visibility of the plugin processors [3] (@jakelandis) ingest: raise visibility of ingest plugin documentation #35048
• default pipeline and index settings [4] ingest: document index.default_pipeline #34500 (@jakelandis)
• Pipeline processor : INGEST: Document Pipeline Processor #33418 (@original-brownbear)
• Processor conditional: INGEST: Document Processor Conditional #33388 (@original-brownbear)
• Drop processor ingest: documentation for the drop processor #34570 (@jakelandis )
• Additional examples for conditionals. Regex, Drop, Pipelines (@jakelandis) ingest: extended if documentation #35044
• Calling processors from the script processor. (@jakelandis)
• Kibana console updates console: ingest node updates for auto-complete kibana#24100 and console: ingest node - add tag and update pipeline processor parameter kibana#24400 (@jakelandis)
• include some mention of if and on_failure (maybe tag too?) on every processor's doc. ingest: add common options to each processor's documentation #35091

[1]

POST _ingest/pipeline/_simulate?verbose
{
  "pipeline" :
  {
    "description": "_description",
    "processors": [
      {
        "set" : {
          "tag": "my_set1",
          "field" : "field1",
          "value" : "value1"
        }
      },
      {
        "set" : {
          "tag": "my_set2",
          "field" : "field2",
          "value" : "value1"
        }
      }
    ]
  },
  "docs": [
    {
      "_index": "index",
      "_type": "_doc",
      "_id": "id",
      "_source": {
        "a": "foo-bar",
        "b": "bar"
      }
    }
  ]
}

[2]

POST _ingest/pipeline/_simulate?verbose
{
  "pipeline": {
    "processors": [
      {
        "set": {
          "field": "field1",
          "value": "{{a}}"
        }
      }
    ]
  },
  "docs": [
    {
      "_index": "index",
      "_type": "_doc",
      "_id": "id",
      "_source": {
        "a": "foo-bar"
      }
    }
  ]
}

[3]
https://www.elastic.co/guide/en/elasticsearch/plugins/current/ingest-geoip.html
https://www.elastic.co/guide/en/elasticsearch/plugins/current/ingest-user-agent.html
https://www.elastic.co/guide/en/elasticsearch/plugins/current/ingest-attachment.html
https://www.elastic.co/guide/en/elastic-stack-overview/6.4/field-and-document-access-control.html#set-security-user-processor

[4]
#32286
#32591
https://www.elastic.co/guide/en/elasticsearch/reference/current/index-modules.html#index-modules-settings

[elasticmachine]

Pinging @elastic/es-core-infra

[tomcallahan]

This is a great list @jakelandis . Is this issue a good place for @eskibars to discuss some documentation items as well or do you want to keep this one tightly scoped?

[jakelandis]

@tomcallahan @eskibars - please add/discuss at will. This is meant to be more of a meta issue and changes will come in multiple pr's.

[eskibars]

I think the documentation for the set_security_user processor may have been lost in the recent x-pack documentation refactoring/shuffle. We should add that in.

[eskibars]

Disregard -- I see you have that in [3]

[eskibars]

Also, cc @zuketo

[jakelandis]

added a few items above. @original-brownbear if you pick up any of these please put your name next to it so we don't collide.

[original-brownbear]

Added item for drop processor

[martijnvg]

@jakelandis What is the status of this meta issue?

[jakelandis]

@martijnvg - thanks for the ping. This can be considered done. The outstanding task was documenting the methods via Painless which is done here: https://www.elastic.co/guide/en/elasticsearch/painless/7.2/painless-api-reference-ingest-org-elasticsearch-ingest-common.html#painless-api-reference-ingest-Processors ...That doc could probably be beefed up some, but for now it is good enough.