[Ingest Processors] [JSON] Add max_depth option to JSON processor

[a03nikki]

For a case where there are multiple applications going through an ingest pipeline with many unique nested fields multiple levels deep, it would be helpful if there was an option to limit how deep the JSON parsing would go. This would be similar to the Filebeat decode_json_fields processor max_depth option.

For example,

2021-02-17T19:00:01.255Z INFO com.example.package.ClassName(ClassName.java:52) { "order": { "id": "1", "to": { "name": "Joe Smith", ...}, "from": {...}}, "message": "Problem X, please fix order."...}

After going through a dissect processor, could be sent through a JSON processor defined as

{
  "json" : {
    "field" : "message",
    "target_field" : "json_fields",
    "max_depth": 1
  }
}

would produce a JSON document to be indexed with the "order" details not parsed into fields.

There is the option to limit the mapping depth via index.mapping.depth.limit , but it will reject the document from being indexed.

This is similar to the logstash-plugins/logstash-filter-json#43 request for Logstash.

[elasticmachine]

Pinging @elastic/es-core-features (Team:Core/Features)

[venom02]

any update on this?