script attribute for terms panel

[avleen]

The terms facet in Elasticsearch can take a script attribute, to perform an action on a field before creating the results. This is handy where you want to extract a string from a field with a regex and then generate the top terms, eg:

{
  "query": {
    "filtered": {
      "query": {
        "query_string": {
          "default_field": "message",
          "query": "type:some_log"
        }
      },
      "filter": {
        "limit" : {"value" : 10000}
      }
    }
  },
  "facets": {
    "tag": {
      "terms": {
        "field": "message.raw",
        "script": "term.replaceAll(\".*Starting job ([a-zA-Z0-9]*) .*\", \"$1\")"
      }
    }
  },
  "size": 0
}

This takes everything in the matching group, and uses that to generate the top terms.
Perhaps another field in the panel editor, where someone can enter:
Starting job ([a-zA-Z0-9]*)
would be sufficient? I can hack this into src/app/panels/terms/module.js but not in a clean way yet.

Additionally, in the above example, is the limit filter. On huge data sets, this can be really helpful when you just want to get a rough approximation of the top terms, rather than an exact count which could take a long time to gather.

[rashidkpc]

Because kibana talks directly to elasticsearch we advise users to disable dynamic scripting for security reasons. Once elasticsearch has a safer scripting model we can revisit script support in kibana. I'll leave this open as a feature request, but using MVEL from the browser is not feasible right now.

[awheeler]

This can be done safely by creating static scripts which are placed on all of the elasticsearch nodes, and then called in a query.
I'll be creating a pull request to implement the multiple fields request of #826.
I plan to implement this via adding a new terms type (in addition to terms, and terms_stats) called terms_multifield, which will accept a comma-delimited list of fields.

[awheeler]

I have a working implementation of #826, with script.disable_dynamic: true enabled on all nodes.
The only question I have is, where in the kibana tree should I place the .mvel scripts?
I'm thinking the scripts belong on the ES nodes in config/scripts/kibana/.

[jayswan]

I think this merge means that Elasticsearch now has sandboxed scripting (or will soon):

elastic/elasticsearch#6571

Does this mean Kibana might be able to get this enhancement? I wish for it every day.

[grvm20]

Is this merge already done, does elastic search have this feature???
If not by when can we get this feature?

[w33ble]

We've added script fields to the roadmap so I'm closing this issue. See #1537 to keep tabs on it.