[Maps] Should be able to style properties based on classifier

[thomasneirynck]

Data driven styling is currently only for ordinal values. The maps app should have a mechanism to apply styling based on some classifier.

[elasticmachine]

Pinging @elastic/kibana-gis

[thomasneirynck]

There is multiple ways we can address this functionality.

For (1) and (2), the maps app can provide a UX for people to select a field to drive a color-property. The actual field-value will be used as the styling-value. E.g. a color-hex-string for a color property, or a number for pixel-width property.

For (3), it requires the introduction of new layer-level settings, but the styling UX does not need to be altered.

1. use field formatters of field

• use color field-formatter from the index pattern. This can drive color styling properties using regex expressions.

2. use value of a field

• users can write script-fields and use painless to write a classifier

3. For smaller classifiers, the maps app can provide a layer-based filtering.

• a KQL query can then classify data.
• this requires multiple layers

(1), (2) and (3) are all limited to data backed by Elasticsearch (ES-doc sources)

cc @nreese @alexfrancoeur @aaronjcaldwell @gingerwizard

[nreese]

@thomasneirynck Just found one flaw with using index-pattern color field formatter, The color formatter is designed to format text and provides a text color and background color options. We only need a single color so I assume we would use the text color option but this might be a little confusing. Maybe we need a new color formatter for classifications?

[alexfrancoeur]

use field formatters of field

I'm a little concerned about this approach as it is space specific. Field formatters for an index pattern are pretty hard to discover and as we begin to introduce features like Feature Controls, some users may not even have access to management. I would hope that we could come up with a way to introduce these types of customizations at the maps level. If it's really a low effort to introduce this functionality to the maps app, I'd say we could move forward on it. I'm not convinced this is worth putting time into, I believe a majority of our users would expect per layer styling and are unaware of field formatters for an index pattern.

use value of a field

This is also an interesting approach and would require whomever is creating the map to have either access to ingest configurations or be forced to only use what is available within the document. For the latter, I think it's a fine approach to start but again, I don't think it would end up being widely adopted. My concern being that as an analyst, you likely will not have the ability to adjust document structure and the ingest pipeline. Taking such a global approach will severely limit the feature itself but it could be a good start. I'd vote for this over option 1.

For smaller classifiers, the maps app can provide a layer-based filtering

Honestly, I prefer this approach over 1 and 2 at the moment. We want layer based filtering anyway. While not ideal, it's super easy to configure three layers with a good, warning and severe styling/status for a given threshold. My concern here is scalability. If this is being done at the individual document level, queries could get expensive quick.

Honestly, I think it is worth discussing the amount of effort for a layer level classifier vs. implementing something like 1 & 2. I think layer filtering in option 3 is something we want regardless. If we add additional styling capabilities to layers (logarithmic gradients, shapes/icons, custom svgs, custom ranges, etc.) and introduce layer filtering, I think this will support a majority of the styling use cases for quite some time. It won't be ideal, but it'll hold our users over for a bit.

[thomasneirynck]

@alexfrancoeur

I think we're already hitting some roadblocks with (1), so it's unlikely we can consider this low-hanging fruit (see #6480 (comment))

For (2), while I think this is low-hanging fruit (and I can be wrong), and I also think this would be worthwhile functionality.

as an analyst, you likely will not have the ability to adjust document structure and the ingest pipeline

This is only part of the story. Agreed that ownership of the ingest-pipeline is not guaranteed. But users/analysts could also write scripted-fields to generate a styling value, and that's where I expect most of the ad-hoc customization could go on. Unlike field-formatters, script-fields are core to Elasticsearch as well, so we can expect long-term value out of it (even if the UX where people can write those classifiers is not necessarily part of the <aps-app (at least, not as a first iteration, where that UX will continue to live on the index-pattern management page).

.

[alexfrancoeur]

++ well said on #2 Thomas. If we go down this route, I think we'll need to consider what the experience might be like for those not using KQL. In 7.x, KQL will be the default search experience, but it can be easily switched back to lucene. If the styling for a layer was based on a scripted field and a customer is using lucene query syntax instead of KQL, what's the experience for a a map in a dashboard? Similar to visualizations, I imagine we can code around it, but wanted to point this out earlier rather than later. It might be helpful to discuss some of this live.

[alexfrancoeur]

Summarizing a discussion from last week (@thomasneirynck correct me if I'm wrong), we'll be moving forward with option 3 in the short term and - multiple layers and KQL filters, and revisiting option 2 - styling by a field value in the future.

[alexfrancoeur]

Are we closing this out with layer based filtering? Or should we keep open for a more curated, single layer, approach?

[thomasneirynck]

We're going to do a first past at this, which will just be a Maps-only approach. In this first pass, we will not introduce the global concept of a classifier that cuts across all Kibana apps (e.g. by making this part of the index-pattern).

We are looking to introduce class-based styling as a simple classifier for string-fields. The colors can either be manually defined, or Maps-app will do a best-effort, similar to how color-ramps for ordinal-fields can be defined manually or statically.

For more advanced classifiers, users will still be able to write a scripted-field in painless that maps a document to a class-value. That class-value can then be used to drive styling.

[thomasneirynck]

Related #43697. Class based styling could also benefit from reusable color palettes.

This is a popular website for color selection as well. http://colorbrewer2.org/#type=sequential&scheme=BuGn&n=3

[Plachey]

Hello guys, who can help me with index color format request
I found this api /api/saved_objects/index-pattern/xxxxx/fields
but i got problem with data for it request
my data:
data_format = json.dumps({
"fields": {
"log": {
"format": "color",
'pattern'
}
}
})

Need like on screenshot

[nreese]

Hello guys, who can help me with index color format request

The best place to ask questions in the kibana forum. I would recommend posting your question there.