[Maps] Add 3rd party vector tile support #62084

thomasneirynck · 2020-03-31T22:30:17Z

This adds the core boilerplate to Maps to display vector-tiles as first-class layer.

Introduces:

a new layer-type (TiledVectorLayer): the layer that is added to the layer-list
a new source-type (ITiledSingleLayerVectorSource): a source which models a url-template and layer-name. It is restricted to a single source-layer.
an implementation and wizard for 3rd party services.

The scope is limited to keep this PR small. Below features should be added in other PRs.

add dynamic styling
tooltip support (this requires some additional refactors in the existing code (see [Maps] [Meta] Add .mvt vector tile support #58519)
limited to a single source-layer per source (fwiw I think that's an ok limitation, especially if we fix the below)
identical tile-sources are not shared across mb-layers. This is performance micro-optimization that would significantly impact the existing mb layer-management code.
require hardcoding of the url-template and layer-name. TileJson support is a larger (separate) effort. (ie. and some 3rd party services do not implement TileJson anyway).
- TileJson support will also give us the field metadata that can be used for dynamic styling
add attribution: this is blocked by [Maps] Cannot put in attribution for all source types #32821 (comment)

Note that the endgoal is primarly to support tiling from Elasticsearch data, which also informs some of the limitations discussed above (ie. single source-layer, no TileJson support)

Reviewer notes:

Note: ~~this PR is stuck in the TS-swamp for a moment, please disregard compilation errors. Will aim to address this some of the larger TS-issues in other PRs. (cf. [Maps] Cleanup sources #63175)~~
vector tile services have native zoom levels. Users will need to configure this to avoid 404s when connecting to the service. It also helps with providing feedback about visibility in the TOC

To test:

enable the feature flag in kibana[.dev].yml. xpack.maps.enableVectorTiles: true
Example tile services:
- EMS (unlicensed):
  - https://tiles.maps.elastic.co/data/v3/{z}/{x}/{y}.pbf
  - zoom levels: 0-10
  - example layers: transportation, landuse, water, waterway
- EMS (licensed): plugin the license key!
  - https://tiles.maps.elastic.co/data/v3/{z}/{x}/{y}.pbf?elastic_tile_service_tos=agree&my_app_name=kibana&my_app_version=8.0.0&license={elastic_license_key}
  - zoom levels: 0-14
  - example layers: building, poi, transportation

elasticmachine · 2020-04-06T22:53:05Z

Pinging @elastic/kibana-gis (Team:Geo)

jsanz · 2020-04-07T10:56:50Z

This is a lot of fun!! Tested locally, I couldn't use EMS tiles on my set up, but worked smoothly with another tile provider. You have already captured all small UI things I also noticed (reusing the source, tooltips, style-driven rendering).

For what is worth, I used this snippet to get the names of the layers using vt2geojson:

$ vt2geojson https://url.tile.provider/0/0/0.mvt | jq -c ".features[].properties.vt_layer" | uniq  
"water"
"landcover"
"boundary"
"water_name"
"place"

Note that layers change depending on the zoom level.

thomasneirynck · 2020-04-07T13:54:23Z

thanks @jsanz for checking out! This PR is a little too rough around the edges for detailed reviewed, but we should certainly discuss scope on this. I'm looking to dial in additional functionality in small steps, but we probably want to avoid getting in a situation where we merged something in master/7.x that isn't feature complete enough to be useful.

An interim option, in absence of clearly defined boundaries, might to hide the vector-tile layer-type behind a feature-flag in the 7.x branch (could keep this enabled by default in master) ?

jsanz · 2020-04-07T14:44:28Z

+1 on keeping this behind a feature flag, maybe some folks will benefit from it if they really know their trade, but for most of our users, this would be confusing at this moment.

x-pack/legacy/plugins/maps/public/connected_components/layer_panel/layer_settings/index.js

.../maps/public/layers/sources/mvt_single_layer_vector_source/mvt_single_layer_vector_source.ts

...blic/layers/sources/mvt_single_layer_vector_source/mvt_single_layer_vector_source_editor.tsx

x-pack/plugins/maps/public/layers/tiled_vector_layer.tsx

x-pack/plugins/maps/public/layers/vector_layer.d.ts

x-pack/plugins/maps/public/layers/vector_layer.js

…iles_boiler

lukeelmers

App Arch code changes LGTM

nreese · 2020-04-16T13:28:55Z

.../maps/public/layers/sources/mvt_single_layer_vector_source/mvt_single_layer_vector_source.ts

+    return new TiledVectorLayer(vectorLayerArguments);
+  }
+
+  getGeoJsonWithMeta(


Can this be deleted? This no longer implements VectorSource so no need for the method

...blic/layers/sources/mvt_single_layer_vector_source/mvt_single_layer_vector_source_editor.tsx

…iles_boiler

nreese

lgtm with green CI
code review, tested in chrome

kibanamachine · 2020-04-16T20:23:29Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request
Commit: 7979d33

History

💔 Build #41300 failed f0abe33
💔 Build #41094 failed 62a3593
💔 Build #40772 failed fa3343d
💚 Build #40663 succeeded d61dba3
💔 Build #40533 failed 99caa2e

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

Adds support for adding an external vector tile service to Maps. This is experimental functionality. To enable, add `xpack.maps.enableVectorTiles: true` to the `kibana.yml`configuration file.

* master: (40 commits) [APM]Upgrade apm-rum agent to latest version to fix full page reload (elastic#63723) add deprecation warning for legacy 3rd party plugins (elastic#62401) Migrate timelion vis (elastic#62819) Replacebad scope link with actual values (elastic#63444) Index pattern management UI -> TypeScript and New Platform Ready (create_index_pattern_wizard) (elastic#63111) [SIEM] Threat hunting enhancements: Filter for/out value, Show top field, Copy to Clipboard, Draggable chart legends (elastic#61207) [Maps] fix term join agg key collision (elastic#63324) [Ingest] Fix agent config key sorting (elastic#63488) [Monitoring] Fixed server response errors (elastic#63181) update elastic charts to 18.3.0 (elastic#63732) Start services (elastic#63720) [APM] Encode spaces when creating ML job (elastic#63683) Uptime 7.7 docs (elastic#62228) [DOCS] Updates remote cluster and ccr docs (elastic#63517) [Maps] Add 3rd party vector tile support (elastic#62084) [Endpoint][EPM] Retrieve Index Pattern from Ingest Manager (elastic#63016) [Endpoint] Host Details Policy Response Panel (elastic#63518) [Uptime] Certificate expiration threshold settings (elastic#63682) Refactor saved object types to use `namespaceType` (elastic#63217) [SIEM][CASE] Create comments sequentially (elastic#63692) ...

kibanamachine · 2020-04-20T20:28:45Z

Looks like this PR has a backport PR but it still hasn't been merged. Please merge it ASAP to keep the branches relatively in sync.

Adds support for adding an external vector tile service to Maps. This is experimental functionality. To enable, add `xpack.maps.enableVectorTiles: true` to the `kibana.yml`configuration file. Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

kmartastic · 2020-07-14T22:19:40Z

Can we update this text before 7.9 releases?

thomasneirynck · 2020-07-14T22:29:09Z

Can we update this text before 7.9 releases?

@kmartastic yes, do you have a preference?

thomasneirynck added 11 commits March 31, 2020 18:29

tmp commit

4ae21f5

Merge branch 'master' into maps/vector_tiles_boiler

d33019f

more boiler

fec373b

more boilerplate

f6fd50e

Merge branch 'master' into maps/vector_tiles_boiler

49fec7b

show constants

c5ab5cc

fix typo

b638411

allow all shape types

c9468cb

enable point styling

8caba58

add points

fe67239

add display

600a007

thomasneirynck added [Deprecated-Use Team:Presentation]Team:Geo Former Team Label for Geo Team. Now use Team:Presentation v7.8.0 v8.0.0 WIP Work in progress labels Apr 6, 2020

thomasneirynck added release_note:enhancement discuss labels Apr 6, 2020

thomasneirynck marked this pull request as ready for review April 6, 2020 22:53

thomasneirynck requested a review from a team as a code owner April 6, 2020 22:53

thomasneirynck added the chore label Apr 6, 2020

thomasneirynck mentioned this pull request Apr 6, 2020

[Maps] [Meta] Add .mvt vector tile support #58519

Closed

16 tasks

remove debug cruft

d9eef55

thomasneirynck added 4 commits April 7, 2020 18:13

add feature flag

b2ed439

tmp commit

83799b4

Merge branch 'master' into maps/vector_tiles_boiler

2f66aa3

more typing

e692fc8

fix import

fa3343d

thomasneirynck requested a review from nreese April 14, 2020 18:56

nreese reviewed Apr 15, 2020

View reviewed changes

thomasneirynck added 4 commits April 15, 2020 15:51

Merge branch 'master' of github.com:elastic/kibana into maps/vector_t…

5765193

…iles_boiler

feedback (1)

6b881ce

feedback (2)

030d1e8

feedback

62a3593

thomasneirynck requested a review from nreese April 15, 2020 20:50

lukeelmers approved these changes Apr 15, 2020

View reviewed changes

nreese reviewed Apr 16, 2020

View reviewed changes

thomasneirynck added 4 commits April 16, 2020 10:57

feedback

6fcc772

Merge branch 'master' of github.com:elastic/kibana into maps/vector_t…

ec7b549

…iles_boiler

fix merge

42e27f4

change wizard loading

f0abe33

nreese approved these changes Apr 16, 2020

View reviewed changes

thomasneirynck added 2 commits April 16, 2020 13:43

Merge branch 'master' into maps/vector_tiles_boiler

688d556

fix zoom

7979d33

thomasneirynck removed WIP Work in progress discuss labels Apr 16, 2020

thomasneirynck merged commit ad41eea into elastic:master Apr 16, 2020

thomasneirynck mentioned this pull request Apr 16, 2020

[7.x] [Maps] Add 3rd party vector tile support (#62084) #63761

Merged

kibanamachine added the backport missing Added to PRs automatically when the are determined to be missing a backport. label Apr 20, 2020

kibanamachine removed the backport missing Added to PRs automatically when the are determined to be missing a backport. label Apr 21, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Maps] Add 3rd party vector tile support #62084

[Maps] Add 3rd party vector tile support #62084

thomasneirynck commented Mar 31, 2020 •

edited

Loading

elasticmachine commented Apr 6, 2020

jsanz commented Apr 7, 2020

thomasneirynck commented Apr 7, 2020

jsanz commented Apr 7, 2020

lukeelmers left a comment

nreese Apr 16, 2020

nreese left a comment

kibanamachine commented Apr 16, 2020

kibanamachine commented Apr 20, 2020

kmartastic commented Jul 14, 2020

thomasneirynck commented Jul 14, 2020

[Maps] Add 3rd party vector tile support #62084

[Maps] Add 3rd party vector tile support #62084

Conversation

thomasneirynck commented Mar 31, 2020 • edited Loading

elasticmachine commented Apr 6, 2020

jsanz commented Apr 7, 2020

thomasneirynck commented Apr 7, 2020

jsanz commented Apr 7, 2020

lukeelmers left a comment

Choose a reason for hiding this comment

nreese Apr 16, 2020

Choose a reason for hiding this comment

nreese left a comment

Choose a reason for hiding this comment

kibanamachine commented Apr 16, 2020

💚 Build Succeeded

History

kibanamachine commented Apr 20, 2020

kmartastic commented Jul 14, 2020

thomasneirynck commented Jul 14, 2020

thomasneirynck commented Mar 31, 2020 •

edited

Loading