Raster Tiles #205

quillasp · 2022-11-11T17:07:31Z

Associated with the issue #71.

This draft PR can load and display the raster tiles. It still needs a way to do different requests based on the tile type (vector or raster for the moment).

💻 Examples

🚨 Test instructions

✔️ PR Todo

Can do both types of requests
Raster data loading and storing
Raster rendering

maxammann · 2022-11-11T19:26:04Z

Just tried to run it on Linux. It compiles and runs, but its glitching quite a bit.
Do you think that is a bug with my environment or another issue?

quillasp · 2022-11-13T22:41:17Z

Just tried to run it on Linux. It compiles and runs, but its glitching quite a bit. Do you think that is a bug with my environment or another issue?

I don't have a Linux machine so I can't test right away but it shouldn't be different from other platforms. 🤔

Still finding a way to figure out different requests depending on the tile type inside the request stage

Use of a projection matrix from one space to another, the tiles are now square. Fix raster Fix raster

maxammann · 2022-11-23T20:57:55Z

@quillasp web is failing. We probably miss some more feature flags

maxammann

Looks very cool already! I mean yeah there are a few rough edges, but that is also due to some architectural changes we need to do in order to add the feature.

maplibre/Cargo.toml

maxammann · 2022-12-01T17:48:17Z

maplibre/Cargo.toml

@@ -85,6 +86,9 @@ smallvec = "1.9.0"

 # Headless
 png = { version = "0.17.5", optional = true }
+image = { version = "0.24", default-features = false, features = ["jpeg", "webp"], optional = true }


I think we might want png here aswell. Also with image we probably can remove the png crate

maxammann · 2022-12-01T17:50:43Z

maplibre/src/io/source_type.rs

+        // Used so the API key is not in the source code.
+        // Put an '.env' file in the root directory with the following content:
+        // MAPTILER_API_KEY=your_api_key
+        let api_key = env::var("MAPTILER_API_KEY").expect("MAPTILER_API_KEY must be set");


This won't work on e.g. web. Fine for now though.

We should just create a task.

maxammann · 2022-12-01T17:53:11Z

maplibre/src/io/tile_pipelines.rs

+        if let TileType::Vector(vector_tile) = &tile {
+            let available_layers: HashSet<_> = vector_tile
+                .layers
+                .iter()
+                .map(|layer| layer.name.clone())
+                .collect::<HashSet<_>>();
+
+            for missing_layer in tile_request.layers.difference(&available_layers) {
+                context
+                    .processor_mut()
+                    .layer_unavailable(coords, missing_layer)?;
+
+                tracing::info!(
+                    "requested layer {} at {} not found in tile",
+                    missing_layer,
+                    &coords
+                );
+            }


Instead of conditionally doing this we should probably create two separate pipeline steps.

maxammann · 2022-12-01T17:54:17Z

maplibre/src/io/tile_pipelines.rs

+    Raster(Vec<u8>),
+}
+
+impl From<TileType> for geozero::mvt::Tile {


Implement TryForm instead of From

maxammann · 2022-12-01T18:07:36Z

maplibre/src/render/shaders/tile_raster.vertex.wgsl

+fn main(
+    @location(0) position: vec2<f32>,
+    @location(1) tex_coords: vec2<f32>,
+    @location(4) translate1: vec4<f32>,
+    @location(5) translate2: vec4<f32>,
+    @location(6) translate3: vec4<f32>,
+    @location(7) translate4: vec4<f32>,
+    @location(9) zoom_factor: f32,
+) -> VertexOutput {
+    let z = 0.0;
+    let width = 3.0 * zoom_factor;
+    let normal = vec2<f32>(0.0,0.0);
+
+    var position = mat4x4<f32>(translate1, translate2, translate3, translate4) * vec4<f32>(position + normal * width, z, 1.0);
+    position.z = 1.0;
+
+    return VertexOutput(tex_coords, position);


Maybe some stuff is duplicated here. Fine for now though!

maxammann · 2022-12-01T18:08:28Z

maplibre/src/render/stages/upload_stage.rs

+                                                raster_resources
+                                                    .texture
+                                                    .as_ref()
+                                                    .unwrap()


If possible remove that unwrap

maxammann · 2022-12-01T18:09:05Z

maplibre/src/render/stages/upload_stage.rs

@@ -223,6 +238,66 @@ impl UploadStage {
                                        &feature_metadata,
                                    );
                                }
+                                StoredLayer::RasterLayer {


I would favor to create a new stage: upload_raster_stage and then conditionally add stat stage, instead of having the inline cfg flag

maxammann · 2022-12-01T18:10:25Z

maplibre/src/render/tile_pipeline.rs

@@ -84,6 +87,25 @@ impl RenderPipeline for TilePipeline {
                    },
                    count: None,
                }]])
+            } else if self.raster {


Not sure yet how we can improve this. This is very specific for a file which tries to keep generic.

maplibre/src/render/stages/upload_stage.rs

maxammann · 2022-12-01T18:13:30Z

maplibre/src/stages/request_stage.rs

+        #[cfg(feature = "raster")]
+        let source = SourceType::Raster(RasterSource::default());
+        #[cfg(not(feature = "raster"))]
+        let source = SourceType::Tessellate(TessellateSource::default());


Does that mean if raster is enabled then vector tiles do no longer work?

maxammann · 2022-12-12T19:07:33Z

I introduced quite a lot of merge conflicts. I will resolve them today!

# Conflicts: # maplibre/src/error.rs # maplibre/src/headless/map.rs # maplibre/src/io/pipeline.rs # maplibre/src/io/source_client.rs # maplibre/src/io/tile_pipelines.rs # maplibre/src/io/transferables.rs # maplibre/src/stages/populate_tile_store_stage.rs # maplibre/src/stages/request_stage.rs # web/src/platform/singlethreaded/apc.rs # web/src/platform/singlethreaded/transferables.rs

maxammann · 2022-12-12T21:30:08Z

Done, I also simplified the pipeline slightly and updated the APC implementation for Web.

# Conflicts: # maplibre/src/io/tile_pipelines.rs # maplibre/src/render/render_commands.rs # maplibre/src/render/stages/resource_stage.rs # maplibre/src/render/stages/upload_stage.rs # maplibre/src/render/tile_pipeline.rs # maplibre/src/stages/request_stage.rs

maxammann · 2022-12-22T09:16:15Z

@quillasp I pushed a simplification which does not require the BufferPool

# Conflicts: # maplibre/src/io/source_client.rs # maplibre/src/io/tile_pipelines.rs

maxammann · 2023-02-18T11:57:31Z

Merged in #253

maxammann changed the title ~~Rebase with the new codebase~~ Raster Tiles Nov 14, 2022

maxammann mentioned this pull request Nov 14, 2022

Feature raster tiles #141

Closed

3 tasks

Rebase with the new codebase

3c9d546

Still finding a way to figure out different requests depending on the tile type inside the request stage

quillasp force-pushed the ft-branch-raster-tiles branch 4 times, most recently from 2f87731 to 7ed0914 Compare November 23, 2022 02:33

Drabble added 2 commits November 23, 2022 04:02

Fix raster

45fbec2

Use of a projection matrix from one space to another, the tiles are now square. Fix raster Fix raster

Use hashmap to manage the texture bind groups

104ff8a

quillasp force-pushed the ft-branch-raster-tiles branch from 7ed0914 to 104ff8a Compare November 23, 2022 03:04

Hiding the raster rendering behind a Cargo feature flag

45be8a2

quillasp marked this pull request as ready for review November 23, 2022 16:01

Quillasp added 2 commits November 24, 2022 13:08

Addition of LinearLayerRaster for the web transferables

bbeaab0

Minor cleanups

6296a8e

maxammann requested changes Dec 1, 2022

View reviewed changes

maxammann reviewed Dec 1, 2022

View reviewed changes

Store raster data as RgbaImage in the tile repository

308d560

maxammann added 4 commits December 12, 2022 21:35

Fixes after merge

7e34ebe

Fix web builds (multithreaded works already)

e0357f8

Fix APC implementation

4b0f170

maxammann added 4 commits December 13, 2022 11:55

Fix tests

e2bee64

Fixes after merge

716c8b4

Replace buffer_pool changes with a more specific shader

eb40680

maxammann added 7 commits December 22, 2022 16:18

Refactor tile requests

a573a74

Fix vector tiles

74289d6

Refactor RasterResources and request stage

279a5ad

Move fns to HasTile

27b8d7a

Refactor

cdcaa64

Merge branch 'main' into ft-branch-raster-tiles

b0b83ff

# Conflicts: # maplibre/src/io/source_client.rs # maplibre/src/io/tile_pipelines.rs

Only upload raster tiles if needed

6aff8c0

maxammann closed this Feb 18, 2023

maxammann mentioned this pull request Feb 18, 2023

Introduce Plugin API along with Raster Tiles #253

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Raster Tiles #205

Raster Tiles #205

quillasp commented Nov 11, 2022 •

edited

Loading

maxammann commented Nov 11, 2022

quillasp commented Nov 13, 2022

maxammann commented Nov 23, 2022

maxammann left a comment

maxammann Dec 1, 2022

maxammann Dec 1, 2022

maxammann Dec 1, 2022

maxammann Dec 1, 2022

maxammann Dec 1, 2022

maxammann Dec 1, 2022

maxammann Dec 1, 2022

maxammann Dec 1, 2022

maxammann Dec 1, 2022

maxammann commented Dec 12, 2022

maxammann commented Dec 12, 2022

maxammann commented Dec 22, 2022

maxammann commented Feb 18, 2023

Raster Tiles #205

Raster Tiles #205

Conversation

quillasp commented Nov 11, 2022 • edited Loading

💻 Examples

🚨 Test instructions

✔️ PR Todo

maxammann commented Nov 11, 2022

quillasp commented Nov 13, 2022

maxammann commented Nov 23, 2022

maxammann left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

maxammann commented Dec 12, 2022

maxammann commented Dec 12, 2022

maxammann commented Dec 22, 2022

maxammann commented Feb 18, 2023

quillasp commented Nov 11, 2022 •

edited

Loading