Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Design Proposal: Contour Line Source from Raster DEM Tiles #583

Open
msbarry opened this issue Mar 30, 2024 · 64 comments · May be fixed by #623
Open

Design Proposal: Contour Line Source from Raster DEM Tiles #583

msbarry opened this issue Mar 30, 2024 · 64 comments · May be fixed by #623
Labels
enhancement New feature or request

Comments

@msbarry
Copy link

msbarry commented Mar 30, 2024

Design Proposal: Contour Line Source from Raster DEM Tiles

Motivation

Give users a built-in way to render contour lines in maplibre from the same DEM tiles that are already used for terrain and hillshading, like this:

image

Proposed Change

Create a new contour source type in maplibre style spec that takes a raster-dem source as input and generates contour isolines as output that can be styled using line layers, for example:

sources: {
  dem: {
    type: "raster-dem",
    encoding: "terrarium",
    tiles: ["https://elevation-tiles-prod.s3.amazonaws.com/terrarium/{z}/{x}/{y}.png"],
    maxzoom: 13,
    tileSize: 256,
  },
  contours: {
    type: "contour",
    source: "dem",
    unit: "feet" | "meters" | number, // default=meters, for custom unit use length of the unit like unit: 1.8288 for fathoms
    // similar syntax to ["step", ["zoom"], ...] style expression 
    // to define contour interval by zoom level
    intervals: [
      200, // 200m interval <=z11
      12, 100, // 100m interval at z12 and z13
      14, 50, // 50m interval at z14
      15, 20 // 20m interval >= z15
    },
    // put a "major=false/true" tag on every Nth line by zoom so styles
    // can highlight major/minor lines differently
    majorMultiplier: [
      5, // every 5th line at < z14
      14, 4, // every 4th line at z14
      15, 5 // every 5th line for >= z15
    ],
    // minzoom inferred from raster-dem source and maxzoom determined automatically by maplibre
    // overzoom z10 tiles to generate z11 contour lines, z11 to make z12, etc...
    overzoom: 1,
  },
},

The generated isolines will have these attributes:

  • ele elevation above sea level in the unit specified
  • interval the fixed interval between isolines at this zoom level in the unit specified
  • major true if this is a major isoline based on majorMultiplier at this zoom level, false otherwise

Layers can refer to the contours with source: contours but they can omit sourceLayer.

This offloads details about how to retrieve and parse DEM tiles to the DEM source definition, and gives style layers the flexibility to render any number of visible lines derived from that contour source.

I've already prototyped this in the maplibre-contour plugin which I'm using for contour lines on onthegomap.com. Here are some of the issues I had to work through to get these contours to look nice:

DEM "overzooming" (smoothing)

The contour lines look blocky when you zoom in much further past the maximum zoom for a DEM source, but they can look nice and smooth if you "overzoom" the DEM tiles by applying iterative bilinear interpolation before generating isolines. For example for onthegomap I use 512px z11 tiles, but overzoom the z11 tiles up to z15 so that the contour lines look smooth even at high zooms. This is why the proposal lets you specify a maxzoom that is higher than the maxzoom of the raster-dem source.

Also to generate smooth contour lines at the border between tiles, the algorithm needs to look at adjacent tiles. This means you need 9 DEM tiles to render a single contour tile. To mitigate this, the overzoom parameter lets you use overzoomed DEM tiles from a lower zoom level to generate contours at the current zoom level, for example overzoom=1 means use the top-left, top-right, botom-left, or bottom-right z10 tile to render a z11 contour line tile. This means you only need 4 DEM tiles to render a single contour tile:

image

Contour levels and units

The user needs to be able to choose what elevations to draw contour lines at, which changes by zoom level (rendering every contour would get too expensive at low zooms in hilly areas). They may also designate "major" and "minor" levels, for example generate thin contour lines every 200m but bold every 1000m. For now we will push this to layers that use the style, but in the future we can either add a major/minor designation to ticks, or pass-through the level and interval so styles can highlight every 5th or 10th line or something.

The unit attribute multiplies raw meter values by a certain amount to change the unit, for example unit=feet changes from meters to feet. When you click the distance indicator on onthegomap, it toggles between unit=meters and unit=feet. You could also set unit to a custom value for less common units like unit=1.8288 for fathoms.

Performance and Bundle Size

I've already implemented the smoothing logic and isoline generation in the maplibre-contour plugin so we would just need to bring that into maplibre-gl-js and port into the native projects. The overall plugin is 33kb (11kb gzipped) but most of that is replicating the web worker communication, cancelable message passing, and vector tile encoding that maplibre-gl-js already has. The actual smoothing+isoline business logic is only 3.7kb (1.6kb gzipped).

The isoline generation algorithm was derived from d3-contour but is much more efficient because it generates isolines in a single pass through the DEM tile instead of using a pass per contour level. For onthegomap users on a range of devices (mostly mobile phones) overzooming a 512px dem tile and generating isolines takes:

  • <10ms 40% of the time
  • >50ms 10% of the time
  • >100ms 2% of the time
  • >200ms 0.5% of the time
  • >500ms 0.05% of the time
  • >1s 0.006% of the time

API Modifications

This should only change the style spec, but shouldn't require any js or native API changes, unless we wanted to expose the default contour layer, elevation or level key as constants?

Migration Plan and Compatibility

This is new functionality, so no migration is necessary.

Rejected Alternatives

Build a plugin for this

I maintain the maplibre-contour plugin which already lets you do this by using the addProtocol integration, but it has a few downsides:

  1. It's an extra step to install: rendering contour lines is a common use-case that users should be able to do by default
  2. 90% of the plugin is duplicating things that maplibre already does like spawning a web worker, communicating with it using cancelable messages, and decoding DEM tiles. The actual code for computing the contours is a small fraction of the overall plugin.
  3. It has to do a wasteful extra step of encoding the result as vector tile bytes only for maplibre to decode immediately after in its own web workers (see https://github.com/onthegomap/maplibre-contour/blob/main/architecture.png)
  4. It can't make use of other registered maplibre request interceptors or protocols like DEM tiles served out of a pmtiles archive
  5. It doesn't work in maplibre-native

Pre-render contour lines

You can render contour line vector tiles ahead of time and serve those for the planet, this will save some browser CPU cycles but rendering them on the fly from DEM tiles has a few advantages:

  1. There are a lot of parameters you can tweak when generating contour lines from elevation data like units, thresholds, and smoothing parameters. Pre-generated contour vector tiles require 100+gb of storage for each variation you want to generate and host. Generating them on-the-fly in the browser gives infinite control over the variations you can use on a map from the same source of raw elevation data that maplibre uses to render terrain and hillshade.
  2. You're likely already downloading DEM tiles for hillshading and terrain, so this eliminates the extra bandwidth used to download those vector tiles.

Implement as a new layer type

We could implement this as a new layer type instead of a source type, but that would tightly couple the display parameters to the logic for how contour lines are generated, and potentially require us to generate the contours in multiple passes over the source DEM data. It seems cleaner to generate contour lines so you can generate as many layers as you want from them afterwards.

Take a DEM tile source URL as input

A contour layer could take as input tiles: ["server.com/{z}/{x}/{y}.png"], but there are a lot of different knobs to tune for how these are interpreted, so by depending on a DEM source we re-use the DEM source control all of those parameters.

@msbarry
Copy link
Author

msbarry commented Mar 30, 2024

This API is what I landed on for configuring the maplibre-contour plugin, but there are a few spots I could go either way on:

  • instead of multiplier=1 multiplier=3.28084 we could use unit=meters / unit=feet - it's less flexible but easier to use
  • do we need a way to set a minimum elevation for the isolines? You may not want bathymetry but there are plenty of places on land that are below sea level
  • should the "major" / "minor" isoline decision happen in style layers? It seemed a little more natural alongside where we define the contour line levels in the source config
  • do elevationKey contourLayer and levelKey need to be configurable?

@HarelM
Copy link
Collaborator

HarelM commented Mar 30, 2024

This is a great write up! Thanks!
Regarding using a dem source, there are some issues with which tiles to fetch and use, so it might prove better to copy some of the raster dem source definitions into this source, there aren't a lot of parameters, so duplication it might not be such a bad idea.

I agree about the units, I think feet and meters are more readable and are the only options available for the scale control.
I also think the major/minor should be a layer configuration, hopefully with some expression logic and avoid placing this configuration in the source itself.

Other than that, I think this should get it as it will give maplibre a competitive advantage over other libraries.

I'll bring it up in the next monthly web meeting, feel free to participate.

@msbarry
Copy link
Author

msbarry commented Mar 31, 2024

Thanks @HarelM!

Regarding using a dem source, there are some issues with which tiles to fetch and use, so it might prove better to copy some of the raster dem source definitions into this source

What are the issues? One of the reasons for moving out of the plugin into maplibre would be to make use of the shared DEM tile cache between sources.

I also think the major/minor should be a layer configuration

Just to illustrate the difference, my config from the demo would look like this with where the source sets a level property, and layers use it when styling the lines:

{
  sources: {
    contour_feet: {
      type: "contour",
      source: "dem",
      maxzoom: 16,
      multiplier: 3.28084,
      thresholds: {
        11: [200, 1000],
        12: [100, 500],
        13: [100, 500],
        14: [50, 200],
        15: [20, 100],
      }
    }
  },
  // ...
  layers: [
    {
      id: "contours",
      type: "line",
      source: "contour_feet",
      "source-layer": "contours",
      paint: {
        "line-color": "rgba(0,0,0, 50%)",
        "line-width": ["match", ["get", "level"], 1, 1, 0.5], // make major contours bolder
      },
      layout: {
        "line-join": "round",
      },
    },
    {
      id: "contour-text",
      type: "symbol",
      source: "contour_feet",
      "source-layer": "contours",
      filter: [">", ["get", "level"], 0], // only put labels on major contours
      layout: {
        "symbol-placement": "line",
        "text-size": 10,
        "text-field": ["number-format", ["get", "ele"], {}],
        "text-font": ["Noto Sans Bold"],
      },
    },
  ]
}

but it would look like this if the major/minor determination is entirely within the layer definition (derived from elevation):

{
  sources: {
    contour_feet: {
      type: "contour",
      source: "dem",
      maxzoom: 16,
      multiplier: 3.28084,
      thresholds: {
        11: 200,
        12: 100,
        13: 100,
        14: 50,
        15: 20
      }
    }
  },
  // ...
  layers: [
    {
      id: "contours",
      type: "line",
      source: "contour_feet",
      "source-layer": "contours",
      paint: {
        "line-color": "rgba(0,0,0, 50%)",
        // thicker major lines
        "line-width": [
          "step",
          ["zoom"],
          ["match", ["%", ["get", "ele"], 1000], 0, 1, 0.5],
          12,
          ["match", ["%", ["get", "ele"], 500], 0, 1, 0.5],
          14,
          ["match", ["%", ["get", "ele"], 200], 0, 1, 0.5],
          15,
          ["match", ["%", ["get", "ele"], 100], 0, 1, 0.5],
        ]
      }
    },
    {
      id: "contour-text",
      type: "symbol",
      source: "contour_feet",
      "source-layer": "contours",
      // only put labels on major lines
      filter: [
        "step",
        ["zoom"],
        ["==", ["%", ["get", "ele"], 1000], 0],
        12,
        ["==", ["%", ["get", "ele"], 500], 0],
        14,
        ["==", ["%", ["get", "ele"], 200], 0],
        15,
        ["==", ["%", ["get", "ele"], 100], 0],
      ],
      layout: {
        "symbol-placement": "line",
        "text-size": 10,
        "text-field": ["number-format", ["get", "ele"], {}],
        "text-font": ["Noto Sans Bold"],
      },
    },
  ]
}

The second one makes layer definitions repeat a lot, and also requires keeping the major/minor level logic in sync with the level thresholds by zoom from the source. For example if you have 50m lines with 250m major lines, but you change the interval to 100m then the 250m/750m major lines will never show up.

@HarelM
Copy link
Collaborator

HarelM commented Mar 31, 2024

What are the issues?

In terrain it is advised to use a different source instead of using the one for hillshade as the logic of which tile to fetch is a bit different due to how the terrain works.

Regarding the source and layer coupling, it is the same for other vector sources as well, if you change the definition of a layer in the source you'll need to adapt the style.
Although you can place certain features in certain zoom levels in the source, so I'm not sure what's "cleaner".

I'll post it on slack to hopefully get more feedback.

@msbarry
Copy link
Author

msbarry commented Mar 31, 2024

In terrain it is advised to use a different source instead of using the one for hillshade as the logic of which tile to fetch is a bit different due to how the terrain works.

Is this because we want to switch what zoom level DEM tile we use based on the current zoom differently with terrain vs. hillshade? It will be slightly different for contour lines too if you set the overzoom parameter, but it seems like they should still all be able to pull from a shared tile cache and share the decoding config?

@HarelM
Copy link
Collaborator

HarelM commented Mar 31, 2024

I'm not entirely sure about the details, but there is an opposite direction between cache and source I think, source cache is managing a source instead of the other way around.

@1ec5
Copy link
Contributor

1ec5 commented Mar 31, 2024

instead of multiplier=1 multiplier=3.28084 we could use unit=meters / unit=feet - it's less flexible but easier to use

If you’re considering bathymetry, then fathoms might also be relevant. Apart from that, would you care for a map contoured in smoots? 😎

@voncannon
Copy link

Wow, this looks great to me. Since in most cases we have hillshading /terrain anyways and thus the bytes are already going through the tubes anyways...

Ship it.

What am I missing?

@msbarry
Copy link
Author

msbarry commented Apr 1, 2024

I'm not entirely sure about the details, but there is an opposite direction between cache and source I think, source cache is managing a source instead of the other way around.

OK, this seems like a gap in the current implementation that would be unfortunate to leak into the style spec. It seems like terrain/hillshading/contours reading from the same tiles should be able share a source definition... I'll understand the limitations better when I get into the implementation, but I think it would be good to try to shoot for something consistent with how terrain lets you refer to a raster source by ID, then adjust if that looks like it's causing more problems than it's worth?

@HarelM
Copy link
Collaborator

HarelM commented Apr 1, 2024

Sure, implementation details of the web library shouldn't affect the spec.
There's no cross reference between sources in the spec right now, but I'm not saying there shouldn't be.
I'm fine either way.

@msbarry
Copy link
Author

msbarry commented Apr 1, 2024

If you’re considering bathymetry, then fathoms might also be relevant. Apart from that, would you care for a map contoured in smoots? 😎

Another option here would be something like:

unit: 'feet' | 'meters' | number

So we get the convenience of being able to specify the most common unit by name, but flexibility of being able to use unit=1.7018 for smoots?

@nitrag
Copy link

nitrag commented Apr 1, 2024

What is the performance of downloading 1 PNG for Hillshade and 1 PNG for Countour versus downloading a DEM and the device calculating/drawing?

@HarelM
Copy link
Collaborator

HarelM commented Apr 2, 2024

The calculations are different, and the browser's cache can help in case it's the same tiles, but this proposal is for native as well so let's try and focus on the style spec changes.
The implementation details can be discussed in the relevant repo in a PR or an issue.

@msbarry
Copy link
Author

msbarry commented Apr 2, 2024

For my setup currently, it's around:

  • tile fetch p50=90ms p90=300ms p99=1s
  • 512px contour generation p50=17ms p90=50ms p99=80ms

Although that includes browser cache, and the extra step of encoding the result as vector tile bytes, which won't be necessary any more. 256px tiles should also take 1/4 the time.

Vector tile decoding and processing also takes a nontrivial amount of time, often longer than the network request to fetch the tile on web.

@wipfli
Copy link
Contributor

wipfli commented Apr 8, 2024

@msbarry I think it is a good idea to add the contour lines plugin's functionality directly to MapLibre GL JS, because

  • contour lines are a widespread use case
  • a lot of code duplication can be avoided
  • size impact and maintenance impact on GL JS are minimal

Regarding the implementation in the style spec, I found it sometimes confusing how the major and minor lines were defined in the plugin with the mapping from thresholds to levels. Maybe we could rename thresholds to levels?

@msbarry
Copy link
Author

msbarry commented Apr 11, 2024

We discussed at the monthly steering committee meeting, it sounds like there's general consensus to move forward on this. I'll try to simplify the source definition a bit and see what other similar tools do to specify contour levels and major/minor ticks for comparison.

My open questions:

  • do the overzoom/maxzoom parameters make sense?
  • are the attributes that control the generated vector feature attributes necessary? (elevationKey/levelKey/contourLayer) If we exclude them then there will just be some magic/default layer and tag names that layer styles need to use to render the lines.

@HarelM
Copy link
Collaborator

HarelM commented Apr 11, 2024

I think a user using this source should be able to tweak these parameters in theory, having said that, I think we can start off by using some hardcoded "magic" strings in the original definition to avoid the extra complexity, if there's a need to tweak them we can add support for it later on in the next versions.
Another option is to have them in the spec and have default values, this way there's no need to specify them explicitly if one doesn't need to change the defaults.
Either options work for me, I personally prefer simplicity and a two phase manner seems like a good approach (i.e. if no one needs to actually tweak this and the defaults are good, then why complicating, right?).

Another input from the meeting yesterday was the zoom definitions, mush like the above, I think we can start of by allowing the logic in the layers and then adding extra complexity if needed in the future.

@lseelenbinder
Copy link
Member

Excited to see this coming to MapLibre!

The multiplier attribute multiplies raw meter values by a certain amount to change the unit, for example multiplier=3.28084 changes from meters to feet. When you click the distance indicator on onthegomap, it toggles between multiplier=1 and multiplier=3.28084.

It may be advisable (on the implementation side, so this is a note for future reference) to include a few common constants we document for people, so we don't have to look up meters to feet indefinitely in the future. 😄

@HarelM
Copy link
Collaborator

HarelM commented Apr 11, 2024

About the multiplier, see @msbarry 's comment which I think is a good solution for this:
#583 (comment)

@lseelenbinder
Copy link
Member

Oh! I missed that. Yes—that's even better.

@msbarry
Copy link
Author

msbarry commented Apr 11, 2024

I think a user using this source should be able to tweak these parameters in theory, having said that, I think we can start off by using some hardcoded "magic" strings in the original definition to avoid the extra complexity, if there's a need to tweak them we can add support for it later on in the next versions.
Another option is to have them in the spec and have default values, this way there's no need to specify them explicitly if one doesn't need to change the defaults.
Either options work for me, I personally prefer simplicity and a two phase manner seems like a good approach (i.e. if no one needs to actually tweak this and the defaults are good, then why complication, right?).

OK great, I updated the definition to include unit = "feet" | "meters" | number and removed elevationKey/levelKey/contourLayer. I kept overzoom, as I think it's important to be able to change since it drives the smoothness of generated lines and tile over-fetching. I could see defaulting it to 1 but still seems like it should be configurable.

It may be advisable (on the implementation side, so this is a note for future reference) to include a few common constants we document for people, so we don't have to look up meters to feet indefinitely in the future. 😄

@lseelenbinder are there any other unit constants you think we should include besides feet and meters to start?

@msbarry
Copy link
Author

msbarry commented Apr 11, 2024

Another input from the meeting yesterday was the zoom definitions, mush like the above, I think we can start of by allowing the logic in the layers and then adding extra complexity if needed in the future.

I think we can omit the major/minor designation for now, but we do need the source to specify at what interval contour lines should be generated. The contour generation performance is proportional to the number of lines that end up on the tile, so if we pick the lowest common denominator then low zoom tiles in hilly areas will slow to a crawl.

For comparison, gdal-contour lets you specify:

  • -a <name> Provides a name for the attribute in which to put the elevation. If not provided no elevation attribute is attached. Ignored in polygonal contouring (-p) mode.
  • -i <interval> Elevation interval between contours.
  • -off <offset> Offset from zero relative to which to interpret intervals.
  • -fl <level> Name one or more "fixed levels" to extract.
  • -e <base> Generate levels on an exponential scale: base ^ k, for k an integer.
  • -nln <name> Provide a name for the output vector layer. Defaults to "contour".

It also lets you generate polygons (isobands instead of isolines) which I'd probably try to avoid in maplibre unless there's a strong use-case?

d3-contour requires you specify a list of "thresholds" (where my original terminology came from).

The most flexible version I could see ending up with might look like:

levels: {
  [zoom: number]: interval | [list, of, thresholds] | { min, max, offset, exponent, ...}
}

but we could start by just supporting a single number then add more complex variations based on feedback?

Another option instead of a map from zoom level to interval would be to allow a zoom-based expression, like:

levels: [
  "step", ["zoom"],
  200,
  11, 100,
  14, 50,
  15, 20
]

WDYT @HarelM ? I'm not sure if it will cause trouble running a maplibre expression outside the context of an individual feature?

@HarelM
Copy link
Collaborator

HarelM commented Apr 11, 2024

Yea, I wouldn't want to try and use the expression engine in a source property, sounds like asking for trouble.
I get how generating "too many" contours for a specific zoom can be problematic.
I like the term intervals more than levels.
In theory one could split the work between two or more sources to reflect min and max zoom for each different interval, but I'm not sure it's any better...

@lseelenbinder
Copy link
Member

It also lets you generate polygons (isobands instead of isolines) which I'd probably try to avoid in maplibre unless there's a strong use-case?

These can create really cool effects (e.g., bathymetry layering), but I don't think it's a problem to leave it off the spec because if we actually have a strong use-case, it's easy enough to add later.

I also prefer intervals to levels because

-fl <level> Name one or more "fixed levels" to extract.

is actually something different than intervals, IIUC. It's more about specific set of levels (e.g., only 0, 250, 1000), so we could potentially add that type of functionality in the future.

@msbarry
Copy link
Author

msbarry commented Apr 12, 2024

These can create really cool effects (e.g., bathymetry layering), but I don't think it's a problem to leave it off the spec because if we actually have a strong use-case, it's easy enough to add later.

Ah sorry to clarify I avoided the isobands because I'm not sure if there's a way to do it efficiently enough in the client yet. D3-contour creates all of the contour polygons then tests which ones contain the others to assign shells and holes, which can be expensive compared to naively creating the lines and not having to worry about closing them. There's probably a way to do it with minimal performance impact, I just haven't fully thought through that yet.

I like the term intervals more than levels.

👍 changed the definition to intervals. @HarelM that makes sense about not wanting to use expressions directly, but people have already learned the quirks of the step expression, instead of adding a new { [zoom]: interval } format that people need to learn what happens when a level is missing, what if we re-use the step syntax without it actually being an expression?

intervals: [
  200, // value when < z11
  11, 100, // >= z11
  14, 50, // >= z14
  15, 20 // >= z15
]

Then people could shorten to

intervals: [200]

or

intervals: 200

if they want the same contours at every zoom?

@HarelM
Copy link
Collaborator

HarelM commented Apr 12, 2024

I see what you mean with the fact that one needs to define what happens for every zoom level, and a "dictionary" style syntax would be very verbose if you have 10 zoom levels for example.
The above syntax let you define what happens for a range of zoom levels.
I don't like using arrays this way and it's super hard to validate, debug, etc, but since, as you said exist in this library in other places in the style, the users may know how to deal with it.
I think it makes sense.
I would love to hear other opinions.
The initial post has the most up-to-date spec that is propose, and at it current state I think it should be approved.
Let's give people one more week to look at this and comment.

@wipfli
Copy link
Contributor

wipfli commented Apr 13, 2024

I would prefer a more explicit definition like for example this one:

intervals: [
  0, 10, 200,   // if  0 <= z <= 10, use 200
  11, 13, 100,  // if 11 <= z <= 13, use 100
  14, 15, 50,   // if 14 <= z <= 15, use 50
  15, 21, 20    // if 15 <= z <= 21, use 20
]

This has on the other hand the downside that the max zoom has to be specified.

EDIT: I might be wrong. The syntax that you suggested @msbarry seems closer to the already existing step syntax...

@1ec5
Copy link
Contributor

1ec5 commented Apr 14, 2024

what if we re-use the step syntax without it actually being an expression?

I appreciate how this approach would leave open the door to introducing support for more flexibility in the future without requiring a brand-new syntax or some sort of deprecation dance. There’s already plenty of precedent in the style specification and API for allowing an expression but not any arbitrary expression. On the other hand, if someone sees a part of the style specification that allows a step expression, they’ll likely be surprised at first if interpolate is unsupported. Regardless, make sure to avoid designing anything that could be confused with zoom functions, which have lingered on long past their expiration date.

@msbarry
Copy link
Author

msbarry commented Apr 14, 2024

@1ec5 I was thinking that it would just include the values from within a ["step", ["zoom"], ...] expression, like just interval: [100, 11, 50] to do 100 <z11 then 50 at z12 and above. That makes it so there's no question about using a different expression type, but also less clear of an upgrade path to more complex expressions.

@HarelM
Copy link
Collaborator

HarelM commented Apr 15, 2024

Here's my understanding of the proposal in a way that we can test and see what can and can't be done:
I've created these jsbins with a geojson source with 5/6 lines, each has an elevation (ele) and the interval that was configured to the relevant zoom level:
Interval 10:
https://jsbin.com/bayesotibo/2/edit?html,output
Interval 20:
https://jsbin.com/bayesotibo/3/edit?html,output
I've created a major and minor using only these two fields - ele and interval using a not so complicated expression (to my taste).

Let's try and continue the conversation using these kinds of examples, I think it will be more productive.

@HarelM
Copy link
Collaborator

HarelM commented Apr 20, 2024

To conclude the above discussion about major and minor lines, given the assumption that major and minor lines are a very common use case I propose the following optional addition to the spec by adding a multiplier that can be changed per zoom level called majorMultiplier (or a better name if you can think of one).
every contour line will have the properties of ele for the elevation and interval for the current zoom interval.
If majorMultiplier is specified every N line will also get a isMajor boolean set to false or true according to multiplier.
This will allow the flexibility to do other stuff other than major and minor by adding filter logic to the layers based on interval value and also simplify the common use case of major and minor lines.
I took the above proposal and modified it to include the "special" zoom 14 case, in a regular case you might probably just write majorMultiplier: [5].

sources: {
  dem: {
    type: "raster-dem",
    encoding: "terrarium",
    tiles: ["https://elevation-tiles-prod.s3.amazonaws.com/terrarium/{z}/{x}/{y}.png"],
    maxzoom: 13,
    tileSize: 256,
  },
  contours: {
    type: "contour",
    source: "dem",
    unit: "feet" | "meters" | number, // default=meters
    // similar syntax to ["step", ["zoom"], ...] style expression 
    // to define contour interval by zoom level
    intervals: [
      200, // 200m interval <=z11
      12, 100, // 100m interval at z12 and z13
      14, 50, // 50m interval at z14
      15, 20 // 20m interval >= z15
    },
    majorMultiplier: [
      5, // every 5th line at < z14
      14, 4, every 4th line at z14
      15, 5 // every 5th line for >= z15
    ],
    minzoom: 11,
    // overzoom z13 DEM tiles up to z16 to generate smooth high-zoom contour lines:
    maxzoom: 16,
    // overzoom z10 tiles to generate z11 contour lines, z11 to make z12, etc...
    overzoom: 1,
  },
},

@zstadler
Copy link
Contributor

I like this approach! It provides a solution for the 80% of simple configurations and a solution for the other 20% of complex needs.

I'd like to suggest the major name for the proposed attribute. Most attributes used in vector tiles are taken "as-is" from OSM where a similar naming convention is used (oneway, fee, covered, ...).

@msbarry
Copy link
Author

msbarry commented Apr 21, 2024

This looks good to me, thanks @zstadler @HarelM! I updated the definition at the top of the thread with majorMultiplier, clarified the generated isolines will have ele interval and major tags, and that layers can just specify source but omit sourceLayer. I'll start working on a PR this week.

@zstadler
Copy link
Contributor

If majorMultiplier is specified every N line will also get a isMajor boolean set to false or true according to multiplier.

Instead, I suggest that only major=true will be set where appropriate:

If majorMultiplier is specified, every N'th line will also have a isMajor boolean property set to true. Contour lines never have a isMajor=false property

It is customary in OSM and in MVT to omit feature attributes that are unset or have a default value .

@HarelM
Copy link
Collaborator

HarelM commented Apr 21, 2024

I'll start working on a PR this week.

Great!! Don't forget to start a PR to this repo for both updating the v8.json file, the spec docs and the validation and diff methods in this repo.
In parallel you can use the output of this package in maplibre-gl-js or wait to finish here so this package will be released and only later continue with a PR to the other repo.
Which ever works for you best.
THANKS!!

@wipfli
Copy link
Contributor

wipfli commented Apr 22, 2024

Looks good to me in general, thanks for the suggestion @HarelM. The only thing that seems off to me is the name majorMultiplier because nothings seems to be multiplied by it.

Maybe it should rather be be majorModulo? I imagine the formula will be something like

var isMajor = (lineCount % majorModulo === 0)

@HarelM
Copy link
Collaborator

HarelM commented Apr 22, 2024

Yeah, I'm not super happy with the name either.
It's more close to "major evety N line", I'm not sure if modulo captures that essence, but I don't have hard feelings either way and I can't think of a better name too...

@zstadler
Copy link
Contributor

I've asked Gemini:

What is the English word to describe something that occurs ever N times of something else where N is a fixed number?

It replied:

There isn't a single, perfect word in English to describe something that occurs every N times of something else. However, depending on the context, here are some options you can consider:

  • N-th: This is a simple and clear way to indicate the specific frequency. For example, "The heartbeat is an N-th occurrence of the electrical signal in the heart."
  • Periodic: This implies a repeating pattern with a fixed interval, which works well if N represents a regular interval. "The train runs periodically every 30 minutes."
  • Cyclic: Similar to periodic, but might emphasize a complete cycle happening every N times. "Sales figures exhibit a cyclic pattern, peaking every quarter."
  • Cadence: This refers to a specific rhythm or pattern of occurrence. "The blinking of a firefly follows a specific cadence." (This might be less appropriate for high values of N)
  • Multiplicity: This emphasizes the fact that something happens in multiples of N. "The engine check light comes on at a multiplicity of 5,000 miles."

The best option depends on the specific situation and the nuance you want to convey.

It looks like the use of the multiplicity term is a viable option.

@msbarry
Copy link
Author

msbarry commented Apr 22, 2024

What about majorIntervals for consistency with intervals ? Although that might make you think the value should be specified in the same unit as intervals. We could also just use major or majors

@zstadler
Copy link
Contributor

Although that might make you think the value should be specified in the same unit as intervals

That's exactly why I like the "multiplier" part.

@wipfli
Copy link
Contributor

wipfli commented Apr 22, 2024

I like majors. Feel free to go with whatever you think is best @msbarry...

@voncannon
Copy link

Naming is hard. So adding more terminology to the docket...

I think traditionally the thicker contour lines are called index contours:

  1. page 9 https://pubs.usgs.gov/circ/1955/0368/report.pdf
  2. https://www3.nd.edu/~cneal/planetearth/Lab-SurfaceHydrology/TopoMaps.html

Following the example below, one advantage here is both intervals and indexIntervals track the same unit as specified .. rather than one being a unit and one a interval/multiplier/skip/etc, respectively.

N/multiplier/skip can be computed per zoom level under the covers: multiplier = index_interval / interval

sources: {
  dem: {
    type: "raster-dem",
    encoding: "terrarium",
    tiles: ["https://elevation-tiles-prod.s3.amazonaws.com/terrarium/{z}/{x}/{y}.png"],
    maxzoom: 13,
    tileSize: 256,
  },
  contours: {
    type: "contour",
    source: "dem",
    unit: "feet" | "meters" | number, // default=meters
    // similar syntax to ["step", ["zoom"], ...] style expression 
    // to define contour interval by zoom level
    intervals: [
      200, // 200m interval <=z11
      12, 100, // 100m interval at z12 and z13
      14, 50, // 50m interval at z14
      15, 20 // 20m interval >= z15
    },
    indexIntervals: [
      5, // every 1000m at < z14
      14, 200, every 200m at z14
      15, 100 // every 100m for >= z15
    ],
    minzoom: 11,
    // overzoom z13 DEM tiles up to z16 to generate smooth high-zoom contour lines:
    maxzoom: 16,
    // overzoom z10 tiles to generate z11 contour lines, z11 to make z12, etc...
    overzoom: 1,
  },
},

@HarelM
Copy link
Collaborator

HarelM commented Apr 22, 2024

If the indexInterval (the word index is far from something intuitive to me BTW, but it's not important for this comment) isn't a full step of the interval things become tricky.
What happen if the interval is 100m and you want to place an index line in 250m?
Is it valid? If not, do you get a console warning?
The idea behind the multiplier is that effectively, the above can't happen. It forces integer multiplier.
I also believe the common case would be the 5th line, or something similar, for all zoom levels which doesn't require to rewrite the intervals again, but simply write “[5]“.
I have thought about what you are proposing before posting my suggestion, but given the above (which can be solve obviously, not saying they are a blocker) I believe what I suggested would be better.

@voncannon
Copy link

voncannon commented Apr 22, 2024

Great thoughts, I agree, that could make a mess of things where as the multiplier directly in the options wouldn't allow the user to get in a pickle as easy.

With that said, the index contour terminology still goes. But either way for me, major/index. I just think traditionally index is what has been used.

@msbarry
Copy link
Author

msbarry commented Apr 23, 2024

I did find that there is one place where a source definition already uses a style expression for the geojson clusterProperties field:

const [operator, mapExpr] = value.clusterProperties[prop];
const reduceExpr = typeof operator === 'string' ? [operator, ['accumulated'], ['get', prop]] : operator;
errors.push(...validateExpression({
key: `${key}.${prop}.map`,
value: mapExpr,
validateSpec,
expressionContext: 'cluster-map'
}));
errors.push(...validateExpression({
key: `${key}.${prop}.reduce`,
value: reduceExpr,
validateSpec,
expressionContext: 'cluster-reduce'
}));

Think we should continue with the abbreviated step-zoom syntax? Or turn intervals and major/index into expressions? We still might run into issues because it's not operating in the context of an individual feature...

@HarelM
Copy link
Collaborator

HarelM commented Apr 23, 2024

I believe it makes more sense to have an expression there if this is not the first time an expression is used, the documentation will need to provide a good example and describe what properties are available in this context (probably only zoom, if I needed to guess).

If we encounter serious issues with the implementation we can reduce the complexity of the spec, but I think it makes sense to try.

@msbarry msbarry linked a pull request Apr 23, 2024 that will close this issue
11 tasks
@zstadler
Copy link
Contributor

I does seem that the term "Index" is used by English speakers, at least in the US. My efforts to find the UK term were unsuccessful...
Perhaps a index-multiplier keyword would make things clear for most people.

It uses kebab-case, similar to other keywords in the style spec, such as icon-text-fit-padding.

@msbarry
Copy link
Author

msbarry commented May 4, 2024

What do people think of omitting maxzoom from the contour source definition? Getting into the gl-js implementation, there are 4 stages of the data transformation:

  1. raw DEM tiles
  2. "overzoomed" DEM tiles with smoothed values ready to generate contours from
  3. vector tile with contours generated from overzoomed DEM tile
  4. overzoomed vector tile

So the overzooming to get from a z12 raster DEM tile to a z16, 17, 18+ vector tile could happen in either step 2 or 4. If you set the source maxzoom parameter correctly, the results will look nearly identical, but if you set them wrong they could look bad.

I'd propose that we let maplibre manage how much happens in 2 vs. 4 in order to optimize performance and visual appearnce without a user needing to know how the internals work. Is it weird for a source not to have a maxzoom though?

@msbarry
Copy link
Author

msbarry commented May 4, 2024

And for unit I was initially thinking of it as a multiplier (like 3.28084 for feet) but wondering if it makes more sense for it to be the length of the actual unit (0.3048 for feet, 1.8288 for fathoms)?

@msbarry
Copy link
Author

msbarry commented May 4, 2024

Making progress on the gl-js implementation... I've got an initial rough proof of concept working:

image

Still need to figure out source tile caching and handling border pixels but I have found that using expressions in the source definition for intervals and major/index works well (both on the evaluation side, and validating that it only uses zoom as an input)

@HarelM
Copy link
Collaborator

HarelM commented May 4, 2024

I think it makes sense to relay on the definitions of the referenced source to simplify stuff. This is the first source that references another source, so it's the first of it's kind.

Light already uses expression in the position property, so I believe using expression here for the interval would make sense as well.

@zstadler
Copy link
Contributor

zstadler commented May 4, 2024

What do people think of omitting maxzoom from the contour source definition?

Perhaps there is no need for minzoom, maxzoom and overzoom in a contour source definition. Let the contour source implementation do its best given the minzoom and maxzoom of the Raster DEM source it is based on, and let each style layers define its minzoom and maxzoom for what needs to be shown in the map.

No harm will be done if the style layer has no minzoom and the contour source cannot perform under-zoom. Similar to hillshade style layers, it will just not be shown.

@msbarry
Copy link
Author

msbarry commented May 4, 2024

Perhaps there is no need for minzoom, maxzoom and overzoom in a contour source definition. Let the contour source implementation do its best given the minzoom and maxzoom of the Raster DEM source it is based on, and let each style layers define its minzoom and maxzoom for what needs to be shown in the map.

That sounds good to omit minzoom/maxzoom - "overzoom" is a bit different though - it uses lower zoom tile to generate higher zoom contours so that 1) the results look smoother and 2) there's less data to fetch to cover the viewport, for example:

overzoom=0 image overzoom=1 image overzoom=2 image overzoom=3 image

In practice I've found an "overzoom" of 1 works well with maplibre - an overzoom of 0 has too many kinks in the lines for mapilbre's label placement to find suitable spots to put the height labels (although there are probably improvements we could make to label placement to automatically "de-kink" spots like this to improve line label placement)

Maybe this is a concept we'd want to enable across other sources? Or maybe we it should be called something like "smoothing"

@zstadler
Copy link
Contributor

zstadler commented May 4, 2024

It seems like some peaks loose elevation when smoothing/overzoom is applied:

image

The size and shape of the 7,000' contour line around this peak also changes significantly as smoothing is increased.

In practice I've found an "overzoom" of 1 works well with maplibre - an overzoom of 0 has too many kinks in the lines for mapilbre's label placement to find suitable spots to put the height labels (although there are probably improvements we could make to label placement to automatically "de-kink" spots like this to improve line label placement)

A similar issue exists for rivers and potentially other natural features. I mitigated that by increasing text-max-angle and text-letter-spacing.

If the elevation labels are the reason for implementing smoothing/overzooming, then I suggest addressing the issue in the symbol layer of the labels, rather than decreasing the accuracy of the contour lines.

P.S., I would love to see "smoothing" available for line labels in general. Perhaps this is a topic for a different discussion.

@zstadler
Copy link
Contributor

zstadler commented May 4, 2024

Some maps, including OpenTopoMap and later also Israel Hiking Map, help the reader to identify the "up" direction by orienting the labels according the direction of the slope:

I wonder how this label would behave when "text-keep-upright": false is set in the contour labels layer definition.

image

If the label keeps this orientation, please reverse the direction of the contour lines. The reversal would only affect label layers that use "text-keep-upright": false and enable achieving the desired orientation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

9 participants