Design proposal: Support arbitrary base and interval values in RasterDEM sources

[ibesora]

Design Proposal: Support arbitrary base and interval values in RasterDEM sources

Motivation

Maplibre assumes a base of -10000 and an interval of 0.1 when decoding Mapbox RasterDEMs (see here).
This limits the range of values you can have on a terrain-rgb-like raster.

Proposed Change

Introduce two new options to RasterDEMSource:

• base: This will define the base value of the raster DEM encoding. This should default to -10000
• interval: This will define what each decoded RGB value accounts for. This should default to 0.1

Right now, Mapbox terrain-rgb encoding is defined as ((r * 256 * 256 + g * 256.0 + b) / 10.0 - 10000.0) but with these changes we could define it as ((r * 256 * 256 + g * 256.0 + b) * interval + base) and support custom encodings.

Optional

We could also go all the way to a generic implementation of the decoding and add the following option to RasterDEMSource instead:

• unpack: A vector that defines 4 values. The first three would multiply RGB channels and the 4th would be the base. It would default to [6553.6, 25.6, 0.1, 10000.0] when using Mapbox encoding and to [256.0, 1.0, 1.0 / 256.0, 32768.0] when using Terrarium encoding. When this is defined, encoding should have no effect.

API Modifications

See the above section

Migration Plan and Compatibility

Using the proposed defaults there should be full compatibility with previous versions

Rejected Alternatives

Instead of defining base/interval or unpack parameters, we could be adding new encoding types but that's not scalable enough.

[wipfli]

Interesting idea, thanks @ibesora. Why was height encoded as rgb in the first place? Could one not use a single channel TIF instead?

I'm asking because maybe we could go one step further introduce a new format altogether...

[HarelM]

Generally speaking, when I think about how terrain RGB is stored I find it out dated.
Yes, it's easy to read and parse, but extremely not efficient in terms of storage.
I would expect something like a polynom representation where you only store the modifiers, or a sparse matrix, but, I might be off here.
In any case, adding some parameters to the terrain RGB spec sounds like a good idea instead of assuming this hardcoded values, which can be the default.

As a workaround, you can do it now with addProtocol in GL JS, but it won't be very efficient...

[dnomadb]

To answer the questions that @HarelM and @wipfli bring up, I think it is worthwhile breaking this issue into 3 sections: past, present, future:

Past

Why was rgb encoding chosen?

The inception of terrain-rgb encoding dates back to the tilemill era, where it was originally created to display elevation on a Leaflet canvas layer. Having an image format was advantageous because adding data to the canvas was fast and simple.

We later used terrain-rgb at Mapbox for our global terrain data, where I came up with the current base and interval through some pretty rudimentary methods (experimentation).

• The base was just to make sure we had enough room for the surface of the dead sea
• The interval was just enough precision we needed to make hillshading look good at zoom 14

Present

Why are we still using it?
This is going to vary user by user, but IMO it has stuck around because while the encoding isn't neccesarily efficient (its tied to the encoding precision, really), having a format that "just works" in the browser is advantageous. Any replacement would have to be more efficient, just as easy to decode in the client, and have support across multiple platforms.
It's a hack, but it continues to be simple to implement, and despite the less than perfect encoding, in many cases it / terrarium are still better than having to write a new encoder / ship added code and complexity.
I also am 👍 on the proposal because it would allow usage of sources that don't work well with the current base / interval:

• Bathymetry sources (the Marianas trench is below the current base)
• High resolution terrain data (the encoding of 0.1 m isn't high enough precision to support it).

Future

Rather than write a custom format, I think it is worth holding out until there is JPEG XL support in all major browsers -- it supports higher bit depth encoding out of the box.

[wipfli]

Thanks for the background info @dnomadb

[wipfli]

@louwers this proposal should be simple to add to MapLibre Native, right?

[HarelM]

@dnomadb thanks for the great explanation!
JPEG XL sounds great! It would be interesting to see if it can be incorporated in maplibre using addProtocol as a POC to see if and how much compressed it is compared to the current terrain RGB.
What I'm trying to say is that we might not need to actually "invent" a new format reader and writer but rather use things that already exist, for example some sort of 3D mesh representation and compression (here's the first google search results I found: https://aa.bbs.tr/publication/imsec/A-Brief-History-of-3D-Mesh-Compression.pdf).
It might be that compressing an image is the best course of action and doing a reduction to this problem is the best solution, IDK, there are smart algorithm developers that can probably answer this question better than I can :-)
I agree about the simplicity of terrain RGB in terms of reading, viewing, etc, but raster also falls under that category, and I tend to think that vector tiles are "winning" now in terms of usability, costs, etc - I think the major factor of this is the shift of the processing to the client to reduce computation and storage costs, I don't think terrain tiles as different in that aspect.
cc: @nyurik - as this might be interesting for you as a continuance to the next gen vector tiles discussion.

[HarelM]

P.S. found this about JPEG XL:
https://arstechnica.com/gadgets/2023/04/free-software-group-decries-google-dropping-space-saving-jpeg-xl-format/
Also it seems that chrome removed the support for it recently... The above article suggests that AV1F is an alternative format.

[louwers]

Instead of defining base/interval or unpack parameters, we could be adding new encoding types but that's not scalable enough.

What do you mean that it is not scalable enough? I don't know much about raster DEM, how many encoding types are there?

What if someone defines both an encoding and one of the new parameters? I feel like an encoding value of custom would need to be provided.

@wipfli Yes.

[ibesora]

What do you mean that it is not scalable enough? I don't know much about raster DEM, how many encoding types are there?

@louwers potentially, infinite. Mapbox uses -10000 and 0.1 as base/interval, as @dnomadb pointed out above, that doesn't allow to represent the lowest currently known point on earth for example. Even if assuming that will never change and setting a lower base would be enough, there can be DEMs with different resolutions so settling on 0.1m precision is limiting. What I meant with that is that adding different named encodings will just allow to work on a subset of those valid encodings.

What if someone defines both an encoding and one of the new parameters? I feel like an encoding value of custom would need to be provided.

My proposal was to not use the encoding parameter when one of the new parameters is used but adding a new "custom" value makes sense

[louwers]

Understood! Makes sense.

Another option may be to allow passing an object or an array to encoding. But I don't know how common it is in the spec to overload a single parameter with multiple types. My concern is just that we should ensure that everything is orthogonal so that there is no confusion which takes precedence.

[wipfli]

Can the parameters be stored as metadata of the rgb tiles? Then we could have dynamic encodings...

[ibesora]

We would need to do the decode ourselves then, wouldn't we? The nice thing about using plain-old PNG is that the browser does the decoding for us.
Also, I see this as a per-source thing, not per-tile. Having different base/encodings on each tile would show artifacts on the hillshade layer

[wipfli]

If you get a terrain rgb file at the moment, then you know it can only be one of two encodings. That is convenient because the person consuming the rgb tiles might not know about what encoding the person generating the tiles used.

With this proposal, the person generating the terrain tiles needs a way to communicate the encodings to the user. Doing this automatically would be favorable I think and the first thing that came to my mind was to store it as some sort of metadata on the source.

Tonight we have the monthly meetings. @ibesora or @dnomadb do you have time to join?

[ibesora]

I will be joining, yeah

[stefankarschti]

Custom fields / textual information can be added to PNGs through text chunks or custom chunks. I'm not sure about web readers, but those can be read from native using lib png.

[acalcutt]

I ran into this issue with base and interval recently when I was messing with gebco bathymetry. I had noticed a spike it terrain in the Marianas trench, which I assume was because the elevation was out of range. Some suggestions were to change the base value, which did work, but meant maplibre is interpreting heights incorrectly, which seems to affect camera/zoom

One idea I had was to switch to using the terrarium encoding, which maplibre already has support for. terrarium default values easily support -12000 to 9000 meters.

Based on that idea, I looked for some tools to make my own terrarium based on the jaxa geotiffs I had, but didn't really find an easy way to do it. I ended up modifying my version of rio rgbify to support an 'encoding' flag, so you could choose between 'mapbox' and 'terrarium' encoding.
https://github.com/acalcutt/rio-rgbify

I added code to rio rgbify that encodes terrarium to
https://github.com/acalcutt/rio-rgbify/blob/master/rio_rgbify/encoders.py#L48-L55
https://github.com/acalcutt/rio-rgbify/blob/master/rio_rgbify/encoders.py#L31-L35
based on https://github.com/tilezen/joerd/blob/master/docs/formats.md

EDIT: I do however think adding base and interval options to mapbox encoding would be nice, because it is really up to the user how they choose to encode the raster-dems.

EDIT 2: recent fixes to maplibre-js seem to have masked the spike I was seeing, but I assume the encoding issue is still there in the background.

[wiesehahn]

This would be great as I exactly have the problem that scaling factor of 0.1 is not enough to encode subtle terrain changes in high resolution DTMs to vizualize e.g. skid trails.

Thanks @dnomadb, I was wondering for a long time why these base and scaling factors were chosen despite their limitations, this is the first time I find an answer.

There is a good description about RGB encoding by @frodrigo at https://medium.com/@frederic.rodrigo/optimization-of-rgb-dem-tiles-for-dynamic-hill-shading-with-mapbox-gl-or-maplibre-gl-55bef8eb3d86

[acalcutt]

I was wondering if anyone has suggestions on what would be good encoding values to use with high resolution sources? Something to use with this new custom encoding if it becomes a thing?

I've been messing with the swissalti 1m geotiffs and for my latest test I made a terrarium zoom 0-17 file, which I made with, https://github.com/acalcutt/rio-rgbify/tree/compact (which combines adding terrarium support + tile deduplication)

I have been testing it with https://stackblitz.com/edit/web-platform-9f92n2?file=index.html and i notice at high zoom levels it really gets blocky, which I assume is some type of rounding error occurring either in my encoding or storing in rgb.

right now I am making a terrainrgb version so I have something to compare it too. However if we do have some recommended defaults, maybe I could do a test with those values next.

[wipfli]

Can this be closed?

[HarelM]

Only if we close this without native implementation.

[louwers]

If it is properly documented let's close it. The implementation should be tracked in issues on the respective repos.