Add basic cog support

[sharkAndshark]

Try to fix #875 . Still a draft and a lot to discuss and finish.

• Remote file on like S3. We will make it in next PRs. Let's make this PR just a basic start.
• Made the padded part transparent
• Made the gdal nodata transparent
• Add tests
• Add doc
• Support rgb/rgba u8 tif tiled image. We will do others in next PRs and leave this PR simple.

Color types and bits per smaple

colory type	bits per sample	supported	status
rgb/rgba	8	✅
rgb/rgba	16/32...	🛠️	working on, will be added in next PRs
gray	8/16/32...	🛠️	working on, will be added in next PRs

Comporession

The crate we used support these methods.

Compression Format	tiff crate(This PR based on this)	GDAL creation optinos
None	✓	✓
LZW	✓	✓ (gdal default)
JPEG		✓
Deflate	✓	✓
ZSTD		✓
WEBP		✓
LERC		✓
LERC_DEFLATE		✓
LERC_ZSTD		✓
LZMA		✓
PackBits	✓	Not sure but couldn't find this on gdal doc

Some terms and ref links

7.1.2 Tiles in COG spec
Tiff 6.0 spec page 67
Chapter 4 of COG spec

Tile and padding

In the context for a TIFF file, Tiling is a strategy for dividing the content in the TIFF file differently than using the classical Strips. Tiles, as defined in the TIFF version 6.0 specification, can be mapped to the ones defined in the OGC Two Dimensional Tile Matrix Set Standard (2D-TMS). For example in 2D-TMS, the TIFF 6.0 forces all tiles to be of the same size. This is possible with the introduction of the concept of padding: if necessary extra blank rows or columns are added to the right-most and bottom-most tile to make them the same shape as other tiles. However, the naming of the TIFF tags used version 6.0 and the property names used in the 2D-TMS differ. The following table provides a mapping between the two standards.

OGC 2D-TMS	TIFF v. 6.0	Definition
TileWidth	TileWidth	The tile width in pixels. The number of columns in each tile
TileHeight	TileLength	The tile height in pixels. The number of rows in each tile
MatrixWidth	TilesAcross	Number of tiles in the width direction
MatrixHeight	TilesDown	Number of tiles in the height direction

[sharkAndshark]

You could made a quick check that the tiles is where it should be.

configuration file

keep_alive: 75

# The socket address to bind [default: 0.0.0.0:3000]
listen_addresses: '0.0.0.0:3000'

# Number of web server workers
worker_processes: 8

# Amount of memory (in MB) to use for caching tiles [default: 512, 0 to disable]
cache_size_mb: 8

# Database configuration. This can also be a list of PG configs.

cog:
  paths:
    - tests/fixtures/cog

npm create ol-app my-app
cd my-app
npm start

Copy below to replace the main.js before npm start

import './style.css';
import { Map, View } from 'ol';
import TileLayer from 'ol/layer/Tile';
import OSM from 'ol/source/OSM';
import XYZ from 'ol/source/XYZ.js';


import TileGrid from 'ol/tilegrid/TileGrid.js';

// the tiling schema could be GoogleMapsCompatible, see https://gdal.org/en/latest/drivers/raster/cog.html#reprojection-related-creation-options
var custom_grid = new TileGrid({
  extent: [
    1620750.2508,
    4271892.7153,
    1625870.2508,
    4277012.7153],
  resolutions: [
    80,
    40,
    20,
    10],
  tileSize: [256, 256],
  origin: [1620750.2508, 4277012.7153]
});

var source = new XYZ({
  url: "http://10.1.155.35:3000/rgb_u8/{z}/{x}/{y}", tileGrid: custom_grid
});

const map = new Map({
  target: 'map',
  layers: [
    new TileLayer({
      source: new XYZ({
        url: 'https://api.maptiler.com/maps/basic/{z}/{x}/{y}.png?key=vgdy5zgzM4IIqQCC6SOn',
        attributions: '© MapTiler © OpenStreetMap contributors'
      })
    }),
    new TileLayer({
      source: source
    }),

  ],
  view: new View({
    center: [(1620750.2508 + 1625870.2508) / 2, (4271892.7153 + 4277012.7153) / 2],
    zoom: 14
  })
});

[CommanderStorm]

Overall, this looks good to me.
I have left a few comments below.

[sharkAndshark]

a few minor nits. Also, I have a silly question - is COG a separate standard from TIFF? Would be good to have some links to the documentation, and also specify why we are using cog instead of tiff configuration

@nyurik Yes they are seperated.

• tiff 6.0 doc
• geotiff doc
• cog doc

COG and GeoTIFF are all valid tiff files.

GeoTIFF extend the tiff tags (something like key-value paris) to save some spatial informations.

COG is based on the GeoTIFF standard and does not introduce additional capabilities to the TIFF v6. And some requirements/recommendation are introduced in COG(still valid tif and geotiff then) like:

In a COG file, and after the TIFF/BigTIFF header/signature and pointer to first IFD (and an eventual ghost area) sections SHOULD follow the following order:

1. IFD of the full resolution image, followed by TIFF tags values, excluding the TileOffsets and TileByteCounts arrays

2. IFD of the mask of the full resolution image, if present, followed by TIFF tags values, excluding the TileOffsets and TileByteCounts arrays

3. IFD of the first (largest in dimensions) Reduced-Resolution Subfile, if present

read more in COG doc

[sharkAndshark]

Any update? @nyurik @CommanderStorm

[sharkAndshark]

I'm a little bit worried about the non-squared pixel size(resolution) tif.

Squared
On X-Axis Or Y-Axis 1 pixel = 10 meters
Pixel Size = (10.000000000000000,-10.000000000000000)

Not Squared
On X it's 10, On Y it's 11
Pixel Size = (10.000000000000000,-11.000000000000000)

If it's not square, we should adjust the output png size based on the resolution of X or Y and we should tell user the resolution we picked in our tilejson.

I will make a proposal in #875

# with gdalinfo you could check the pixelsize
gdalinfo /path/to/some.tif

[CommanderStorm]

From my POV there are no things left to do

[nyurik]

almost there! Please fix the tiny nits i listed, and feel free to merge

[nyurik]

meh, should be ok i guess?

[nyurik]

seems like all is good, feel free to merge once it passes - although something tells me there is an issue with the current main branch due to some tests failing there

[nyurik]

And huge thanks for such a hard work on this!