Binary tiles

[kylebarron]

Explore using binary tiles, possibly with Apache Arrow, to offload computation off the CPU.

"Use binary data" is a section of the deck.gl performance optimization guide.

Also see:
visgl/deck.gl#3467

[kylebarron]

Let's try to use protobuf. Essentially you want a binary file that has:

{
  positions: [x1, y1, x2, y2, x3, y3, ...],
  timestamps: [t1, t2, t3, t4, ...],
  startIndices: [0, 2, 5, 8, ...]
}

The positions includes all coordinates for all lines in the tile; similarly the timestamps includes all timestamps for all lines in the tile. The startIndices is what allows deck.gl to keep track of which coordinates are a single line.

See: https://deck.gl/#/documentation/deckgl-api-reference/layers/path-layer?section=use-binary-attributes

which specifically talks about how to use binary attributes with the PathLayer, and presumably it's the same with the TripsLayer.

[kylebarron]

Something like this seems like it would work but I got a few errors, from malformed data?

getTileData: ({ x, y, z }) =>
  fetch(`${baseurl}/${z}/${x}/${y}.pbf`)
    .then(response => {
      if (response.status === 200) {
        return response.arrayBuffer();
      }
      return null;
    })
    .then(buffer => {
      if (buffer) {
        const pbf = new Protobuf(buffer);
        const obj = ScheduleTile.read(pbf);
        console.log(obj);

        const data = {
          length: obj.startIndices.slice(-1)[0],
          startIndices: new Uint16Array(obj.startIndices), // this is required to render the paths correctly!
          attributes: {
            getPath: {
              value: new Float32Array(obj.positions),
              size: 2
            },
            getTimestamps: {
              value: new Float32Array(obj.timestamps),
              size: 1
            }
          }
        };
        console.log(data);
        return data;
      }
      return {
        length: 0,
        startIndices: new Uint16Array(0),
        positions: new Float32Array(0),
        timestamps: new Float32Array(0)
      };
    }),