WIP feat(experimental): MultiShapeStream and TransactionalMultiShapeStream APIs

[samwillis]

WIP - Opening for visibility + discussion.

This adds two new APIs to the @electric-sql/experimental package:

MultiShapeStream

A multi-shape stream is a stream that can subscribe to multiple shapes. It ensures that all shapes will receive at least an up-to-date message from Electric within the checkForUpdatesAfter interval.

The emitted messages have an additional shape prop that is the name of the shape provided in the shapes object when constructing the MultiShapeStream.

const multiShapeStream = new MultiShapeStream({
  shapes: {
    shape1: {
      url: 'http://localhost:3000/v1/shape1',
    },
    shape2: {
      url: 'http://localhost:3000/v1/shape2',
    },
  },
})

multiShapeStream.subscribe((msgs) => {
  console.log(msgs)
})

// or with ShapeStream instances
const multiShapeStream = new MultiShapeStream({
  shapes: {
    shape1: new ShapeStream({ url: 'http://localhost:3000/v1/shape1' }),
    shape2: new ShapeStream({ url: 'http://localhost:3000/v1/shape2' }),
  },
})

TransactionalMultiShapeStream

A transactional multi-shape stream is a multi-shape stream that emits the messages in transactional batches.
It uses the lsn metadata to infer transaction boundaries, and the op_position metadata to sort the messages within a transaction.

Note that you will only receive change massages on the main subscription (in transactional batches), and not any control massages.

const transactionalMultiShapeStream = new TransactionalMultiShapeStream({
  shapes: {
    shape1: {
      url: 'http://localhost:3000/v1/shape1',
    },
    shape2: {
      url: 'http://localhost:3000/v1/shape2',
    },
  },
})

transactionalMultiShapeStream.subscribe((msgs) => {
  console.log(msgs)
})

// or with ShapeStream instances
const transactionalMultiShapeStream = new TransactionalMultiShapeStream({
  shapes: {
    shape1: new ShapeStream({ url: 'http://localhost:3000/v1/shape1' }),
    shape2: new ShapeStream({ url: 'http://localhost:3000/v1/shape2' }),
  },
})

To to:

• Explore adding additional shapes after the initial multi-shape stream is created.