Iceberg Time Travel Support

[sungwy]

The IcebergScanOperator currently doesn't support time traveling on snapshot IDs.

This issue tracks the work of introducing a snapshot ID argument to the Daft Iceberg APIs to enable scanning data at a specific snapshot ID.

[jaychia]

Seems simple enough - we just need to forward the argument to the PyIceberg table during the scan?

self._table.scan(limit=limit, snapshot_id=...)

How does this interact with .schema? Is it possible that Iceberg tables might have different schemas at different snapshots?

[sungwy]

Yes that's right. The schema id associated with a specific snapshot is stored within the snapshot metadata. When we time travel, we will be projecting the schema associated with that snapshot ID (instead of the current schema ID).