You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This feature request tracks the remaining effort to fully implement Row Tracking.
The feature required changes in Spark that are being released with Spark 3.5. With Spark 3.5 being released, fully implementing the feature is now unblocked.
Reading and preserving row IDs / row commit versions on the other does require changes that are shipping with Spark 3.5 (see e.g. apache/spark#40677 and apache/spark#40545). With Spark 3.5 we can now implement reading and preserving row tracking information
Willingness to contribute
The Delta Lake Community encourages new feature contributions. Would you or another member of your organization be willing to contribute an implementation of this feature?
Yes. I can contribute this feature independently.
Yes. I would be willing to contribute this feature with guidance from the Delta Lake community.
No. I cannot contribute this feature at this time.
The text was updated successfully, but these errors were encountered:
All the changes that allow reading and preserving row tracking information are in 3.2 thanks to @longvu-db!
Some of the more relevant PRs part of that effort:
Read support:
Feature request
Which Delta project/connector is this regarding?
Overview
This feature request tracks the remaining effort to fully implement Row Tracking.
The feature required changes in Spark that are being released with Spark 3.5. With Spark 3.5 being released, fully implementing the feature is now unblocked.
Further details
Row tracking specification was added with 8dfa534 and #1747. See Row ID design document for the initial design.
Support for writing fresh row IDs / row commit versions was introduced in the following PRs and didn't require any changes in Spark:
Reading and preserving row IDs / row commit versions on the other does require changes that are shipping with Spark 3.5 (see e.g. apache/spark#40677 and apache/spark#40545). With Spark 3.5 we can now implement reading and preserving row tracking information
Willingness to contribute
The Delta Lake Community encourages new feature contributions. Would you or another member of your organization be willing to contribute an implementation of this feature?
The text was updated successfully, but these errors were encountered: