Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

RecordBatchTransformer: Handle schema migration and column re-ordering in table scans #602

Merged
merged 10 commits into from
Oct 11, 2024
Merged

RecordBatchTransformer: Handle schema migration and column re-ordering in table scans #602

merged 10 commits into from
Oct 11, 2024

Commits on Oct 3, 2024

  1. feat: Add skeleton of RecordBatchEvolutionProcessor

    @sdd
    sdd committed Oct 3, 2024
    Configuration menu
    Copy the full SHA
    9fe476e View commit details
    Browse the repository at this point in the history

  2. feat: Add initial implementation of RecordBatchEvolutionProcessor

    @sdd
    sdd committed Oct 3, 2024
    Configuration menu
    Copy the full SHA
    afc86ea View commit details
    Browse the repository at this point in the history

  3. feat: support more column types. Improve error handling. Add more com… 

    …ments
    @sdd
    sdd committed Oct 3, 2024
    Configuration menu
    Copy the full SHA
    7172cce View commit details
    Browse the repository at this point in the history

  4. feat(wip): adress issues with reordered / skipped fields

    @sdd
    sdd committed Oct 3, 2024
    Configuration menu
    Copy the full SHA
    3d5d8c3 View commit details
    Browse the repository at this point in the history

  5. feat: RecordBatchEvolutionProcessor handles skipped fields in projection

    @sdd
    sdd committed Oct 3, 2024
    Configuration menu
    Copy the full SHA
    3421fe1 View commit details
    Browse the repository at this point in the history

  6. chore: add missing license header

    @sdd
    sdd committed Oct 3, 2024
    Configuration menu
    Copy the full SHA
    657f58b View commit details
    Browse the repository at this point in the history

  7. chore: remove unneeded comment

    @sdd
    sdd committed Oct 3, 2024
    Configuration menu
    Copy the full SHA
    81480b9 View commit details
    Browse the repository at this point in the history

  8. refactor: rename to RecordBatchTransformer. Improve passthrough handling

    @sdd
    sdd committed Oct 3, 2024
    Configuration menu
    Copy the full SHA
    0b17465 View commit details
    Browse the repository at this point in the history

  9. feat: more performant handling of case where only schema transform is… 

    … required but columns can remain unmodified
    @sdd
    sdd committed Oct 3, 2024
    Configuration menu
    Copy the full SHA
    0ed5721 View commit details
    Browse the repository at this point in the history

  10. refactor: import arrow_cast rather than arrow

    @sdd
    sdd committed Oct 3, 2024
    Configuration menu
    Copy the full SHA
    61d4bdc View commit details
    Browse the repository at this point in the history