Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Low level create table #342

Merged
merged 13 commits into from
Aug 4, 2021
Merged

Low level create table #342

merged 13 commits into from
Aug 4, 2021

Commits on Jul 30, 2021

  1. added constructors for some necessary structs. created function frame… 

    … for create table
    @Smurphy000
    Smurphy000 committed Jul 30, 2021
    Configuration menu
    Copy the full SHA
    99b0fde View commit details
    Browse the repository at this point in the history

Commits on Jul 31, 2021

  1. added create() to deltaTable 

    need to fix test cleanup
need to add logic to make create() safer
want to return an updated DeltaTable (new state) after a create() is successful
    @Smurphy000
    Smurphy000 committed Jul 31, 2021
    Configuration menu
    Copy the full SHA
    593100b View commit details
    Browse the repository at this point in the history

  2. changed commit, to try_commit_transaction for version 0 of a new delt… 

    …a table
    @Smurphy000
    Smurphy000 committed Jul 31, 2021
    Configuration menu
    Copy the full SHA
    3f5ac4d View commit details
    Browse the repository at this point in the history

  3. timestamp to timestamp_millis to match delta protocol

    @Smurphy000
    Smurphy000 committed Jul 31, 2021
    Configuration menu
    Copy the full SHA
    dd56aff View commit details
    Browse the repository at this point in the history

  4. updated test to use TempDir, added comments for where I need asserts … 

    …in the test.

Still need to do better error handling in the create() method and figure out how to create a _last_checkpoint and update the DeltaTableState
    @Smurphy000
    Smurphy000 committed Jul 31, 2021
    Configuration menu
    Copy the full SHA
    98b5792 View commit details
    Browse the repository at this point in the history

Commits on Aug 1, 2021

  1. added version assert to test, did some clean up, need to work more on… 

    … how to handle possible errors in the create() method
    @Smurphy000
    Smurphy000 committed Aug 1, 2021
    Configuration menu
    Copy the full SHA
    5991b91 View commit details
    Browse the repository at this point in the history

  2. added derive Clone to Action enum, Txn struct, and Protocol struct; u… 

    …sing cloned actions, can now update the DeltaTable state after the commit portion of create() method
    @Smurphy000
    Smurphy000 committed Aug 1, 2021
    Configuration menu
    Copy the full SHA
    02e4ffe View commit details
    Browse the repository at this point in the history

Commits on Aug 2, 2021

  1. added get_provider to Format 

    added more asserts for test
    @Smurphy000
    Smurphy000 committed Aug 2, 2021
    Configuration menu
    Copy the full SHA
    aab839d View commit details
    Browse the repository at this point in the history

Commits on Aug 3, 2021

  1. implemeneted the Default trait for Format to be parquet 

    updated new DeltaTableMetadata to take in a format option
change create method resulting Error type to be a transaction error
    @Smurphy000
    Smurphy000 committed Aug 3, 2021
    Configuration menu
    Copy the full SHA
    6587826 View commit details
    Browse the repository at this point in the history

  2. Update rust/src/delta.rs 

    as suggested by houqp

Co-authored-by: QP Hou <dave2008713@gmail.com>
    @Smurphy000 @houqp
    Smurphy000 and houqp authored Aug 3, 2021
    Configuration menu
    Copy the full SHA
    35c4c11 View commit details
    Browse the repository at this point in the history

  3. fix to some lint and formatting issues

    @Smurphy000
    Smurphy000 committed Aug 3, 2021
    Configuration menu
    Copy the full SHA
    ca9a8ba View commit details
    Browse the repository at this point in the history

  4. added a few more asserts on the metadata after state change

    @Smurphy000
    Smurphy000 committed Aug 3, 2021
    Configuration menu
    Copy the full SHA
    e196a22 View commit details
    Browse the repository at this point in the history

Commits on Aug 4, 2021

  1. added PartialEq trait for some actions and schema, and added asserts … 

    …for the data actually written to the delta table from the create method
    @Smurphy000
    Smurphy000 committed Aug 4, 2021
    Configuration menu
    Copy the full SHA
    654b42e View commit details
    Browse the repository at this point in the history