Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Low level create table #342

Merged
merged 13 commits into from
Aug 4, 2021
Merged

Conversation

Smurphy000
Copy link
Contributor

Description

This is the current implementation for a low level create table api

Looking for feedback

  • I think the new() method for DeltaTableMetaData needs to take in format (just defaulted to parquet to get things working)
  • I think the create() method should return an updated version of the DeltaTable instead of nothing at the moment

need to fix test cleanup
need to add logic to make create() safer
want to return an updated DeltaTable (new state) after a create() is successful
rust/src/delta.rs Outdated Show resolved Hide resolved
rust/src/delta.rs Outdated Show resolved Hide resolved
rust/src/action.rs Outdated Show resolved Hide resolved
rust/src/delta.rs Outdated Show resolved Hide resolved
rust/src/delta.rs Outdated Show resolved Hide resolved
rust/src/delta.rs Outdated Show resolved Hide resolved
…in the test.

Still need to do better error handling in the create() method and figure out how to create a _last_checkpoint and update the DeltaTableState
… how to handle possible errors in the create() method
rust/src/delta.rs Outdated Show resolved Hide resolved
…sing cloned actions, can now update the DeltaTable state after the commit portion of create() method
added more asserts for test
@houqp
Copy link
Member

houqp commented Aug 2, 2021

@Smurphy000 could you help manually resolve the comments you already addressed with follow up commits? That will make it a little bit easier for us to do follow up reviews :)

updated new DeltaTableMetadata to take in a format option
change create method resulting Error type to be a transaction error
@Smurphy000 Smurphy000 marked this pull request as ready for review August 3, 2021 15:10
@houqp houqp requested review from fvaleye and houqp August 3, 2021 16:23
Copy link
Member

@houqp houqp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me overall, great work @Smurphy000 !

rust/src/delta.rs Show resolved Hide resolved
rust/src/delta.rs Outdated Show resolved Hide resolved
@houqp houqp requested review from xianwill and mosyp August 3, 2021 16:27
…for the data actually written to the delta table from the create method
Copy link
Member

@houqp houqp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@houqp houqp merged commit 129940f into delta-io:main Aug 4, 2021
@houqp
Copy link
Member

houqp commented Aug 4, 2021

Thank you @Smurphy000, I hope you had a good time hacking on rust and delta ;)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants