Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Decode path in Add and Remove actions. #434

Merged
merged 2 commits into from
Sep 20, 2021
Merged

Decode path in Add and Remove actions. #434

merged 2 commits into from
Sep 20, 2021

Conversation

zijie0
Copy link
Contributor

@zijie0 zijie0 commented Sep 17, 2021

Description

Fix file path decode according to delta spec.

Might need another PR to do the path encoding while writing delta log and checkpoints.

Related Issue(s)

houqp
houqp previously approved these changes Sep 18, 2021
View reviewed changes
Copy link
Member

@houqp houqp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch @zijie0! Could you resolve the conflict with the latest main branch?

rust/src/action.rs Outdated Show resolved Hide resolved
@houqp
Copy link
Member

houqp commented Sep 18, 2021

cc @xianwill @mosyp we need to encode the file path in version commit and checkpoint otherwise it would break the scalar reader.

houqp
houqp approved these changes Sep 19, 2021
View reviewed changes
Copy link
Member

@houqp houqp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

fvaleye
fvaleye approved these changes Sep 19, 2021
View reviewed changes
Copy link
Collaborator

@fvaleye fvaleye left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thanks @zijie0

@houqp houqp merged commit 7588bd4 into delta-io:main Sep 20, 2021
@houqp
Copy link
Member

houqp commented Sep 20, 2021

Thanks @zijie0 !

FYI @dispanser @mgill25, this could add overhead to table loading, but I feel like it's probably unavoidable.

@mosyp mosyp mentioned this pull request Oct 12, 2021
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@fvaleye fvaleye fvaleye approved these changes

@houqp houqp houqp approved these changes

@xianwill xianwill Awaiting requested review from xianwill

@mosyp mosyp Awaiting requested review from mosyp

@rtyler rtyler Awaiting requested review from rtyler

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Open delta table failed with special partition paths.
3 participants
@zijie0 @houqp @fvaleye