fix: properly deserialize percent-encoded file paths of Remove actions, to make sure tombstone and file paths match #2035
Conversation
…ns, to make sure tombstone and file paths match
|
@sigorbor can you add a small test? |
Sure, done |
|
@ion-elgreco @rtyler - please review and approve |
|
@ion-elgreco Integration tests are failing on a clearly unrelated issue. |
|
@ion-elgreco @rtyler guys, can we proceed to merging the PR? It is approved and all the checks passed |
|
@ion-elgreco this PR is approved, can it please be merged? BTW, are there any plans to release a new delta-rs Rust version soon? Any ETA? Would be great if it can happen soon, and include the current PR - we have customers waiting for this fix... |
|
@sigorbor I don't know when the next rust release will be planned, we are waiting for the arrow log functionality to be added and some other things. If customers need it now, you can compile it from source and share that in the mean time to unblock them : ). |
|
thanks @ion-elgreco "Integration tests" fail on apparently unrelated issue.. Could you please merge this PR? |
|
@sigorbor merged it |
|
sorry for chiming in so late, I held off on this PR since the entire serde logic will be more or less gone once #2037 lands. Is there a specific scenario that needs to be considered in tests? |
…s, to make sure tombstone and file paths match (delta-io#2035) Percent-encoded file paths of Remove actions were not properly deserialized, and when compared to active file paths, the paths didn't match, which caused tombstones to be recognized as active files (be kept in the state) <!--- For example: - closes delta-io#106 ---> <!--- Share links to useful documentation ---> --------- Co-authored-by: Igor Borodin <igborodi@microsoft.com> Co-authored-by: Ion Koutsouris <15728914+ion-elgreco@users.noreply.github.com> Co-authored-by: R. Tyler Croy <rtyler@brokenco.de>
…s, to make sure tombstone and file paths match (delta-io#2035) # Description Percent-encoded file paths of Remove actions were not properly deserialized, and when compared to active file paths, the paths didn't match, which caused tombstones to be recognized as active files (be kept in the state) # Related Issue(s) <!--- For example: - closes delta-io#106 ---> # Documentation <!--- Share links to useful documentation ---> --------- Co-authored-by: Igor Borodin <igborodi@microsoft.com> Co-authored-by: Ion Koutsouris <15728914+ion-elgreco@users.noreply.github.com> Co-authored-by: R. Tyler Croy <rtyler@brokenco.de>
Description
Percent-encoded file paths of Remove actions were not properly deserialized, and when compared to active file paths, the paths didn't match, which caused tombstones to be recognized as active files (be kept in the state)
Related Issue(s)
Documentation