Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Empty snapshot ID should be Null instead of -1 #352

Open
Tracked by #348
Fokko opened this issue Apr 25, 2024 · 4 comments · May be fixed by #374
Open
Tracked by #348

Empty snapshot ID should be Null instead of -1 #352

Fokko opened this issue Apr 25, 2024 · 4 comments · May be fixed by #374
Assignees
@s-akhtar-baig

Comments

@Fokko
Copy link
Contributor

Fokko commented Apr 25, 2024

This is an old bug from Java. Where the Snapshot was set to -1 instead of None:

iceberg-rust/crates/iceberg/src/spec/table_metadata.rs

Line 44 in aba6209

pub(crate) static EMPTY_SNAPSHOT_ID: i64 = -1;

From Java 1.5 and later this is fixed. For older version of Java, the current-snapshot-id is required. We solved this by setting a flag: apache/iceberg-python#473
@Fokko Fokko mentioned this issue Apr 25, 2024
Closed
73 tasks
@s-akhtar-baig
Copy link
Contributor

@Fokko, can you please assign this to me? Thanks!

@Fokko
Copy link
Contributor Author

Fokko commented Apr 25, 2024

@s-akhtar-baig With pleasure 🙌

@gupteaj
Copy link
Contributor

gupteaj commented May 6, 2024

@Fokko , @liurenjie1024
what changes are we looking for ?
Since rust is using option to hold snapshot id internally, but writing as -1 for V2 manifest files.
Does it need similar changes ( some variable ) as iceberg-python or empty snapshot id as -1 looks valid for rust.

@s-akhtar-baig s-akhtar-baig linked a pull request May 15, 2024 that will close this issue
Open
@liurenjie1024
Copy link
Collaborator

I took a look the code and found that maybe we don't need to change anything since we already handled this case:

iceberg-rust/crates/iceberg/src/spec/table_metadata.rs

Line 678 in 6f85456

current_snapshot_id: if let &Some(id) = &value.current_snapshot_id {

But it would be better to add some tests for it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@s-akhtar-baig s-akhtar-baig

Labels
None yet
Projects
None yet
Milestone
No milestone
4 participants
@Fokko @liurenjie1024 @s-akhtar-baig @gupteaj