Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow checkpoint creation when partion column is "timestampNtz " #2381

Closed
thomasfrederikhoeck opened this issue Apr 4, 2024 · 3 comments · Fixed by #2383
Closed

Allow checkpoint creation when partion column is "timestampNtz " #2381

thomasfrederikhoeck opened this issue Apr 4, 2024 · 3 comments · Fixed by #2383
Labels
enhancement New feature or request

Comments

@thomasfrederikhoeck
Copy link
Contributor

Description

Right now you can create a table partioned on a column of type timestampNtz. This is all on main.:

import pandas as pd
dates = pd.date_range(datetime(2021,1,1,3,4,6,3),datetime(2021,1,3,3,4,6))
df = pd.DataFrame({"time":dates, "a":[i for i in range(len(dates))]})
write_deltalake("mytable",df, partition_by="time")
dt = DeltaTable("mytable")
print(dt.schema())
> Schema([Field(time, PrimitiveType("timestampNtz"), nullable=True), Field(a, PrimitiveType("long"), nullable=True)])

However when trying to create a checkpoint I'm hit by


dt.create_checkpoint()
> PanicException: not implemented: Primitive type timestampNtz is not supported for partition column values.

Use Case

Related Issue(s)

@thomasfrederikhoeck thomasfrederikhoeck added the enhancement New feature or request label Apr 4, 2024
@ion-elgreco
Copy link
Collaborator

@thomasfrederikhoeck We just need to add an arm for PrimitiveType::TimestampNtz`

PrimitiveType::Timestamp => {

I'll see if I can find some time tomorrow

@thomasfrederikhoeck
Copy link
Contributor Author

GIven #2380 it might also be worth looking at the Timestamp arm?

@thomasfrederikhoeck
Copy link
Contributor Author

thomasfrederikhoeck commented Apr 4, 2024

@ion-elgreco I can grap this one :-) I seems like you are already started :-D

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants