Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

vacuum isn't actually deleting files #663

Closed
MrPowers opened this issue Jun 29, 2022 · 2 comments · Fixed by #664
Closed

vacuum isn't actually deleting files #663

MrPowers opened this issue Jun 29, 2022 · 2 comments · Fixed by #664
Labels
bug Something isn't working

Comments

@MrPowers
Copy link
Collaborator

Environment

Delta-rs version: 0.5.8

Binding: Python

Environment: localhost


Bug

What happened: Ran dt.vacuum(retention_hours=0, enforce_retention_duration=False, dry_run=False) and files didn't actually get deleted.

What you expected to happen: I thought the stale files would actually get deleted.

How to reproduce it: Here's the notebook to reproduce this.

More details: enforce_retention_duration was added recently and that's successfully letting me bypass the retention duration error I was facing before. Let me know if I'm missing something here and there's a way to get the files vacuumed with the existing code.

@MrPowers MrPowers added the bug Something isn't working label Jun 29, 2022
@wjones127
Copy link
Collaborator

wjones127 commented Jun 29, 2022

Thanks for reporting this!

From what I can tell, the bug is that if you provide a relative path to DeltaTable() constructor, then vacuum doesn't delete any files. But it works if you pass an absolute path. It seems like we've probably only tested with the latter.

I'll look at fixing this soon.

@MrPowers
Copy link
Collaborator Author

@wjones127 - looks like you added a fix already, awesome ;)

I verified in the notebook that vacuum does work as anticipated with absolute paths and this is a relative path bug, like you expected. Thanks for the rapid response time!!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants