Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Snapshot Interop] Add Shallow Copy Snapshots list in Snapshot Repository Data #8610

Open
harishbhakuni opened this issue Jul 11, 2023 · 4 comments · May be fixed by #13827
Open

[Snapshot Interop] Add Shallow Copy Snapshots list in Snapshot Repository Data #8610

harishbhakuni opened this issue Jul 11, 2023 · 4 comments · May be fixed by #13827
Labels
bug Something isn't working Storage:Durability Issues and PRs related to the durability framework

Comments

@harishbhakuni
Copy link
Contributor

harishbhakuni commented Jul 11, 2023

Describe the bug
During Snapshot Deletion, Today we are relying on repository setting to determine if we should use the new delete snapshots method which cleans up lock files as well or the old one. IF user enables shallow copy snapshot flag and creates multiple shallow copy snapshots and then disables flag and triggers deletion of those shallow copy snapshots, we will end up leaving stale shallow copy blobs belonging to those snapshots.

To Fix this, we need to add the list of Shallow Copy Snapshots in Repository Data. So that we don't rely just on the repository flag.

@harishbhakuni harishbhakuni added bug Something isn't working untriaged labels Jul 11, 2023
@gbbafna
Copy link
Collaborator

gbbafna commented Jul 11, 2023

I would also think about making shallow copy as a static parameter in repo . That would make the code paths quite simple.

Existing user can delete the existing repo and reregister as a read-only repo with shallow snapshot as a static parameter

@gbbafna gbbafna added Storage:Durability Issues and PRs related to the durability framework and removed untriaged labels Jul 11, 2023
@gbbafna gbbafna changed the title [BUG] [Snapshot Interop] Add Shallow Copy Snapshots list in Snapshot Repository Data [Snapshot Interop] Add Shallow Copy Snapshots list in Snapshot Repository Data Jul 11, 2023
@harishbhakuni
Copy link
Contributor Author

We need to fix this before supporting migration from remote backed indices to doc-rep indices (#12718). as we cannot anymore rely on not changing remote_store_index_shallow_copy repository setting.

@harishbhakuni
Copy link
Contributor Author

@gbbafna ^^

@gbbafna
Copy link
Collaborator

gbbafna commented Apr 8, 2024

Post this, we can make shallow snapshot a dynamic property of repository as well .

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working Storage:Durability Issues and PRs related to the durability framework
Projects
Status: Next (Next Quarter)
2 participants