Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

remove special case for 8M range covers #9752

Merged
merged 2 commits into from
Aug 19, 2024
Merged

remove special case for 8M range covers #9752

merged 2 commits into from
Aug 19, 2024

Conversation

mekarpeles
Copy link
Member

@mekarpeles mekarpeles commented Aug 14, 2024

Closes #9560 (at least for critical items in the 8M range).

This does not update the database -- the filename field is the only field which requires an update, but they are kept as is in case there's any need to use the existing tar offsets. The zip filenames are automatically inferred by the cover id.

The following script can be used to generate db statements if we want to update the filename field:

x = 10_000
for i in range(100):
     p = str(i).zfill(2)
     print(f"update cover set filename='covers_0008/covers_0008_{p}.zip' where id >={str(8_000_000 + (i * x))} and id < { str(8_000_000 + ((i+1) * x))};")

For the purpose of doing this in a non-breaking way, I've already added a field to the coverstore.cover table called filename_old which equals the current value of filename so if anything were to go wrong, we could overwrite filename with filename_old. After this work is completed, we likely want to delete the filename_old column as it takes up necessary space for the ~1M covers where these values have been backed up.

Technical

Testing

Screenshot

Stakeholders

@mekarpeles mekarpeles added the Priority: 1 Do this week, receiving emails, time sensitive, . [managed] label Aug 14, 2024
Copy link
Collaborator

@cdrini cdrini left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

openlibrary/coverstore/code.py Show resolved Hide resolved
@cdrini cdrini added the Needs: Submitter Input Waiting on input from the creator of the issue/pr [managed] label Aug 19, 2024
@cdrini cdrini added the Patch Deployed This PR has been deployed to production independently, outside of the regular deploy cycle. label Aug 19, 2024
@github-actions github-actions bot removed the Needs: Submitter Input Waiting on input from the creator of the issue/pr [managed] label Aug 19, 2024
@cdrini cdrini merged commit 6e0eb79 into master Aug 19, 2024
4 checks passed
@cdrini cdrini deleted the update-coverstore- branch August 19, 2024 13:38
SivanC pushed a commit to SivanC/openlibrary that referenced this pull request Aug 20, 2024
Co-authored-by: Drini Cami <cdrini@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Patch Deployed This PR has been deployed to production independently, outside of the regular deploy cycle. Priority: 1 Do this week, receiving emails, time sensitive, . [managed]
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Followup to #7478: Move Cover Tars -> Zips
2 participants