Cover Service Improvements

[mekarpeles]

Tasks

See 2023 yearly planning notes:

• Fix amz cover imports
  • Fix Amazon book-cover imports (data mapping?) #7244
  • fix amz cover import key names #7316
  • Re-enable affiliate-server amz imports with tests #7340
• Roadmap for fixing cover archival #7257
• Add monthly cron to run Cover Server Archival #8278
  • Related to Cover Service Improvements #6822 and Extend BWB Monthly Cron to archive + import covers #7691
• Extend BWB Monthly Cron to archive + import covers #7691 Extend the BWB monthly importer cron to archive & import covers: DRAFT: Batch add Better World Books covers #6706
  • This depends on / requires space on the host via Roadmap for fixing cover archival #7257
• Covers to webp #7211: Task to investigate performance opportunities

Bonus opportunities

• Create monthly dumps which map a cover ID to isbn/lccn/olid -> access url; e.g. archive.org zip offset
• Add author cover urls to solr via Librarian-Verified Authors #7659

@cclauss's Notes

Describe the problem that you'd like solved

• What documentation do we have about book cover images?
• What book cover image cron jobs do we currently run?
• Are we properly backing up the book cover images that we get from affiliates?
• Can we restore our book cover images from those backups?
• Do we provide our clients a covers API?
• If we have an API for accessing book cover images, how often is it used?
• If we have an API for accessing book cover images, how could it be improved?

ol-covers on Sentry

Proposal & Constraints

Pull request #6706 -- Add book cover images from Better World Book imports
Steps:

1. Create a dictionary of all book cover images that have been imported from Better World Books so far this year.
2. Create a dictionary of all Open Library ISBNs in the previous dictionary that do not have a book cover image.
3. Enhance the Open Library record of all ISBNs that appear in the second dictionary to add the image to the OL record.

Additional context

Stakeholders

---

Initial timeline:

2022-08-30 Present the draft roadmap on a community call to solicit feedback.
2022-09-15 Send out a survey of needs and ideas to Librarians, Developers, Operations
2022-09-30 Release final roadmap and present on a community call

---

Open Library: Roadmap for Book Cover Images

Summary

Workstreams: Covers acquisition, storage, retrieval, and documentation

How do we get cover images?

Open Library obtains book cover images via the covers service which is a cron job that runs in a dedicated Docker container.

How do we store cover images?

Book cover images via the covers service are stored in ol-home0's /1/var/tmp/imports directory in YYYY-MM.zip files. Each file is named isbn_13.jpg (e.g.: 9780425030134.jpg)

How do we provide access to cover images?

The work on #6706 has made it clear that a low-level API that rapidly retrieves cover images based on isbn_13s would:

1. simplify the creation of advanced search on above, and
2. simplify integration into other systems.

[cclauss]

ISBN, Move off /1, easy bulk access, Handle redactions (take down request).