Add scripts/cron_watcher.py

[cclauss]

A script that checks to see that our cron jobs have competed their respective tasks.

Currently it checks to see if the ol-dump and ol-cdumps for the previous month are stored at

• https://archive.org/details/ol_exports?sort=-publicdate

Long term, the script should cover more cron jobs:
Daily Cron-audit task (Python) sentry (who watches the watchers)
If dump and cdump for last YYYY-MM were not uploaded to archive.org
Or if sitemaps updated for this YYYY-MM were not on transferred to ol-www
Or if partner dumps for this YYYY-MM were not uploaded to archive.org
Or if there have been no imports in last 48 hours (i.e. 2 days)
Or if DD>17 for YYYY-MM and bwb batchname doesn’t exist in import psql table
Then send daily email with failures only or slack failures

Sentry cron-jobs:

https://sentry.archive.org/organizations/ia-ux/projects/ol-cron-jobs

Technical

Testing

1. Run the script. Should print True and return 0
2. Pass in garbage for yyyy_mm. Should print False and return 1.
3. Change the URL. Should print False and return 1.

Screenshot

Stakeholders

[jimman2003]

I believe that we should use the ia library to fetch the newest dump, eliminating the need for html scraping

[jimman2003]

I suggest we rewrite find_last_months_dumps_on_ia as

from internetarchive import search_items
from datetime import date, timedelta
last_day_of_last_month = date.today().replace(day=1) - timedelta(days=1)
yyyy_mm = f"{last_day_of_last_month:%Y-%m}"
def find_last_months_dumps_on_ia(yyyy_mm: str = yyyy_mm) -> bool:
    """
    Return True if both ol_dump_yyyy and ol_cdump_yyyy files have been saved on the
    Internet Archive.
    """
    prefixes = (f"ol_dump_{yyyy_mm}", f"ol_cdump_{yyyy_mm}")
    found=0
    for item in search_items("collection:ol_exports"):
        if item["identifier"].startswith(prefixes):
            found+=1
        if found >= 2:
            break
    return found>=2

[cclauss]

Unfortunately, the current internetarchive is based on requests so it only supports old style synchronous calls. Given that we eventually want to add at least 4 more network tests, I was planning to use asyncio.

[jimman2003]

ok, maybe we should talk to api directly then (with httpx)?

[mekarpeles]

I think these calls should be negligible in cost and using the IA tool seems right

[cclauss]

&& || ; in cron?