Scrape breach details logos using Puppeteer.
Since this invokes Puppeteer (a 100MB-ish Chromium penalty), it's probably best to clone this repo locally and run the script versus trying to use npx
and redownload the puppeteer/Chromium binary repeatedly.
git clone https://github.com/pdehaan/blurts-logo-scraper.git
cd blurts-logo-scraper
npm install
npm run scrape
If you want to run against a non-production environment, you can specify a SERVER=
ENV var, or --server
argument on the CLI to a development/stage server instead:
SERVER=https://fx-breach-alerts.herokuapp.com/ npm scrape
# OR
npm run scrape -- --server=https://fx-breach-alerts.herokuapp.com/
# OR
npm run scrape:dev # which sets --server=https://fx-breach-alerts.herokuapp.com/ for you.
By default, all breaches will be scraped and generated screenshots will be saved to (and minified from) the "./shots/" directory. If you want to customize this behavior, there are two other attributes you can set:
ENV var | Argument | Description | Default |
---|---|---|---|
OUTDIR |
--outdir |
Directory to write screenshots to. | "./shots" |
MAX_BREACHES |
--max-breaches |
Maximum number of breaches to scrape. | undefined (all breaches) |