Skip to content

Chrome Extension Backend - Removes paywalled articles from Delfi.ee and Postimees.ee news portal VERSION 2

Notifications You must be signed in to change notification settings

Kypsis/PaywallDetectorCrawler

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

41 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Delfi/Postimees paywall article detector backend

Backend for Chrome Extension that removes/hides/changes opacity of paywalled links on the main page. Scrapes main page using Puppeteer as headless browser and Axios requests. Initial footprint populating empty DynamoDB table is quite large (ca 500 links), afterwards scrapes roughly 0-10 new links every 5-7 minutes. Written in Node and utilizing AWS EC2 and DynamoDB.

Endpoints

http://ec2-18-185-111-192.eu-central-1.compute.amazonaws.com:3000/delfi

http://ec2-18-185-111-192.eu-central-1.compute.amazonaws.com:3000/postimees

About

Chrome Extension Backend - Removes paywalled articles from Delfi.ee and Postimees.ee news portal VERSION 2

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published