Skip to content

deps: upgrade Scrapy + Chrome to stable 84

Compare
Choose a tag to compare
@s-pace s-pace released this 17 Aug 13:50
· 40 commits to master since this release

This PR upgrades Scrapy to its latest version (2.2.1). It also removes unnecessary use of CustomContextFactory. It also upgrades the chrome version to its latest stable available v84.

Upgrading Scrapy introduces many benefits such as:

File extensions that LinkExtractor ignores by default now also include 7z, 7zip, apk, bz2, cdr, dmg, ico, iso, tar, tar.gz, webm, and xz
Upgrading Twisted to its lates version. This is required to mitigate with CVE-2020-10109
Better logging system
A new DNS_RESOLVER setting allows enabling IPv6 support