Installation

Requirements installation

First, install Scrapy and redis-py.

$ pip install scrapy>=2.0 redis>=4.0

Second, setup redis server and enable port 6379 for redis server.

$ curl -fsSL https://packages.redis.io/gpg | sudo gpg --dearmor -o /usr/share/keyrings/redis-archive-keyring.gpg
$ echo "deb [signed-by=/usr/share/keyrings/redis-archive-keyring.gpg] https://packages.redis.io/deb $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/redis.list
$ sudo apt update && sudo apt install -y redis

Or setup redis server by the official tutorial.

Stable release

To install Scrapy-Redis, run this command in your terminal:

$ pip install scrapy-redis

If you don’t have pip installed, this Python installation guide can guide you through the process.

From sources

The sources for Scrapy-Redis can be downloaded from the Github repo.

You can either clone the public repository:

$ git clone git://github.com/rolando/scrapy-redis

Or download the tarball:

$ curl  -OL https://github.com/rolando/scrapy-redis/tarball/master

Once you have a copy of the source, you can install it with:

$ pip install -e .