Hi I'm ismhac !

This project uses Scrapy and MongoDb to collect some data for learning purposes.

I collected data from website

You can also try it, let's get started !

---

create virtual environment:

python -m venv venv  

activate virtual environment:

source ./venv/Scripts/activate 

install packages

pip install -r ./src/pip_mgmt/requirements.txt

create scrapy project:

cd ./src && scrapy startproject <project_name>

run project

cd ./src/<project_name>

rm -rf ../export/data.json && scrapy crawl <spider_name> -o ../export/data.json