Selenium_Airflow

This repository use selenium with Airflow to schedule scraping task

Windows

• Mongodb setup: https://www.geeksforgeeks.org/how-to-install-mongodb-on-windows/
• Download docker desktop: https://www.docker.com/products/docker-desktop/
• cd /path/airflow_tutorial
• docker build . --tag extending_airflow_tutorial:latest
• docker-compose up -d
• Go to http://localhost:8080/ and run test_sl