An Awesome Url Crawler: (Developed Using Python)

How to Use ? Usage: main.py -u url -d depth

Options:
-h, --help show this help message and exit
-u URL Specify the begin url.
-d DEPTH Specify the crawling depth. Default: 5.
-t THREAD The amount of threads. Default: 10.

Licence