This is a URL crawler.

Usage: main.py -u url -d depth

Options:
-h, --help show this help message and exit
-u URL Specify the begin url.
-d DEPTH Specify the crawling depth. Default: 5.
-t THREAD The amount of threads. Default: 10.