Python crawler-downloader for greek parliaments conference records.
BeautifulSoup, Requests, sqlite3, pandas
First create a database table on sqlite, the file 'vouliDBschema' describes the schema that is used.
python scraper.py -p <number> or --pagecrawl <number>
python scraper.py -f <number> or --startfrom <number>
python scraper.py -a or --all
The data are extracted and stored to sqlite db. The db file can be used as input to the downloader script.
sqlite3, urllib2, requests
python recordsDownload.py -d or --download '/UserFiles/32323-2312382-232852-2929/es2828582.doc'
python recordsDownload.py -f or --fromID <id>
python recordsDownload.py -a or --all