PRE-ALPHA this is in no way complete!
Stock Scraper is a simple application to pull stock price information from google spreadsheet and store it in a database. It can pull data for a whole index using FT index constituent data stored in the index sheet.
$ python main.py
This will pull all the indexes in the index sheet and save each index in the index table. For each index it will fetch all the constituent shares and update the share data in the share table.
$ python main.py --index FTSE:FSI 2017-05-01 2017-05-31
# or leave the dates out and it'll default to yesterday
$ python main.py --index FTSE:FSI
This will update that Index as described above and fetch the price data for the specified period for each share in that index.
$ python main.py --share AAL:LSE 2017-05-01 2017-05-31
# or leave the dates out and it'll default to yesterday
$ python main.py --share AAL:LSE
This will update the given share price data for the given period for the given period
The easiest way to get started is to with this right now is to checkout the code run it using docker:
$ git clone git@github.com:martinffx/stock-scraper.git
$ cd stock-scraper
$ docker-compose run scraper python main.py
- Get index data from sheet
- Save index data into database
- Fetch index constituents
- Save share data in database
- Fetch price data from sheet
- Save price data
- Parallelize scraping
- Setup CI to build and publish a docker image