Install NodeJS
$git clone thisrepoURL
From project root folder:
$npm install
From the project root folder:
$npm start
Navigate to localhost:39761
We also have other branches for some of the other scrapers we used on other sites. You can run these in a similar way, just change the kbb in the run command to the name of the new script. (Note, cargurus.js in cargurus branch got lost, so there is no way to replicate the code. It used very similar methods as the other scrapers.)
From the project root folder:
KBB.com: $npm run kbb
This combines all the data files listed in data/ into a single file and performs some final update operations. Each file should indicate which columns are which in the first line and special operations can be requested on certain columns.
EX: Labeling a column 'mileage_km' will convert it to miles and then set mileage based on what the converted value is.
From the project root folder:
Normalize: $npm run normalize
What we used to upload our data since the GUI got stuck about ~500000 rows in
These were used to take the edmunds, cars.com, and cargurus.com data and remove uneccessary columns and split strings to make new columns.
$ python 'pathToFile + filename'