Web scraper for archived c9.io projects. Built with WebDriverJS (Selenium for Node.js). Bootstrapped from webdriverjs-recipes.
brew install node
Chrome and ChromeDriver:
brew cask install google-chrome
brew install chromedriver
Clone repository:
git clone https://github.com/remarkablemark/c9.io-scraper.git
cd c9.io-scraper
If you're using nvm, you can set the node version:
nvm use
Install dependencies:
npm install
Fill out .env
before running the scraper:
USERNAME=user # required
PASSWORD=pass # required
DOWNLOADS_DIR=~/Downloads/ # optional
If DOWNLOADS_DIR
isn't set, it defaults to ~/Downloads/
.
In the project directory, you can run:
The script logs into your c9.io account, prepares all workspaces for download, and then downloads them.
Downloaded workspaces are recorded in tmp/downloaded_workspaces.txt
.