Skip to content

A Python web crawler stores job info to database and updates the database automatically

Notifications You must be signed in to change notification settings

QingxinL/Job_Info

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

21 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Job_Info

A Python Project aimed to web crawl different companies' websites to gather the job information and store data into database.
Companies Include: Two Sigma, Jane Street Capital, Hudson River Trading, ...(In Process adding more)
Information Include: Job Title, Location, Website, and Requirements

Built With

Selenium: WebDriver used to auto the web.
MongoDB: NoSQL Database used.
BeautifulSoup & Urllib & Request: Web Scraping tools used.
Advanced Python Scheduler: Auto run tool (need main-revised.py & mongod running), keep updating the database

About

A Python web crawler stores job info to database and updates the database automatically

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages