Web-crawling-with-Python

In this project, I will undertake four different sub-projects, ranging from simple data scraping to more complex tasks.

1. Building a bot to collect user data on LinkedIn for your own job-related purposes

• Data Source: https://www.linkedin.com/
• Project Location: Folder: Scraping_UserLinkedIn
• How to use my code
  • Input your personal LinkedIn username and password into the text file (User_pass.txt).
  • Execute each shell in the scraping_LinkedIn.ipynb file.
• Requirements:
  • Python 3.8 or higher
  • Selenium
  • WebDriver
  • BeautifulSoup
  • Time
• Achieved Results: A result table in a .csv file with the specified amount of data that the bot collects.

2. Simple Data Scraping of Job Information from a Website

• Data Source: https://www.timesjobs.com/candidate/contact.html

• Project Location: file Scraping_Jobs_Website.ipynb

• Requirements:

  • Python 3.8 or higher
  • Requests
  • BeautifulSoup

• Achieved Results:

  • Detailed job information such as job title, location, and job description.

  

  3. Scraping Revenue Data of Companies from Wikipedia

  • Data source: https://en.wikipedia.org/wiki/List_of_largest_companies_in_the_United_States_by_revenue
  • Project location: File craw_data_web.ipynb
  • Requirements:
    • Python 3.8 or higher
    • Requests
    • BeautifulSoup

• Achieved Results: A ranking table of companies along with their respective revenues(table below is an example of one of the tables scraped from the web)

4. Scraping Top movie Data

• Data source: https://www.imdb.com/chart/top/

• Project location: File Crawl_TopMovie_DataWeb.ipynb

• Requirements:

  • Python 3.8 or higher
  • Requests
  • BeautifulSoup

• Achieved Results: Top 250 most popular movies

• Contact me to more detailed information or how to use code

  • email: longpm211@gmail.com
  • LinkedIn: https://www.linkedin.com/in/minhlongba/