Skip to content

Latest commit

 

History

History
32 lines (26 loc) · 1.13 KB

README.md

File metadata and controls

32 lines (26 loc) · 1.13 KB
Raw

JanLinders-Webscraper

Web scraper that collects information about products at Jan linders.

Gets its data from jan linders.

Scraper in action

How to use

All information is uploaded to a MySQL database. You can either run a database server locally using a program like xampp, or get a company to host one for you.

Change your credentials and database name in the db.py file to your own login details.

The table that is automatically created in your database will look like this:

CREATE TABLE `products` (
  `name` varchar(100) NOT NULL,
  `price` double NOT NULL,
  `brand` varchar(45) NOT NULL,
  `weight` varchar(45) NOT NULL,
  `group` varchar(45) NOT NULL,
  PRIMARY KEY (`name`,`price`,`brand`,`weight`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;

Ideas

  • Improve UI.
  • Show elapsed time when done.
  • Scrape offers page and store them in a seperate table.
  • Option to choose which groups to scrape.
  • Build test to see if the site structure is still the same.