-
Notifications
You must be signed in to change notification settings - Fork 0
Overview
This programme essentially takes in a user request from the client, and uses the input parameters such as the source URL to begin scraping pages on Wikipedia on the server-side. Once all the web scraping has been done, the programme then processes the data it has collected, filtering out unnecessary or repeated information. These data is then channeled back to the client in the form of JSON objects, which would be used to generate an interactive graph on the browser.
The main bulk (if not all) of the back-end is written in Python. The main web scraping functionality of this programme uses libraries such as Requests and Beautiful Soup for making HTTP requests and parsing HTML documents, whereas Flask is used to as a framework to handle almost everything that concerns server back-end tasks.
The front-end, on the other hand, is written in the usual HTML-Javascript-CSS mix. In particular, the creation of graphs is handled by the D3.js library. Jinja is also used sparingly to generate templates for the web application.