This system is a complex web application for users to read pensonalized news content around the world.
The major components are:
- Front-End:
- renders a React single-page application
- Node Server:
- handles the client-side Authentication request (SignUp, Login)
- hands over "get_news(userID, pageID)" to backend server
- hands over "post_click_log(userID, newsID)" to backend server
- Backend Server (RPC):
- handles the "get_news" request from the web server
- sends click_log to message queue
- Recommendation (RPC):
- updates the user's topic-preference list based on their click behavior
- recommends news that belong to the user's interested topics
- Real-time News Data Pipeline:
- monitors and scrapes news from various media sources and save them into a database
- removes literal and semantic duplicate news
- News Classifier:
- classifies news into topic categories given their text
- React - A JavaScript library for building user interfaces
- ExpressJS - a minimal and flexible Node.js web application framework that provides a robust set of features for web and mobile applications
- MongoDB - a cross-platform document-oriented database program
- Redis - an in-memory data structure project implementing a distributed, in-memory key-value database with optional durability
- CloudAMQP - RabbitMQ (an open-source message-broker software that originally implemented the Advanced Message Queuing Protocol) as a service
- Keras - an open-source neural-network library written in Python
- Tensorflow - a free and open-source software library for dataflow and differentiable programming across a range of tasks
- News API - A JSON API for live news and blog articles
- JSONRPClib - a library implements the JSON-RPC 2.0 proposed specification in pure Python
- UI-Design
- Node Server
- Authentication
- MongoDB, RabbitMQ
- Data Pipeline
- Pagination, Click log collector
- Recommendation
- Classifier
