socialBridge: Tracking and Optimising your Leetcode, Linkdin, Instagram, Twitter and Github usage (in progress not yet complete)

status: In progress
Through this project, I hope to demonstrate my growing skills and knowledge in several areas:

• Data Engineering
• Machine Learning
• Data Analytics
• Backend and frontend Development
• Cloud Technologies

Features:

• Closely tracking usage of multiple social media platforms to provide accurate insights via daily text message updates.
• Leetcode stats like total questions solved, new concept questions done or not on a particular day, time spent etc sent via daily text message updates.
• Dashboard for indepth time series data analysis.
• RAG based chat agent to query a users db and understand patterns and habits.

Tech-Stack:

1. Airflow + Python:- For establishing and managing data pipelines across various social media platforms.

2. FastAPI:- for backend and API building.

3. PostgreSQL:- going with a sql database. Separate tables for separate social media sites.

4. Langchain + PGvector vector database:- For Retrival Augmented Generation(RAG) chat agent utility, pgvector for retrival as already postgresql being used.

5. Llama-chat-7b and llama.cpp:- A lighweight Large Language Model for Rag feature.

6. Clerk:- registration and for managing users.

7. React.js and TailwindCSS:- Frontend, web interface for users

8. Tableau :- for Dashboard and in-depth visulaisation customised for every user.

9. AWS (ec2, S3, RDS) + Docker + Github Actions :- for cloud services and ci/cd pipeline.

System Diagram: