YouTube Data Harvesting and Warehousing

Domain

Social Media

Problem Statement

Create a Streamlit application that allows users to access and analyze data from multiple YouTube channels. The app should enable users to input a YouTube channel ID and retrieve relevant data using the Google API. Users can store data in MongoDB as a data lake, collect data for up to 10 YouTube channels, and migrate it to a SQL database. Querying the data warehouse with SQL and displaying the data in the Streamlit app.

Technologies Used

• Python scripting
• API integration
• Data collection
• MongoDB (Atlas)
• Streamlit
• postgreSQL

Approach

1. Set up a Streamlit app for a user-friendly interface.
2. Connect to the YouTube API to retrieve channel and video data.
3. Store data in MongoDB for flexibility with unstructured data.
4. Migrate data to a SQL data warehouse for structured querying.
5. Display retrieved data with Streamlit's data visualization features.