A fully automated ETL pipeline that fetches and stores real-time traffic and weather data in BigQuery, with a live Looker dashboard for visualization.
-
Updated
Oct 1, 2024 - Python
A fully automated ETL pipeline that fetches and stores real-time traffic and weather data in BigQuery, with a live Looker dashboard for visualization.
Build a data pipeline on Google Cloud using an event-driven architecture, leveraging GCS, Cloud Run functions, and BigQuery. Explore both VM and Composer options for Airflow management, and utilize Logging & Monitoring for pipeline health. Discover how SQL-based BigQuery ML can be used for initial ML implementation in specific scenarios.
Add a description, image, and links to the cloud-run-functions topic page so that developers can more easily learn about it.
To associate your repository with the cloud-run-functions topic, visit your repo's landing page and select "manage topics."