Export data from mysql-database to REDSHIFT using kafka
Problem Statement:
I need to build an ETL pipeline to dump mysql data base record to redshift using kafka
RedShift Dataware house
Approach
Launch entire server setup
docker-compose up
Dump data in mysql db
docker exec -i mysql sh -c 'exec mysql -uroot -p"$MYSQL_ROOT_PASSWORD"' < "./database-dump/mysqlsampledatabase.sql"
I will design Star Schema so that i can export above attached OLTP to OLAP
-
MYSQL KAFKA S3 Project Description
-
S3 Redshift Project Description