Skip to content

ArkanNibrastama/Twitter-to-S3-pipeline-use-airflow

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

4 Commits
 
 
 
 
 
 

Repository files navigation

Documentation

My Image

Project Explanation:


1. Extract tweet (json format) using tweepy library.

2. Transform the data so that just get the data is needed, such as username, quote (content), quote by (writer of the quote), created time (tweet), total of like (tweet), and total of retweet (tweet).

3. Transform the json data into dataframe.

4. Load data into AWS S3.

5. Build a Data Pipeline (DAG) using airflow from the step above.

6. Finally, run the DAG on the cloud AWS EC2.

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages