-
Notifications
You must be signed in to change notification settings - Fork 1
ArkanNibrastama/Twitter-to-S3-pipeline-use-airflow
Folders and files
Name | Name | Last commit message | Last commit date | |
---|---|---|---|---|
Repository files navigation
1. Extract tweet (json format) using tweepy library.
2. Transform the data so that just get the data is needed, such as username, quote (content), quote by (writer of the quote), created time (tweet), total of like (tweet), and total of retweet (tweet).
3. Transform the json data into dataframe.
4. Load data into AWS S3.
5. Build a Data Pipeline (DAG) using airflow from the step above.
6. Finally, run the DAG on the cloud AWS EC2.
About
No description, website, or topics provided.
Resources
Stars
Watchers
Forks
Releases
No releases published
Packages 0
No packages published