Farecast is a project which analyzes the Green NYC Taxis Dataset , and takes decisions in cleaning and preprocessing depending on this analysis. then it creates a pipeline to automate data ingestion, cleaning, preprocessing, feature engineering and writing to a file and saving into a databaese using docker and a python script. Also there is an equivalent feature engineering and cleaning using pyspark, find it in milestone3/pyspark.ipynb. in milestone 4 orchestration is done using airflow to order tasks, also a new task is added which is visualizing data using dash which is a web UI tool to visualise data.