Skip to content

Releases: nkoda/Work-Sample_Data-Engineer

Data Pipeline for Predicting Trading Volume

06 May 18:36
Compare
Choose a tag to compare

Description

This release contains a data pipeline that predicts the trading volume of equities using a LightGBM model. The pipeline consists of several components, including data ingestion, data transformation, feature engineering, model training, and a Flask web app for serving predictions.

Key features of the project release include:

  • Efficient data ingestion using pandas and thread pools to reduce ingestion time
  • Data transformation and feature engineering for preparing the data for modeling
  • A LightGBM model trained on the transformed data for predicting trading volume
  • A Flask web app for serving predictions to users
  • Docker containerization for easy deployment and reproducibility