Training_Spark_ML

Training ML classifiers using Spark

This is part of the work: Machine Learning Pipelines with Modern Big Data Tools for High Energy Physics Comput Softw Big Sci 4, 8 (2020).
This directory contains notebooks and code for model training using Spark to parallelize ML, and in particular the High Level particle Classifier.

The techniques tested include:

• Spark ML Random forest
• Spark parallelizing XGBoost
• Spark parallelizing lightGBM