Muon momentum estimation using Deep learning algorithms in the CMS Trigger System

The Compact Muon Solenoid (CMS) is a general-purpose detector at the Large Hadron Collider (LHC). During a run, it generates about 40 TB data per second. Since It is not feasible to readout and store such a vast amount of data, so a trigger system selects and stores only interesting events or events likely to reveal new physics phenomena. The goal of this project is to benchmark the muon momentum estimation performance of Fully Connected Neural Networks (FCNN), Convolutional Neural Networks (CNN), and Graph Neural Networks (GNN), on the prompt and displaced muon samples detected by CSC stations at CMS to aid trigger system's transverse momentum (pT) muon estimation.

About

In the project FCNNs, CNNs, and GNNs are trained and evaluated on the prompt muon samples (two versions of same samples with different sampling approaches), and displaced muon samples generated by Monte Carlo simulation. The other details are -

• Target Variables: Three types of predictions are benchmarked with each type of algorithm.

Target	Loss
1/Transverse_momentum (1/pT)	Mean Square Error (MSE)
Transverse Momentum (pT)
4 class classification (0-10 GeV, 10-30 GeV, 30-100 GeV, >100 GeV)	Focal Loss

• Validation Scheme: 10 fold out-of-fold predictions (i.e. dataset is splitted into 10 small batches, out of them 8 are used for training, 1 as validation dataset and 1 as holdout. This holdout is changed 10 times to give the final scores.)

• Metrices Tracked:
  

  • MAE - Mean Absolute Error at a given transverse momentum (pT).
    
  • MAE/pT - Ratio of Mean Absolute Error to transverse momentum at a given transverse momentum.
    
  • Acurracy - At a given pT, muon samples can be divided into two classes, one muons with pT more than this given and another class of muons with pT less than this. So, Acurracy at a given pT is the accuracy for these two classes.
    
  • F1-score (of class pT>x GeV) - At a given pT, this is the f1-score of the class of muons with pT more than this given pT.
    
  • F1-score (of class pT<x GeV) - At a given pT, this is the f1-score of the class of muons with pT less than this given pT.
    
  • ROC-AUC Score of each class - only in case of four class classification

• Preprocessing: Standard scaling of input coordinates

How to use

An example of how to use is available here
Get the csv data from here

1. Make sure that all the libraries mentioned in requirements.txt are installed
2. Clone the repo

git clone https://github.com/Ayush-Parhi/CMS_Muon_Momentum_Estimation.git

3. Change current directory to the cloned directory and execute main.py with the required arguments

python main.py --path='' path to csv file \
                --dataset='prompt_new'\
                --predict='pT'\
                --model='FCNN'\
                --epochs=50 \
                --batch_size=512\
                --folds="0,1,2,3,4,5,6,7,8,9" \
                --results='' path to results directory

Note: Give absolute paths as argument

Arguments

1. path - path of the csv having the coordinates of generated muon samples
2. dataset - specify the samples that you are using (i.e. prompt_new, prompt_old, or displaced)
3. predict - target variable (i.e. pT, 1/pT, or pT_classes)
4. model - architecture to use (i.e. FCNN, CNN, or GNN)
5. epochs - max number of epochs to train, if score converges than due to early-stopping training may stop earlier
6. batchsize - number of samples in a batch
7. folds - a string containing the info on which folds one wants the result
8. results - path of the directory to save the results

Results

Regressing 1/pT

Metric	Prompt Muons Samples-1	Prompt Muons Samples-2	Displaced Muons Samples
MAE/pT
MAE
Accuracy
F1-score (pT>x)
F1-score (pT<x)

Regressing pT

Metric	Prompt Muons Samples-1	Prompt Muons Samples-2	Displaced Muons Samples
MAE/pT
MAE
Accuracy
F1-score (pT>x)
F1-score (pT<x)

Four class classification

• Prompt Muons Samples-1

Model	0-10 GeV	10-30 GeV	30-100 GeV	>100GeV
FCNN	0.990	0.970	0.977	0.969
CNN	0.991	0.973	0.980	0.983

• Prompt Muons Samples-2

Model	0-10 GeV	10-30 GeV	30-100 GeV	>100GeV
FCNN	0.990	0.975	0.981	0.958
CNN	0.991	0.976	0.983	0.983

• Displaced Muons Samples

Model	0-10 GeV	10-30 GeV	30-100 GeV	>100GeV
FCNN	0.944	0.898	0.910	0.839
CNN	0.958	0.907	0.932	0.910