A Swarm Intelligence Approach: Combination of Different EEG-Channel Optimization Techniques to Enhance Emotion Recognition

This repository contains the Code for the published Paper:

Balic, S., Kleybolte, L., & Märtin, C. (2022). A Swarm Intelligence Approach: Combination of Different EEG-Channel Optimization Techniques to Enhance Emotion Recognition. In Human-Computer Interaction. Technological Innovation (pp. 303–317). Springer International Publishing. https://doi.org/10.1007/978-3-031-05409-9_23

Please cite our paper if you use This Repository for your Work

Notes

---

Implement an modified version of the LSTM RNN Model wich is used by Acharya D. et al. [1] to get an accuracy of about 92.17% (Arousal) and about 94.46% (Valence)

Preprocessing

Here, the DEAP dataset [2] is used, where each of the 32 participant's data consists of 8064 readings for 32 EEG channels and for each of the 40 video trials.

Shape: (Subjects, Trials, Steps, Channels)
Shape: (32, 40, 8064, 32)

Using the FFT to extract the seperate Bandwaves Delta-δ (1–4 Hz), Theta-θ (4–8 Hz), Alpha-α (8–14 Hz), Beta-β (14–31 Hz), and Gamma-γ (31–50 Hz). The FFT use a window of 256 which averages the band power of 2sec each video, wherby the Window slides every 0.125 sec.

Shape: (Subjects, Trials, FFT steps, Channels, Bandwaves)
Shape: (32, 40, 488, 32, 5)
Transform to 
Shape: (624640, 160, 1)

LSTM architecture

_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
bidirectional (Bidirectional (None, 160, 256)          133120    
_________________________________________________________________
dropout (Dropout)            (None, 160, 256)          0         
_________________________________________________________________
lstm_1 (LSTM)                (None, 160, 256)          525312    
_________________________________________________________________
dropout_1 (Dropout)          (None, 160, 256)          0         
_________________________________________________________________
lstm_2 (LSTM)                (None, 160, 64)           82176     
_________________________________________________________________
dropout_2 (Dropout)          (None, 160, 64)           0         
_________________________________________________________________
lstm_3 (LSTM)                (None, 160, 64)           33024     
_________________________________________________________________
dropout_3 (Dropout)          (None, 160, 64)           0         
_________________________________________________________________
lstm_4 (LSTM)                (None, 32)                12416     
_________________________________________________________________
dropout_4 (Dropout)          (None, 32)                0         
_________________________________________________________________
dense (Dense)                (None, 16)                528       
_________________________________________________________________
activation (Activation)      (None, 16)                0         
_________________________________________________________________
dense_1 (Dense)              (None, 2)                 34        
_________________________________________________________________
activation_1 (Activation)    (None, 2)                 0         
=================================================================
Total params: 786,610
Trainable params: 786,610
Non-trainable params: 0
_________________________________________________________________

Classification

The DEAP dataset contains 4 different labels: dominance, liking, arousal, and valence. Here we used Arousal and Valence to obtain emotional trends in the Russell's circumplex model. To predict trends only, we need to threshold the labels in the middle to obtain binary values, since each label in the DEAP dataset was scored between 1 and 10.

Conclusion

After training the moddel with an 75/25 split we get an accuracy of about 92.17% and 0.2001 as Loss-value (Arousal) (Left) and about 94.46% and 0.1553 as Loss-value (Valence) (Right)

PCA

In order to achieve a better result, the channel optimization algorithm Principal component analysis (PCA) was applied, however, this did not optimize the accuracy in classifying the arousal, also the training process was significantly less effective than in the previous attempt. Arousal was classified with an accuracy of 83.39% and a loss of 0.3658

mRMR

With the channel-selected algorithm Minimum redundancy maximum relevance (mrmr), the same accuracy was achieved as without channel optimisation, however, 20 channels were still used here, probably only when more channels are removed different results are obtained. Arousal was classified with an accuracy of 92.74% and a loss of 0.1892, and Valence with an Accuracy of 92.36% and a loss of 0.1983

PSO

After adding the partical swarm optimization (PSO) as channel selection moethod, the results differ relativ strongly, so for arousal we get an Accuracy of about 87.08% and a loss of 0.3120 on the first training procedure but on a second run, the Model didn't train correctly, so accuracy remained at 58.91% throughout the training process. You can see the ruslts below. On the left side the first training procedure and on the right side the second one.

On the other side the training with the PSO and the Valence labels work very well, so there we get an accuracy of about 93.15% and a loss of 0.1874.

GWO

The Grey Wolf optimizer also belongs to the swarm based feature selection algorithms and was also used here to reduce the channels, this algorithm provided a performant and accurate classification. Arousal accuracy: 88.74% and loss: 0.2765, Valence accuracy 93.83% and Loss: 0.1669

CS

The Cuckoo Search algorithm is also part of the swarm based feature selection algorithms and in our case the algorithm boost the classification: Arousal Accuracy: 93.33% Loss: 0.1846 and Valence Accuracy: 93.67% Loss: 0.1738

References

[1] Acharya, D., Jain, R., Panigrahi, S.S., Sahni, R., Jain, S., Deshmukh, S.P., Bhardwaj, A.: Multi-class Emotion Classification Using EEG Signals. In: Garg, D., Wong, K., Sarangapani, J., Gupta, S.K. (eds.) Advanced Computing. 10th International Conference, IACC 2020, Panaji, Goa, India, December 5–6, 2020, Revised Selected Papers, Part I. Springer eBook Collection, vol. 1367, pp. 474–491. Springer Singapore; Imprint Springer, Singapore (2021). doi: 10.1007/978-981-16-0401-0_38

[2] Koelstra, S., Muhl, C., Soleymani, M., Lee, J.-S., Yazdani, A., Ebrahimi, T., Pun, T., Nijholt, A., Patras, I.: DEAP: A Database for Emotion Analysis ;Using Physi-ological Signals. IEEE Transactions on Affective Computing, vol. 3, 18–31 ( 2012). doi: 10.1109/T-AFFC.2011.15