RNN-LSTM: From Applications to Modeling Techniques and Beyond - Systematic Review

---

Long Short-Term Memory (LSTM) networks are pivotal in processing sequential data with long-term dependencies. Despite their popularity, the challenge of effectively initializing and optimizing LSTM models persists, often hindering model performance and accuracy. This study systematically reviews the breadth of literature to answer how weight initialization and optimization techniques can be used to bolster LSTM performance. Utilizing the PRISMA methodology, 95 peer-reviewed articles spanning 2018-2023 are analyzed. Our analysis encompasses numerous modeling techniques, offering a comprehensive overview across various application domains. This paper stands out by comprehensively analyzing modeling techniques, datasets, evaluation metrics, programming languages, and the diverse applications and domains of LSTM networks. Our findings provide a roadmap for researchers and practitioners to enhance LSTM networks and achieve superior results.

Research Highlights

• In-depth four-step approach to methodically include studies concerning the initialization and optimization of LSTM weights.
• Extensive discussion on the current domains and applications into which LSTM is used.
• An extensive analysis of various weight initialization strategies in RNN-LSTM networks.
• Comprehensively surveys the different optimization algorithms used in optimizing RNN-LSTM networks.
• Identified the current dominated techniques and tools in LSTM models development, including the preferred and most used programming languages and the most used evaluation metrics.

Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA)

The literature review study mapping process

Weights optimization techniques in LSTM

Weights initialization techniques in LSTM

Please consider citing our work, if you use our template or methodology:

S. M. Al-Selwi, M. F. Hassan, S. J. Abdulkadir, A. Muneer, E. H. Sumiea, A. Alqushaibi, and M. G. Ragab, "RNN-LSTM: From applications to modeling techniques and beyond—Systematic review," Journal of King Saud University - Computer and Information Sciences, vol. 36, no. 5, 2024, Art. no. 102068. doi: https://doi.org/10.1016/j.jksuci.2024.102068.

BibTeX

@article{Alselwi20241slr,
title = {RNN-LSTM: From applications to modeling techniques and beyond—Systematic review},
author = {S. M. Al-Selwi and M. F. Hassan and S. J. Abdulkadir and A. Muneer and E. H. Sumiea and A. Alqushaibi and M. G. Ragab},
journal = {Journal of King Saud University - Computer and Information Sciences},
volume = {36},
number = {5},
pages = {102068},
year = {2024},
issn = {1319-1578},
doi = {https://doi.org/10.1016/j.jksuci.2024.102068},
url = {https://www.sciencedirect.com/science/article/pii/S1319157824001575}
}

Data Availability

The Microsoft Excel file utilized for this SLR is publicly available at this link.

The Python code used to generate the figures of this SLR is publicly available at this link.

---

Corresponding author Safwan Mahmood Al-Selwi @ 2024

Thank you