RNN-LSTM: From Applications to Modeling Techniques and Beyond - Systematic Review

---

Long Short-Term Memory (LSTM) networks are pivotal in processing sequential data with long-term dependencies. Despite their popularity, the challenge of effectively initializing and optimizing LSTM models persists, often hindering model performance and accuracy. This study systematically reviews the breadth of literature to answer how weight initialization and optimization techniques can be used to bolster LSTM performance. Utilizing the PRISMA methodology, 95 peer-reviewed articles spanning 2018-2023 are analyzed. Our analysis encompasses numerous modeling techniques, offering a comprehensive overview across various application domains. This paper stands out by comprehensively analyzing modeling techniques, datasets, evaluation metrics, programming languages, and the diverse applications and domains of LSTM networks. Our findings provide a roadmap for researchers and practitioners to enhance LSTM networks and achieve superior results.

Research Highlights

• In-depth four-step approach to methodically include studies concerning the initialization and optimization of LSTM weights.
• Extensive discussion on the current domains and applications into which LSTM is used.
• An extensive analysis of various weight initialization strategies in RNN-LSTM networks.
• Comprehensively surveys the different optimization algorithms used in optimizing RNN-LSTM networks.
• Identified the current dominated techniques and tools in LSTM models development, including the preferred and most used programming languages and the most used evaluation metrics.

Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA)

The literature review study mapping process

Weights optimization techniques in LSTM

Weights initialization techniques in LSTM

---

Data Availability

The Microsoft Excel file utilized for this SLR is publicly available at this link.

The Python code used to generate the figures of this SLR is publicly available at this link.

A full step-by-step guide on how to conduct a systematic review is available at this link.

---

Papers that explicitly followed / adopted our SLR methodology:

• Abdullahi, S., & et al. (2026). The rise of hallucination in large language models: systematic reviews, performance analysis and challenges. Cluster Computing, 29(2), 124.
• Al-haimi, B., & et al. (2025). Digital transformation in the real estate industry: A systematic literature review of current technologies, benefits, and challenges. International Journal of Information Management Data Insights, 5(1), 100340.
• Atta, M. R., & et al. (2025). Artificial Intelligence and Machine Learning in Thermodynamic Gas Hydrate Studies: A Review. Energy & Fuels, 39(38), 18287-18310.
• Alghurabi, A., & et al. (2025). A systematic review of material erosion prediction techniques: Incorporating model parameters variability and the lack of field-scale representation. Results in Engineering, 107194.
• Fayyaz, A. M., & et al. (2025). Grad-CAM (Gradient-weighted Class Activation Mapping): A systematic literature review. Computers in Biology and Medicine, 198, 111200.
• Hassan, S. U., & et al. (2025). Local interpretable model-agnostic explanation approach for medical imaging analysis: A systematic literature review. Computers in Biology and Medicine, 185, 109569.
• Yalli, J. S., & et al. (2025). Authentication schemes for Internet of Things (IoT) networks: A systematic review and security assessment. Internet of Things, 30, 101469.
• Abdullahi, M., & et al. (2025). A Systematic Literature Review of Concept Drift Mitigation in Time-Series Applications. IEEE Access.
• Sumiea, E. H., & et al. (2024). Deep deterministic policy gradient algorithm: A systematic review. Heliyon, 10(9).
• Read more

So please consider citing our work, if you use our template or methodology:

S. M. Al-Selwi, M. F. Hassan, S. J. Abdulkadir, A. Muneer, E. H. Sumiea, A. Alqushaibi, and M. G. Ragab, "RNN-LSTM: From applications to modeling techniques and beyond—Systematic review," Journal of King Saud University - Computer and Information Sciences, vol. 36, no. 5, 2024, Art. no. 102068. doi: https://doi.org/10.1016/j.jksuci.2024.102068.

BibTeX

@article{Alselwi20241slr,
title = {RNN-LSTM: From applications to modeling techniques and beyond—Systematic review},
author = {S. M. Al-Selwi and M. F. Hassan and S. J. Abdulkadir and A. Muneer and E. H. Sumiea and A. Alqushaibi and M. G. Ragab},
journal = {Journal of King Saud University - Computer and Information Sciences},
volume = {36},
number = {5},
pages = {102068},
year = {2024},
issn = {1319-1578},
doi = {https://doi.org/10.1016/j.jksuci.2024.102068},
url = {https://www.sciencedirect.com/science/article/pii/S1319157824001575}
}

---

Corresponding author Safwan Mahmood Al-Selwi @ 2024

Thank you