Downloads

https://doi.org/10.62836/jissh.v2i1.437
Mao, H., & Han, Q. (2025). Applications of Transformer-Based Language Models for Depression Detection: A Scoping Review. Journal of Integrated Social Sciences and Humanities, 1–8. https://doi.org/10.62836/jissh.v2i1.437
Volume 2, Issue 1, 2025

Applications of Transformer-Based Language Models for Depression Detection: A Scoping Review

Depression is a substantial public health issue, with global ramifications. Transformer-Based Language Models (TBLM), with their ability to capture nuanced emotional and semantic information, are particularly well-suited for identifying depressive states from such unstructured textual data. The objective of this scoping review is to examine the usefulness of TBLM in detecting depression on text-based data. This study is based on a comprehensive search of Web of Science Core Collection, which includes studies that focused on the application of TBLM in diagnosing and classifying depression through social media texts. Our findings indicate that models such as BERT and its variants predominate in the existing literature, due to BERT’s demonstrated generalizability in natural language understanding and its capacity to capture semantic and affective features in social media texts. However, generative models like GPT are rarely exploring. Twitter was the most frequently used data source, attributed to its public accessibility, real-time content generation, and large user base. Domain-adapted models, including MentalBERT and RedditBERT, demonstrate promising capabilities through fine-tuning on mental health-related corpora, potentially enhancing the detection of context-specific linguistic cues. This study further suggests that TBLM are increasingly being incorporated into psychiatric practice, with evidence of rapid advancements and encouraging outcomes.

Transformer-Based Language Models depression detection BERT GPT mental health

References

  1. Penninx BWJH. Depression and Cardiovascular Disease: Epidemiological Evidence on Their Linking Mechanisms. Neuroscience & Biobehavioral Reviews 2017; 74: 277–286.
  2. Thornicroft G, Chatterji S, Evans-Lacko S, et al. Undertreatment of people with major depressive disorder in 21 countries. The British Journal of Psychiatry 2017; 210(2): 119–124.
  3. Vindegaard N, Benros ME. COVID-19 Pandemic and Mental Health Consequences: Systematic Review of the Current Evidence. Brain, Behavior, and Immunity 2020; 89: 531–542.
  4. First MB, Westen D. Classification for Clinical Practice: How to Make ICD and DSM Better Able to Serve Clinicians. International Review of Psychiatry 2007; 19(5): 473–481.
  5. Wong E. Media Review: Deep Medicine: How Artificial Intelligence Can Make Healthcare Human again. InnovAiT: Education and Inspiration for General Practice 2022; 15(10): 611. https://doi.org/10.1177/17557380211018237.
  6. Calvo RA, Milne DN, Hussain MS, et al. Natural Language Processing in Mental Health Applications Using Non-Clinical Texts. Natural Language Engineering 2017; 23(5): 649–685. https://doi.org/10.1017/S1351324916000383.
  7. De Choudhury M, Gamon M, Counts S, et al. Predicting Depression via Social Media. In Proceedings of the Seventh International AAAI Conference on Weblogs and Social Media, Cambridge, MA, USA, 8–11 July 2013; pp. 128–137. https://doi.org/10.1609/icwsm.v7i1.14432.
  8. Tackman AM, Sbarra DA, Carey AL, et al. Depression, Negative Emotionality, and Self-Referential Language: A Multi-lab, Multi-Measure, and Multi-Language-Task Research Synthesis. Journal of Personality and Social Psychology 2019; 116(5): 817–834.
  9. Mohmand R, Habib U, Usman M, et al. A Deep Learning Approach for Automated Depression Assessment Using Roman Urdu. IEEE Access 2024; 12: 193387–193401. https://doi.org/10.1109/ACCESS.2024.3519264.
  10. Beniwal R, Saraswat P. A Hybrid BERT-CPSO Model for Multi-Class Depression Detection Using Pure Hindi and Hinglish Multimodal Data on Social Media. Computers and Electrical Engineering 2024; 120: 109786. https://doi.org/10.1016/j.compeleceng.2024.109786.
  11. Zeberga K, Attique M, Shah B, et al. A Novel Text Mining Approach for Mental Health Prediction Using Bi-LSTM and BERT Model. Computational Intelligence and Neuroscience 2022; 2022: 7893775.
  12. Bokolo BG, Liu Q. Advanced Comparative Analysis of Machine Learning and Transformer Models for Depression and Suicide Detection in Social Media Texts. Electronics 2024; 13(20): 3980. https://doi.org/10.3390/electronics13203980.
  13. Narvaez Burbano R, Caicedo Rendon OM, Astudillo CA. An Encoder-Only Transformer Model for Depression Detection from Social Network Data: The DEENT Approach. Applied Sciences 2025; 15(6): 3358. https://doi.org/10.3390/app15063358.
  14. Tiwari SS, Pandey R, Deepak A, et al. An Ensemble Approach to Detect Depression from Social Media Platform: E-CLS. Multimedia Tools and Applications 2024; 83(28): 71001–71033. https://doi.org/10.1007/s11042-023-17971-6.
  15. Ilias L, Mouzakitis S, Askounis D. Calibration of Transformer-Based Models for Identifying Stress and Depression in Social Media. IEEE Transactions on Computational Social Systems 2024; 11(2): 1979–1990. https://doi.org/10.1109/TCSS.2023.3283009.
  16. Jain M, Jain S, Jain A, Garg B. CDME-GAT: Context-Aware Depression Detection Using Multiembedding and Graph Attention Networks in Social Media Text. IEEE Transactions on Computational Social Systems 2024; 11: 7212–7222. https://doi.org/10.1109/TCSS.2024.3402689.
  17. Bokolo BG, Liu Q. Deep Learning-Based Depression Detection from Social Media: Comparative Evaluation of ML and Transformer Techniques. Electronics 2023; 12(21): 4396. https://doi.org/10.3390/electronics12214396.
  18. Wang X, Chen S, Li T, et al. Depression Risk Prediction for Chinese Microblogs via Deep-Learning Methods: Content Analysis. Content analysis. JMIR Medical Informatics 2020; 8(7): e17958.
  19. Pourkeyvan A, Safa R, Sorourkhah A. Harnessing the Power of Hugging Face Transformers for Predicting Mental Health Disorders in Social Networks. IEEE Access 2024; 12: 28025–28035. https://doi.org/10.1109/access.2024.3366653.
  20. Khan S, Alqahtani S. Hybrid Machine Learning Models to Detect Signs of Depression. Multimedia Tools and Applications 2024; 83(13): 38819–38837. https://doi.org/10.1007/s11042-023-16221-z.
  21. Xin C, Zakaria LQ. Integrating Bert with CNN and BiLSTM for Explainable Detection of Depression in Social Media Contents. IEEE Access 2024; 12: 161203–161212. https://doi.org/10.1109/access.2024.3488081.
  22. Zhou S, Mohd M. Mental Health Safety and Depression Detection in Social Media Text Data: A Classification Approach Based on a Deep Learning Model. IEEE Access 2025; 13: 63284–63297. https://doi.org/10.1109/access.2025.3559170.
  23. Abbas MA, Munir K, Raza A, et al. Novel Transformer Based Contextualized Embedding and Probabilistic Features for Depression Detection from Social Media. IEEE Access 2024; 12: 54087–54100. https://doi.org/10.1109/ACCESS.2024.3387695.
  24. Jamali AA, Berger C, Spiteri RJ. Momentary Depressive Feeling Detection Using X (Formerly Twitter) Data: Contextual Language Approach. JMIR AI 2023; 2: e49531.
  25. Owen D, Antypas D, Hassoulas A, et al. Enabling Early Health Care Intervention by Detecting Depression in Users of Web-Based Forums using Language Models: Longitudinal Analysis and Evaluation. JMIR AI 2023; 2: e41205.

Supporting Agencies

  1. Funding : This research was funded by Project of Zhejiang Federation of Humanities and Social Sciences (NO. 2024B014) and Zhejiang Provincial Medical and Health Science and Technology Plan Project (NO. 2023RC192).