SpringerLink
Log in

Utilizing deep convolutional neural architecture with attention mechanism for objective diagnosis of schizophrenia using wearable IoMT devices

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Mental health diagnosis often relies on subjective evaluations, which can be intrusive and lack objectivity. With the current global situation brought about by the COVID-19 pandemic, the need for real-time, on-demand healthcare services has become more apparent than ever. Fortunately, wearable Internet of Medical Things (IoMT) devices, such as wrist actigraphs and smartphones, offer a promising solution by generating objective data that can aid in early-stage mental health diagnosis. This paper presents a novel Deep Convolutional Neural Architecture (CNN) with a split attention mechanism, that outperforms the traditional methods of analyzing motor activity data in diagnosing schizophrenia. To address the unique characteristics of motor activity, the proposed method includes a novel imputation method, a sampling technique based on a sliding window for sample expansion, and the Synthetic Minority Over-sampling Technique (SMOTE) technique for class balancing. The results demonstrate the highest accuracy of 94% using 24-h actigraphy data. Overall, the application of this methodology can greatly contribute to the development of pervasive healthcare systems, providing non-invasive, objective, and real-time mental healthcare services.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Data availability

The dataset is freely available. One can directly download this dataset from this source: https://datasets.simula.no/psykose.

References

  1. Arora A, Chakraborty P, Bhatia MP (2021) Intervention of wearables and smartphones in real time monitoring of sleep and behavioral health: an assessment using adaptive neuro-fuzzy technique. Arabi J Sci Eng 47:1999–2024

    Article  Google Scholar 

  2. Laursen TM, Nordentoft M, Mortensen PB (2014) Excess early mortality in schizophrenia. Ann Rev Clin Psychol 10:425–448. https://doi.org/10.1146/annurev-clinpsy-032813-153657

    Article  Google Scholar 

  3. Zanello A, Berthoud L, Ventura J, Merlo MC (2013) The brief psychiatric rating scale (version 4.0) factorial structure and its sensitivity in the treatment of outpatients with unipolar depression. Psych Res 210(2):626–633. https://doi.org/10.1016/j.psychres.2013.07.001

    Article  Google Scholar 

  4. Kumari S, Malik M, Florival C, Manalai P, Sonje S (2017) An Assessment of Five (PANSS, SAPS, SANS, NSA-16, CGI-SCH) commonly used symptoms rating scales in schizophrenia and comparison to newer scales (CAINS, BNSS). J Addict Res Ther 8(3):324. https://doi.org/10.4172/2155-6105.1000324

    Article  Google Scholar 

  5. Sadeghi D, Shoeibi A, Ghassemi N, Moridian P, Khadem A, Alizadehsani R, Teshnehlab M, Gorriz JM, Khozeimeh F, Zhang YD, Nahavandi S, Acharya UR (2022) An overview of artificial intelligence techniques for diagnosis of Schizophrenia based on magnetic resonance imaging modalities: Methods, challenges, and future works. Comput Biol Med 146:105554. https://doi.org/10.1016/j.compbiomed.2022.105554

    Article  Google Scholar 

  6. Marco EM, Velarde E, Llorente R, Laviola G (2016) Disrupted circadian rhythm as a common player in developmental models of neuropsychiatric disorders. Curr Topics Behav Neurosci 29:155–181. https://doi.org/10.1007/7854_2015_419

    Article  Google Scholar 

  7. Bonato P (2005) Advances in wearable technology and applications in physical medicine and rehabilitation. J Neuroeng Rehabil 2(1):2. https://doi.org/10.1186/1743-0003-2-2

    Article  Google Scholar 

  8. Liu Y, Dong B, Guo B, Yang J, Peng W (2015) Combination of cloud computing and Internet of Things (IoT) in medical monitoring systems. Int J Hybrid Inform Technol 8(12):367–376

    Google Scholar 

  9. Arora A, Chakraborty P, Bhatia MP (2020) Analysis of data from wearable sensors for sleep quality estimation and prediction using deep learning. Arab J Sci Eng 45:10793–10812

    Article  Google Scholar 

  10. Granholm E, Holden JL, Mikhael T, Link PC, Swendsen J, Depp C, Moore RC, Harvey PD (2020) What do people with schizophrenia do all day? Ecological momentary assessment of real-world functioning in schizophrenia. Schizophr Bull 46(2):242–251. https://doi.org/10.1093/schbul/sbz070

    Article  Google Scholar 

  11. Akhtar MM, Shatat RSA, Shatat ASA, Shabi A, Sakher A (2022) IoMT-based smart healthcare monitoring system using adaptive wavelet entropy deep feature fusion and improved RNN. Multimed Tools Appl 82:17353–17390. https://doi.org/10.1007/s11042-022-13934-5

    Article  Google Scholar 

  12. Cooper MA, Kluding PM, Wright DE (2016) Emerging relationships between exercise, sensory nerves, and neuropathic pain. Front Neurosci 10:214506. https://doi.org/10.3389/fnins.2016.00372

    Article  Google Scholar 

  13. Martin JL, Hakim AD (2011) Wrist actigraphy. Chest 139(6):1514–1527. https://doi.org/10.1378/chest.10-1872

    Article  Google Scholar 

  14. Garcia Ceja E, Riegler M, Jakobsen P, Torresen J, Nordgreen T, Oedegaard K, Fasmer OB (2018) Depresjon: a motor activity database of depression episodes in unipolar and bipolar patients. Association for Computing Machinery, New York, NY, pp 472–477. https://doi.org/10.1145/3204949.3208125

  15. Akiskal HS, McKinney Jr WT (1975) Overview of recent research in depression. Integration of ten conceptual models into a comprehensive clinical frame. Arch Gen Psych 32(3):285–305. https://doi.org/10.1001/archpsyc.1975.01760210019001

    Article  Google Scholar 

  16. Andreasen NC (1997) The evolving concept of schizophrenia: from Kraepelin to the present and future. Schizophr Res 28(2–3):105–109. https://doi.org/10.1016/s0920-9964(97)00112-6

    Article  Google Scholar 

  17. Saeb S, Zhang M, Kwasny MM, Karr CJ, Kording K, Mohr DC (2015) The relationship between clinical, momentary, and sensor-based assessment of depression. In: International Conference on Pervasive Computing Technologies for Healthcare. ICST. https://doi.org/10.4108/icst.pervasivehealth.2015.259034

  18. Zanella-Calzada LA, Galván-Tejada CE, Chávez-Lamas NM, Gracia-Cortés MDC, Magallanes-Quintanar R, Celaya-Padilla JM, Galván-Tejada JI, Gamboa-Rosales H (2019) Feature extraction in motor activity signal: towards a depression episodes detection in unipolar and bipolar patients. Diagnostics (Basel, Switzerland) 9(1):8. https://doi.org/10.3390/diagnostics9010008

    Article  Google Scholar 

  19. Bondugula RK, Udgata SK, Sivangi KB (2023) A novel deep learning architecture and MINIROCKET feature extraction method for human activity recognition using ECG, PPG and inertial sensor dataset. Appl Intell 53:14400–14425. https://doi.org/10.1007/s10489-022-04250-4

    Article  Google Scholar 

  20. Lee S, Kim H, Park MJ, Jeon HJ (2021) Current advances in wearable devices and their sensors in patients with depression. Front Psych 12:672347. https://doi.org/10.3389/fpsyt.2021.672347

    Article  Google Scholar 

  21. Wang R, Wang W, Obuchi M, Scherer E, Brian R, Ben-Zeev D, Choudhury T, Kane J, Hauser M, Walsh M, Campbell A (2020) On Predicting Relapse in Schizophrenia using Mobile Sensing in a Randomized Control Trial. In: 2020 IEEE International Conference on Pervasive Computing and Communications (PerCom). IEEE, Austin, TX, pp 1–8. https://doi.org/10.1109/PerCom45495.2020.9127365

  22. Tyagi A, Singh VP, Gore MM (2022) Towards artificial intelligence in mental health: a comprehensive survey on the detection of schizophrenia. Multimed Tools Appl 82:20343–20405

    Article  Google Scholar 

  23. Wiem MB, Lachiri Z (2017) Emotion Classification in Arousal Valence Model using MAHNOB-HCI Database. Int J Adv Comput Sci Appl 8(3):318–323

    Google Scholar 

  24. Aydemir E, Tuncer T, Dogan S, Gururajan R, Acharya UR (2021) Automated major depressive disorder detection using melamine pattern with EEG signals. Appl Intell 51:6449–6466

    Article  Google Scholar 

  25. Wang Z, Zhang J, He Y, Zhang J (2022) EEG emotion recognition using multichannel weighted multiscale permutation entropy. Appl Intell 52:12064–12076

    Article  Google Scholar 

  26. Haag A, Goronzy S, Schaich P, Williams JB (2004) Emotion recognition using bio-sensors: first steps towards an automatic system. In: Affective Dialogue Systems. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-24842-2_4

  27. Kim KH, Bang SW, Kim SR (2004) Emotion recognition system using short-term monitoring of physiological signals. Med Biol Eng Comput 42:419–427. https://doi.org/10.1007/BF02344719

    Article  Google Scholar 

  28. Seal A, Bajpai R, Karnati M, Agnihotri J, Yazidi A, Herrera-Viedma E, Krejcar O (2022) Benchmarks for machine learning in depression discrimination using electroencephalography signals. Applied Intelligence 53(10):12666–12683. https://doi.org/10.1007/s10489-022-04159-y

    Article  Google Scholar 

  29. Krishnan PT, Joseph Raj AN, Balasubramanian P, Chen Y (2020) Schizophrenia detection using multivariate empirical mode decomposition and entropy measures from multichannel EEG signal. Biocybernetics Biomed Eng 10(3):1124–1129

    Article  Google Scholar 

  30. Teicher MH, Glod CA, Magnus E, Harper D, Benson G, Krueger K, McGreenery CE (1997) Circadian rest-activity disturbances in seasonal affective disorder. Arch Gen Psych 54(2):124–130. https://doi.org/10.1001/archpsyc.1997.01830140034007

    Article  Google Scholar 

  31. Armitage R, Hoffmann R, Emslie G, Rintelman J, Moore J, Lewis K (2004) Rest-activity cycles in childhood and adolescent depression. J Amer Acad Child & Adolesc Psych 43(6):761–769. https://doi.org/10.1097/01.chi.0000122731.72597.4e

    Article  Google Scholar 

  32. Cheniaux E, Silva RAD, Santana CM, Filgueiras A (2018) Changes in energy and motor activity: core symptoms of bipolar mania and depression? Revista Brasileira de Psiquiatria (Sao Paulo, Brazil : 1999) 40(3):233–237. https://doi.org/10.1590/1516-4446-2017-2301

  33. Ashraf A, Gunawan T, Rahman F, Kartiwi M, Ismail N, Ulfiah (2020) A summarization of the visual depression databases for depression detection. In: 2020 6th International Conference on Wireless and Telematics (ICWT). IEEE, Yogyakarta, Indonesia, pp 1–6. https://doi.org/10.1109/ICWT50448.2020.9243625

  34. Jakobsen P, Garcia Ceja E, Stabell L, Oedegaard K, Berle J, Thambawita V, Hicks S, Halvorsen P, Riegler M, Fasmer OB (2020) PSYKOSE: a motor activity database of patients with schizophrenia. In: 2020 IEEE 33rd International Symposium on Computer-Based Medical Systems (CBMS). IEEE, Rochester, MN. https://doi.org/10.31219/osf.io/e2tzf

  35. García-Magariño I, Sarkar D, Lacuesta R (2019) Wearable technology and mobile applications for healthcare. Mobile Inform Syst 2019:1–2. https://doi.org/10.1155/2019/6247094

    Article  Google Scholar 

  36. Liu J, Wang X, Zhang X, Pan Y, Wang X, Wang J (2018) MMM: classification of schizophrenia using multi-modality multi-atlas feature representation and multi-kernel learning. Multimedia Tools Appl 77(22):29651–29667

    Article  Google Scholar 

  37. Pacheco-González SL, Zanella-Calzada LA, Galván-Tejada CE, Chávez-Lamas NM, Rivera-Gómez JF, Galván-Tejada JI (2019) Evaluation of five classifiers for depression episodes detection. Res Comput Sci 148(10):129–138

    Article  Google Scholar 

  38. Adamczyk J, Malawski F (2021) Comparison of manual and automated feature engineering for daily activity classification in mental disorder diagnosis. Comput Inform 40:850–879. https://doi.org/10.31577/cai_2021_4_850

  39. Zanella-Calzada LA, Galván-Tejada CE, Chávez-Lamas NM, Gracia-Cortés MDC, Magallanes-Quintanar R, Celaya-Padilla JM, Galván-Tejada JI, Gamboa-Rosales H (2019) Feature extraction in motor activity signal: towards a depression episodes detection in unipolar and bipolar patients. Diagnostics (Basel, Switzerland) 9(1):8. https://doi.org/10.3390/diagnostics9010008

    Article  Google Scholar 

  40. Kumar A, Sangwan SR, Arora A, Menon VG (2022) Depress-DCNF: a deep convolutional neuro-fuzzy model for detection of depression episodes using IoMT. Applied Soft Comput 122:108863. https://doi.org/10.1016/j.asoc.2022.108863

  41. Rodríguez-Ruiz JG, Galván-Tejada CE, Vázquez-Reyes S, Gálvan-Tejada JI, Gamboa-Rosales H (2019) Nighttime depression episodes classification using a formal method: knowledge discovery in databases. Proceedings - 2019 7th International Conference in Software Engineering Research and Innovation, CONISOFT 2019, (pp 207–212)

  42. Strous RD, Koppel M, Fine J, Nachliel S, Shaked G, Zivotofsky A (2009) Automated characterization and identification of schizophrenia in writing. J Nerv Mental Dis 197(8):585–588. https://doi.org/10.1097/NMD.0b013e3181b09068

    Article  Google Scholar 

  43. Feng C, Wang H, Lu N, Chen T, He H, Lu Y, Tu XM (2014) Log-transformation and its implications for data analysis. Shanghai Arch Psychiat 26(2):105–109. https://doi.org/10.3969/j.issn.1002-0829.2014.02.009

    Article  Google Scholar 

  44. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Polosukhin I et al (2017) Attention is all you need. In: 31st Conference on Neural Information Processing Systems (NIPS 2017). IEEE, Long Beach, CA, pp 5998–6008

  45. Niu Zhaoyang, Zhong Guoqiang, Yu Hui (2021) A review on the attention mechanism of deep learning. Neurocomputing 452:48–62. https://doi.org/10.1016/j.neucom.2021.03.091

    Article  Google Scholar 

  46. Garcia-Ceja E, Riegler M, Jakobsen P, Tørresen J, Nordgreen T, Oedegaard KJ, Fasmer OB (2018) Motor activity based classification of depression in unipolar and bipolar patients. In: 2018 IEEE 31st International Symposium on Computer-Based Medical Systems (CBMS). IEEE, Karlstad, Sweden, pp 316–321

  47. Jakobsen P, Garcia-Ceja E, Riegler M, Stabell LA, Nordgreen T, Tørresen J, Fasmer OB, Oedegaard KJ (2020) Applying machine learning in motor activity time series of depressed bipolar and unipolar patients compared to healthy controls. PLoS ONE 15(8):e0231995. https://doi.org/10.1371/journal.pone.0231995

  48. Nguyen D, Chan C, Li AA, Phan D, Lan C (2022) Decision support system for the differentiation of schizophrenia and mood disorders using multiple deep learning models on wearable devices data. Health Inform J 28(4):14604582221137537. https://doi.org/10.1177/14604582221137537

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science & Engineering, Netaji Subhas University of Technology, New Delhi, India

    Muzafar Mehraj Misgar & MPS Bhatia

Authors
  1. Muzafar Mehraj Misgar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. MPS Bhatia
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Muzafar Mehraj Misgar.

Ethics declarations

Conflict of interest

There are no conflicts of interest between the authors and the subject matter or materials discussed in the manuscript. The authors declare that they have no personal or financial interest in the findings or outcomes of the research.

Conceptualization, designing methodology, and experimentation were majorly done by author 1, and Writing proofreading, revising, and editing were done equally by both.

No consent is required as the dataset is made freely available by original contributors.

Additional information

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Misgar, M.M., Bhatia, M. Utilizing deep convolutional neural architecture with attention mechanism for objective diagnosis of schizophrenia using wearable IoMT devices. Multimed Tools Appl 83, 39601–39620 (2024). https://doi.org/10.1007/s11042-023-17119-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-17119-6

Keywords

Search

Navigation