SpringerLink
Log in

A multimodal framework based on deep belief network for human locomotion intent prediction

  • Original Article
  • Published:
Biomedical Engineering Letters Aims and scope Submit manuscript

Abstract

Accurate prediction of human locomotion intent benefits the seamless switching of lower limb exoskeleton controllers in different terrains to assist humans in walking safely. In this paper, a deep belief network (DBN) was developed to construct a multimodal framework for recognizing various locomotion modes and predicting transition tasks. Three fusion strategies (data level, feature level, and decision level) were explored, and optimal network performance was obtained. This method could be tested on public datasets. For the continuous performance of steady state, the best prediction accuracy achieved was 97.64% in user-dependent testing and 96.80% in user-independent testing. During the transition state, the system accurately predicted all transitions (user-dependent: 96.37%, user-independent: 95.01%). The multimodal framework based on DBN can accurately predict the human locomotion intent. The experimental results demonstrate the potential of the proposed model in the volition control of the lower limb exoskeleton.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Sun W, Lin JW, Su SF, Wang N, Er MJ. Reduced adaptive fuzzy decoupling control for lower limb exoskeleton. IEEE Transact Cybernetics. 2021;51(3):1099–109.

    Article  Google Scholar 

  2. Li Z, Zhao K, Zhang L, Wu X, Su CY. Human-in-the-loop control of a wearable lower limb exoskeleton for stable dynamic walking. IEEE/ASME Trans Mechatron. 2021;26(5):2700–11.

    Article  Google Scholar 

  3. Ma Y, Wu XY, Yang SX, Dang C. Online gait planning of lower-limb exoskeleton robot for paraplegic rehabilitation considering weight transfer process. IEEE Trans Autom Sci Eng. 2021;18(2):414–25.

    Article  Google Scholar 

  4. Anderson AJ, Hudak YF, Muir BC, Aubin PM. Design, control, and evaluation of a robotic ankle-foot prosthesis emulator. IEEE Transact Med Robot Bionics. 2023;5(3):741–52.

    Article  Google Scholar 

  5. Zheng E, Wang Q, Qiao H. Locomotion mode recognition with robotic transtibial prosthesis in inter-session and inter-day applications. IEEE Transact Neural Syst Rehabilitation Eng. 2019;27(9):1836–45.

    Article  Google Scholar 

  6. Li XL, Hao YF, Zhang JH, Wang C, Li D, Zhang JJ. Design, modeling and experiments of a variable stiffness soft robotic glove for stroke patients with clenched fist deformity. IEEE Robot Autom Lett. 2023;8(7):4044–51.

    Article  Google Scholar 

  7. Gao Y, Jia B, Houston M, Zhang Y. Hybrid EEG-fNIRS brain computer interface based on common spatial pattern by using EEG-informed general linear model. IEEE Trans Instrum Meas. 2023;72:1–10.

    Google Scholar 

  8. Huang C, **ao Y, Xu G. Predicting human intention-behavior through EEG signal analysis using multi-scale CNN. IEEE/ACM Trans Comput Biol Bioinf. 2021;18(5):1722–9.

    Article  Google Scholar 

  9. Elsayed NE, Tolba AS, Rashad MZ, Belal T, Sarhan S. A deep learning approach for brain computer interaction-motor execution EEG signal classification. IEEE Access. 2021;9:101513–29.

    Article  Google Scholar 

  10. Park S, Lee D, Chung WK, Kim K. Hierarchical motion segmentation through sEMG for continuous lower limb motions. IEEE Robot Autom Lett. 2019;4(4):4402–9.

    Article  Google Scholar 

  11. Ryu J, Kim DH. Real-time gait subphase detection using an EMG signal graph matching (ESGM) algorithm based on EMG signals. Expert Syst Appl. 2017;85:357–65.

    Article  Google Scholar 

  12. Gautam A, Panwar M, Biswas D, Acharyya A. MyoNet: a transfer-learning-based LRCN for lower limb movement recognition and knee joint angle prediction for remote monitoring of rehabilitation progress from sEMG. IEEE J Transl Eng Health Med. 2020;8:1–10.

    Article  Google Scholar 

  13. Shi X, Qin P, Zhu J, Zhai M, Shi W. Feature extraction and classification of lower limb motion based on sEMG signals. IEEE Access. 2020;8:132882–92.

    Article  Google Scholar 

  14. Kang I, Molinaro DD, Choi G, Camargo J, Young AJ. Subject-independent continuous locomotion mode classification for robotic hip exoskeleton applications. IEEE Trans Biomed Eng. 2022;69(10):3234–42.

    Article  Google Scholar 

  15. Su BY, Wang J, Liu SQ, Sheng M, **ang K. An improved motion intent recognition method for intelligent lower limb prosthesis driven by inertial motion capture data. Acta Autom Sin. 2018;46(7):1517–30.

    Google Scholar 

  16. Su BY, Wang J, Liu SQ. A CNN-based method for intent recognition using inertial measurement units and intelligent lower limb prosthesis. IEEE Trans Neural Syst Rehabil Eng. 2019;27(5):1032–42.

    Article  Google Scholar 

  17. Narayan A, Reyes FA, Ren M, Haoyong Y. Real-time hierarchical classification of time series data for locomotion mode detection. IEEE J Biomed Health Inform. 2022;26(4):1749–60.

    Article  Google Scholar 

  18. Papapicco V, Chen B. A classification approach based on directed acyclic graph to predict locomotion activities with one inertial sensor on the thigh. IEEE Transact Med Robot Bionics. 2021;3(2):436–45.

    Article  Google Scholar 

  19. Attal F, Amirat Y, Chibani A, Mohammed S. Automatic recognition of gait phases using a multiple-regression hidden markov model. IEEE/ASME Trans Mechatron. 2018;23(4):1597–607.

    Google Scholar 

  20. Huang L, Zheng J, Hu H. A gait phase detection method in complex environment based on DTW-mean templates. IEEE Senors J. 2021;21(13):15114–23.

    Article  Google Scholar 

  21. Hu H, Zheng JB, Zhan E, Yu L. Curve similarity model for real-time gait phase detection based on ground contact forces. Sensors. 2019;19:3235.

    Article  Google Scholar 

  22. Cui C, Bian G, Hou Z, Zhao J. A multimodal framework based on integration of cortical and muscular activities for decoding human intentions about lower limb motions. IEEE Trans Biomed Circuits Syst. 2017;11(4):889–99.

    Article  Google Scholar 

  23. Tucker MR. Control strategies for active lower extremity prosthetics and orthotics: a review. J Neuroeng Rehabil. 2015;12(1):1–29.

    Article  Google Scholar 

  24. Godiyal AK, Mondal M, Joshi SD. Force myography based novel strategy for locomotion classification. IEEE Transact Human-Machine Syst. 2018;48(6):648–57.

    Article  Google Scholar 

  25. Al-Quraishi MS, Elamvazuthi I, Tang TB. Multimodal fusion approach based on EEG and EMG signals for lower limb movement recognition. IEEE Senors Journal. 2021;21(24):27640–50.

    Article  Google Scholar 

  26. Wang Y, Cheng X, Jabban L, Sui X. Motion intention prediction and joint trajectories generation toward lower limb prostheses using EMG and IMU signals. IEEE Senors J. 2022;22(11):10719–29.

    Article  Google Scholar 

  27. Camargo J, Flanagan W, Csomay-Shanklin N, Kanwar B, Young A. A machine learning strategy for locomotion classification and parameter estimation using fusion of wearable sensors. IEEE Trans Biomed Eng. 2021;68(5):1569–78.

    Article  Google Scholar 

  28. Cui C, Bian GB, Hou ZG, Zhao J, Su GD, Zhou H, Peng L, Wang WQ. Simultaneous recognition and assessment of post-stroke hemiparetic gait by fusing kinematic, kinetic, and electrophysiological data. IEEE Transact Neural Syst Rehabil Eng. 2018;26(4):856–64.

    Article  Google Scholar 

  29. Zhao Y, Wang J, Zhang Y, Liu H. Flexible and wearable EMG and PSD sensors enabled locomotion mode recognition for IoHT-based in-home rehabilitation. IEEE Senors J. 2021;21(23):26311–9.

    Article  Google Scholar 

  30. Sun Y, Lo B. An artificial neural network framework for gait-based biometrics. IEEE J Biomed Health Inform. 2019;23(3):987–98.

    Article  Google Scholar 

  31. Zhang K, Wang J, De Silva CW, Fu CL. Unsupervised cross-subject adaptation for predicting human locomotion intent. IEEE Transact Neural Syst Rehabil Eng. 2020;28(3):646–57.

    Article  Google Scholar 

  32. Lee UH, Bi J, Patel R, Fouhey D. Image Transformation and cnns: a strategy for encoding human locomotor intent for autonomous wearable robots. IEEE Robot Automation Lett. 2020;5(4):5440–7.

    Article  Google Scholar 

  33. Moon J, Jung J, Kang E, Choi SI. Open set user identification using gait pattern analysis based on ensemble deep neural network. IEEE Senors J. 2022;22(17):16975–84.

    Article  Google Scholar 

  34. Hu B, Rouse E. Benchmark datasets for bilateral lower-limb neuromechanical signals from wearable sensors during unassisted locomotion in able-bodied individuals. Front Robot AI. 2018;5(14):1–5.

    Google Scholar 

  35. Pan T, Tsai WL, Chang CY, Yeh CW. A hierarchical hand gesture recognition framework for sports referee training-based EMG and accelerometer sensors. IEEE Transact Cybern. 2022;52(5):3172–83.

    Article  Google Scholar 

  36. He H, Fan Z, Levi JH, Zhi D, Daniel RR, Kevin BE. Continuous Locomotion-Mode Identification for Prosthetic Legs Based on Neuromuscular-Mechanical Fusion. IEEE Transact Biomed Eng. 2011;58(10):2867–75.

    Article  Google Scholar 

Download references

Funding

This work was supported by [Tian** Outstanding Youth Fund Project, Natural Science Foundation of Hebei Province, National Natural Science Foundation of China and National Key R&D Program of China] (Grant numbers [19JCJQJC61600], [F2020202051], [F2020202053], [62203149] and [2022YFC3601704]).

Author information

Authors and Affiliations

  1. School of Mechanical Engineering, Hebei University of Technology, Tian**, 300401, China

    Jiayi Li & Jian Cao

  2. School of Mechanical Engineering, University of Science and Technology Bei**g, Bei**g, 100083, China

    Jianhua Zhang & Hui Li

  3. School of Mechanical and Materials Engineering, North China University of Technology, Bei**g, 100144, China

    Kexiang Li

Authors
  1. Jiayi Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jianhua Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kexiang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jian Cao
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Hui Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors contributed to the study conception and design. Material preparation, and analysis were performed by [JL], [JZ], [KL], [JC] and [HL]. The first draft of the manuscript was written by [JL] and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Jianhua Zhang.

Ethics declarations

Conflict of interest

The authors have no relevant financial or non-financial interests to disclose.

Ethical approval

This experiment was based on a public dataset and did not require ethics approval.

Consent to participate

Not applicable.

Consent to publications

All authors are consent for publication.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, J., Zhang, J., Li, K. et al. A multimodal framework based on deep belief network for human locomotion intent prediction. Biomed. Eng. Lett. 14, 559–569 (2024). https://doi.org/10.1007/s13534-024-00351-w

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13534-024-00351-w

Keywords

Search

Navigation