SpringerLink
Log in

An Integrated CNN-LSTM Model for Micro Hand Gesture Recognition

  • Conference paper
  • First Online:
Intelligent Computing and Optimization (ICO 2020)

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 1324))

Included in the following conference series:

  • 1262 Accesses

Abstract

Vision based micro gesture recognition systems enable the development of HCI (Human Computer Interaction) interfaces to mirror real-world experiences. It is unlikely that a gesture recognition method will be suitable for every application, as each gesture recognition system rely on the user cultural background and application domain. This research is an attempt to develop a micro gesture recognition system suitable for the asian culture. However, hands vary in shapes and sizes while gesture varies in orientation and motion. For accurate feature extraction, deep learning approaches are considered. Here, an integrated CNN-LSTM (Convolutional Neural Network- Long Short-Term Memory) model is proposed for building micro gesture recognition system. To demonstrate the applicability of the system two micro hand gesture-based datasets namely, standard and local dataset consisting of ten significant classes are used. Besides, the model is tested against both augmented and unaugmented datasets. The accuracy achieved for standard data with augmentation is 99.0%, while the accuracy achieved for local data with augmentation is 96.1% by applying CNN-LSTM model. In case of both the datasets, the proposed CNN-LSTM model appears to perform better than the other pre-trained CNN models including ResNet, MobileNet, VGG16 and VGG9 as well as CNN excluding LSTM.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free ship** worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Ahmed, T.U., Hossain, M.S., Alam, M.J., Andersson, K.: An integrated cnn-rnn framework to assess road crack. In: 22nd International Conference on Computer and Information Technology (ICCIT), pp. 1–6. IEEE (2019)

    Google Scholar 

  2. Ahmed, T.U., Hossain, S., Hossain, M.S., ul Islam, R., Andersson, K.: Facial expression recognition using convolutional neural network with data augmentation. In: Joint 8th International Conference on Informatics, Electronics & Vision (ICIEV) and 2019 3rd International Conference on Imaging, Vision & Pattern Recognition (icIVPR), pp. 336–341. IEEE (2019)

    Google Scholar 

  3. Akoum, A., Al Mawla, N., et al.: Hand gesture recognition approach for asl language using hand extraction algorithm. J. Softw. Eng. Appl. 8(08), 419 (2015)

    Article  Google Scholar 

  4. Basnin, N., Hossain, M.S., Nahar, L.: An integrated cnn-lstm model for bangla lexical sign language recognition. In: Proceedings of 2nd International Conference on Trends in Computational and Cognitive Engineering (TCCE-2020) Springer Joint 8th International Conference on Informatics (2020)

    Google Scholar 

  5. Chowdhury, R.R., Hossain, M.S., ul Islam, R., Andersson, K., Hossain, S.: Bangla handwritten character recognition using convolutional neural network with data augmentation. In: Joint 8th International Conference on Informatics, Electronics & Vision (ICIEV) and 2019 3rd International Conference on Imaging, Vision & Pattern Recognition (icIVPR), pp. 318–323. IEEE (2019)

    Google Scholar 

  6. Donahue, J., Anne Hendricks, L., Guadarrama, S., Rohrbach, M., Venugopalan, S., Saenko, K., Darrell, T.: Long-term recurrent convolutional networks for visual recognition and description. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2625–2634 (2015)

    Google Scholar 

  7. Goyal, P., Dollar, P., Girshick, R., Noordhuis, P., Wesolowski, L., Kyrola, A., Tulloch, A., Jia, Y., He, K.: Accurate, large minibatch sgd: training imagenet in 1 hour. ar**v preprint ar**v:1706.02677 (2017)

  8. Greenfield, P., Miller, J.T., Hsu, J., White, R.L.: Numarray: a new scientific array package for python. PyCon DC (2003)

    Google Scholar 

  9. Grundland, M., Dodgson, N.A.: Decolorize: fast, contrast enhancing, color to grayscale conversion. Pattern Recogn. 40(11), 2891–2896 (2007)

    Article  Google Scholar 

  10. Gti: Hand gesture recognition database (2018), https://www.kaggle.com/gtiupm/leapgestrecog

  11. Gulli, A., Pal, S.: Deep learning with Keras. Packt Publishing Ltd (2017)

    Google Scholar 

  12. Haralick, R.M., Sternberg, S.R., Zhuang, X.: Image analysis using mathematical morphology. IEEE Trans. Pattern Anal. Mach. Intell. 4, 532–550 (1987)

    Article  Google Scholar 

  13. Hossain, M.S., Amin, S.U., Alsulaiman, M., Muhammad, G.: Applying deep learning for epilepsy seizure detection and brain map** visualization. ACM Trans. Multimedia Comput. Commun. Appl. (TOMM) 15(1s), 1–17 (2019)

    Article  Google Scholar 

  14. Islam, M.Z., Hossain, M.S., ul Islam, R., Andersson, K.: Static hand gesture recognition using convolutional neural network with data augmentation. In: Joint 8th International Conference on Informatics, Electronics & Vision (ICIEV) and 2019 3rd International Conference on Imaging, Vision & Pattern Recognition (icIVPR), pp. 324–329. IEEE (2019)

    Google Scholar 

  15. Islam, R.U., Hossain, M.S., Andersson, K.: A deep learning inspired belief rulebased expert system. IEEE Access 8, 190637–190651 (2020)

    Article  Google Scholar 

  16. Jalab, H.A.: Static hand gesture recognition for human computer interaction. Inf. Technol. J. 11(9), 1265 (2012)

    Article  Google Scholar 

  17. Kabir, S., Islam, R.U., Hossain, M.S., Andersson, K.: An integrated approach of belief rule base and deep learning to predict air pollution. Sensors 20(7), 1956 (2020)

    Article  Google Scholar 

  18. Nandagopalan, S., Kumar, P.K.: Deep convolutional network based saliency prediction for retrieval of natural images. In: International Conference on Intelligent Computing & Optimization. pp. 487–496. Springer (2018)

    Google Scholar 

  19. Nguyen, T.N., Huynh, H.H., Meunier, J.: Static hand gesture recognition using artificial neural network. J. Image Graph. 1(1), 34–38 (2013)

    Article  Google Scholar 

  20. Nguyen, T.N., Huynh, H.H., Meunier, J.: Static hand gesture recognition using principal component analysis combined with artificial neural network. J. Autom. Control Eng. 3(1), 40–45 (2015)

    Article  Google Scholar 

  21. Oyedotun, O.K., Khashman, A.: Deep learning in vision-based static hand gesture recognition. Neural Comput. Appl. 28(12), 3941–3951 (2017)

    Article  Google Scholar 

  22. Oz¨o˘g¨ur-Aky¨uz, S., Otar, B.C., Atas, P.K.: Ensemble cluster pruning via convex- concave programming. Comput. Intell. 36(1), 297–319 (2020)

    Google Scholar 

  23. Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15(1), 1929–1958 (2014)

    MathSciNet  MATH  Google Scholar 

  24. Stergiopoulou, E., Papamarkos, N.: Hand gesture recognition using a neural network shape fitting technique. Eng. Appl. Artif. Intell. 22(8), 1141–1158 (2009)

    Article  Google Scholar 

  25. Uddin Ahmed, T., Jamil, M.N., Hossain, M.S., Andersson, K., Hossain, M.S.: An integrated real-time deep learning and belief rule base intelligent system to assess facial expression under uncertainty. In: 9th International Conference on Informatics, Electronics & Vision (ICIEV). IEEE Computer Society (2020)

    Google Scholar 

  26. Wang, W., Yang, J., **ao, J., Li, S., Zhou, D.: Face recognition based on deep learning. In: International Conference on Human Centered Computing. pp. 812– 820. Springer (2014)

    Google Scholar 

  27. Yingxin, X., **ghua, L., Lichun, W., Dehui, K.: A robust hand gesture recognition method via convolutional neural network. In: 6th International Conference on Digital Home (ICDH), pp. 64–67. IEEE (2016)

    Google Scholar 

  28. Zhu, Y., Huang, C.: An improved median filtering algorithm for image noise reduction. Phys. Procedia 25, 609–616 (2012)

    Article  Google Scholar 

  29. Zisad, S.N., Hossain, M.S., Andersson, K.: Speech emotion recognition in neurological disorders using convolutional neural network. In: International Conference on Brain Informatics. pp. 287–296. Springer (2020)

    Google Scholar 

  30. Zivkovic, Z., Van Der Heijden, F.: Efficient adaptive density estimation per image pixel for the task of background subtraction. Pattern Recogn. Lett. 27(7), 773–780 (2006)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. International Islamic University Chittagong, Chittagong, Bangladesh

    Nanziba Basnin & Lutfun Nahar

  2. University of Chittagong, Chittagong, Bangladesh

    Mohammad Shahada Hossain

Authors
  1. Nanziba Basnin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lutfun Nahar
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mohammad Shahada Hossain
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Fundamental and Applied Sciences, Universiti Teknologi Petronas, Tronoh, Perak, Malaysia

    Pandian Vasant

  2. Faculty of Electrical Engineering and Computer Science, VÅ B TU Ostrava, Ostrava, Czech Republic

    Ivan Zelinka

  3. Faculty of Engineering Management, Poznan University of Technology, Poznan, Poland

    Gerhard-Wilhelm Weber

Rights and permissions

Reprints and permissions

Copyright information

© 2021 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Basnin, N., Nahar, L., Hossain, M.S. (2021). An Integrated CNN-LSTM Model for Micro Hand Gesture Recognition. In: Vasant, P., Zelinka, I., Weber, GW. (eds) Intelligent Computing and Optimization. ICO 2020. Advances in Intelligent Systems and Computing, vol 1324. Springer, Cham. https://doi.org/10.1007/978-3-030-68154-8_35

Download citation

Publish with us

Policies and ethics

Search

Navigation