Abstract
Human-Computer interaction (HCI) with gesture recognition is designed to recognize a number of meaningful human expressions, and has become a valuable and intuitive computer input technique. Hand gestures are one of the most intuitive and common forms of communication, and can communicate a wide range of meaning. Vision-based hand gesture recognition has received a significant amount of research attention in recent years. However, the field still presents a number of challenges for researchers. In the vision-based hand gesture interaction process between humans and computers, gesture interpretation must be performed quickly and with high accuracy. In this paper, a low-cost HCI system with hand gesture recognition is proposed. This system uses several vision techniques. Skin and motion detection is used for capturing the region-of-interest from the background regions. A connected component labeling algorithm is proposed to identify the centroid of an object. To identify the exact area of hand gesture, the arm area is removed with the aid of a convex hull algorithm. Moreover, a real-time demonstration system is developed, based on a single-camera mechanism which allows for the use of wearable devices. Simulation results show that the recognition rate is still high, although some interference is encountered in the simulated environments.
Similar content being viewed by others
References
Aksaç A, Öztürk O, Özyer T (2011) Real-time multi-objective hand posture/gesture recognition by using distance classifiers and finite state machine for virtual mouse operations. IEEE International Conference on Electrical and Electronics Engineering (ELECO) 7th, pp. 457–461
Aviles-Arriaga HH, Sucar LE, Mendoza CE, Vargas B (2003) Visual recognition of gestures using dynamic naive Bayesian classifiers. Robot and Human Interactive Communication, Proceedings. The 12th IEEE International Workshop on Robot and Human Interactive Communication, pp. 133–138
Bellarbi A, Benbelkacem S, Henda NZ, Belhocine M (2011) Hand gesture interaction using color-based method for tabletop interfaces. IEEE International Symposium on Intelligent Signal Processing (WISP):1–6
Berman S, Stern S (2012) Sensors for gesture recognition systems. IEEE Transactions on Systems, Man, and Cybernetics, Part C:277–290. https://doi.org/10.1109/TSMCC.2011.2161077
Burger T, Caplier A, Mancini S (2005) Cued speech hand gestures recognition tool. IEEE European Signal Processing Conference:1–4
Chen Q, Georganas ND, Petriu EM (2008) Hand gesture recognition using haar-like features and a stochastic context-free grammar. IEEE Trans Instrum Meas 57(8):1562–1571
Cheng LT, Chih WK, Tsai A, Chih WC (2009) Hand posture recognition using hidden conditional random fields. IEEE/ASME International Conference on Advanced Intelligent Mechatronics, pp.1828–1833, pp. 14–17
Chiang T, Fan CP (2018) 3D Depth Information Based 2D Low-Complexity Hand Posture and Gesture Recognition Design for Human Computer Interactions. International Conference on Computer and Communication Systems (ICCCS). https://doi.org/10.1109/CCOMS.2018.8463327
Deyou X (2006) A neural approach for hand gesture recognition in virtual reality driving training system of SPG. Proc. of International Conference on Pattern Recognition, ICPR’06, pp. 519–522
Dias DB, Madeo RCB, Rocha T, Biscaro HH, Peres SM (2009) Hand movement recognition for Brazilian Sign Language: A study using distance-based neural networks. Neural Networks, IEEE - INNS - ENNS International Joint Conference on, pp. 697–704
Dipietro L, Sabatini AM, Dario P (2008) A survey of glove-based systems and their Applications. IEEE Transactions on Systems, Man, and Cybernetics, Part C:461–482. https://doi.org/10.1109/TSMCC.2008.923862
Duan HX, Zhang QY, Ma W (2011) An approach to dynamic hand gesture modeling and real-time extraction. IEEE International Conference on Communication Software and Networks (ICCSN):139–142
Elmezain M, Al-Hamadi A, Michaelis B (2008) Real-time capable system for hand motion detection, labeling, data association and tracking gesture recognition using hidden Markov models in stereo color image sequences. The Journal of WSCG’08 16:65–72
Erol A, Bebis G, Nicolescu M, Boyle RD, Twombly X (2007) Vision-based hand pose estimation: A review. Comput Vis Image Understanding 108(1–2):52–73
Foxlin E (2002) Motion tracking requirements and technologies. Handbook of Virtual Environment Technology, pp. 163–210
Ghosh DK, Ari S (2011) A static hand gesture recognition algorithm using k-mean based radial basis function neural network. IEEE International Conference on Information, Communications and Signal Processing:1–5
Han J, Shao L, Xu D, Shotton J (2013) Enhanced computer vision with Microsoft Kinect sensor: A review. IEEE Transactions on Systems, Man, and Cybernetics:1318–1334. https://doi.org/10.1109/TCYB.2013.2265378
Heung-Il S, Kee SB, Whan LS (2010) Hand gesture recognition based on dynamic Bayesian network framework. Pattern Recogn 43(9):3059–3072
Hsieh CC, Liou DH, Lee D (2010) A real time hand gesture recognition system using motion history image. IEEE International Conference on Singal Processing Systems (ICSPS) 2:394–398
Kukharev G, Nowosielski A (2004) Visitor identification: elaborating real time face recognition system. in Proceedings of the 12th International Conference in Central Europe on Computer Graphics, Visualization and Computer Vision, Plzen-Bory, Czech Republic, pp. 157–164
Kumar S, Kaurav A (2018) Hand Gesture through Geometric Moments (HCI Based). International Conference on Inventive Systems and Control (ICISC), DOI: https://doi.org/10.1109/ICISC.2018.8398862
Kumar P, Rautaray SS, Agrawal A (2012) Hand data glove: A new generation real-time mouse for human-computer interaction. International Conference on Recent Advances in Information Technology (RAIT), pp. 750–755
Lacassagne L, Milgram M, Garda P (1999) Motion detection, labeling, data association and tracking, in real-time on RISC computer. IEEE Image Analysis and Processing, Proceedings. International Conference on, pp. 520–525
Lee C, Xu Y (1996) Online Interactive learning of gestures for human/robot interfaces. IEEE International Conference on Robotics and Automation 4:2982–2987
Lin L, Cong Y, Tang Y (2012) Hand gesture recognition using RGB-D cue. IEEE International Conference on Information and Automation (ICIA):311–316
Lu Z, Chen X, Li Q, Zhang X, Zhou P (2014) A Hand Gesture Recognition Framework and Wearable Gesture-Based Interaction Prototype for Mobile Devices. IEEE Transactions on Human-Machine Systems:293–299. https://doi.org/10.1109/THMS.2014.2302794
Mitra S, Acharya T (2007) Gesture recognition: A survey. IEEE Transactions on Systems, Man, and Cybernetics, Part C:311–324. https://doi.org/10.1109/TSMCC.2007.893280
Modler P, Myatt T (2008) Recognition of separate hand gestures by Time-Delay Neural Networks based on multistate spectral image patterns from cyclic hand movements. Systems, Man and Cybernetics, 2008. SMC 2008. IEEE International Conference on, pp. 1539–1544
Nguyen DB, Enokida S, Toshiaki E (2005) Real-time hand tracking and gesture recognition system. IGVIP’05, pp. 362–368
Panwar M, Mehra PS (2011) Hand tracking and gesture recognition for human-computer interaction. Image Information Processing (ICIIP), 2011 International Conference on
Rahmat RW, Al-Tairi ZH, Saripan MI, Sulaiman PS (2012) Removing shadow for hand segmentation based on background subtraction. International Conference on Advanced Computer Science Applications and Technologies (ACSAT), pp. 481–485
Rautaray SS, Agrawal A (2012) Design of gesture recognition system for dynamic user interface analysis. IEEE International Conference on Technology Enhanced Education (ICTEE):1–6
Rossol N, Cheng I, Basu A (2016) A Multisensor Technique for Gesture Recognition Through Intelligent Skeletal Pose Analysis. IEEE Transactions on Human-Machine Systems:350–359. https://doi.org/10.1109/THMS.2015.2467212
Sahoo JP, Ari S, Ghosh DK (2018) Hand gesture recognition using DWT and F-ratio based feature descriptor. IET Image Process 12(10):1780–1787
Song S, Yan D, **e Y (2018) Design of control system based on hand gesture recognition. International Conference on Networking, Sensing and Control (ICNSC). DOI:https://doi.org/10.1109/ICNSC.2018. 8361351
Takahashi T, Kishino F (1991) Hand gesture coding based on experiments using a hand gesture interface device. SIGCHI Bull 23(2):67–74
Tang C, Ou Y, Jiang G, **e Q, Xu Y (2012) Hand tracking and pose recognition via depth and color information. IEEE International Conference on Robotics and Biomimetics (ROBIO):1104–1109
Turk M (2001) Handbook of Virtual Environment Technology. Lawrence Erlbaum Associates, Gesture Recognition, Chap. 9
Wachs JP, Kolsch M, Stern H, Edan Y (2011) Vision-based hand gesture applications. Commun ACM 54(2):60–71
Wan M (2014) Feature extraction using two-dimensional maximum embedding difference. Inf Sci 274:55–69
Wan M, Yang G, Gai S, Yang Z (2017) Two-dimensional Discriminant Locality Preserving Projections (2DDLPP) and Its Application to Feature Extraction via Fuzzy Set. Multimed Tools Appl 76:355–371
Wan M et al (2017) Local graph embedding based on maximum margin criterion via fuzzy set. Fuzzy Sets Syst 318:120–131
Yi B, Harris FC, Wang L, Yan Y (2005) Real-time natural hand gestures. Proceedings of IEEE Computing in Science & Engineering and the American Institute of Physics 7(3):92–97
Zaletelj J, Perhavc J, Tasic JF (2007) Vision-based human-computer interface using hand gestures. International Workshop on Image Analysis for Multimedia Interactive Services (WIAMIS'07)
Zhan X, Chen X, Li Y, Lantz V, Wang K, Yang J (2011) A framework for hand gesture recognition based on accelerometer and EMG sensors. IEEE Transactions on Systems, Man, and Cybernetics, Part A:1064–1076. https://doi.org/10.1109/TSMCA.2011.2116004
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Tsai, TH., Huang, CC. & Zhang, KL. Design of hand gesture recognition system for human-computer interaction. Multimed Tools Appl 79, 5989–6007 (2020). https://doi.org/10.1007/s11042-019-08274-w
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-019-08274-w