Abstract
Gesture recognition is very important for Human-Robot Interfaces. In this paper, we present a novel depth based method for gesture recognition to improve the interaction of a service robot autonomous shop** cart, mostly used by reduced mobility people. In the proposed solution, the identification of the user is already implemented by the software present on the robot where a bounding box focusing on the user is extracted. Based on the analysis of the depth histogram, the distance from the user to the robot is calculated and the user is segmented using from the background. Then, a region growing algorithm is applied to delete all other objects in the image. We apply again a threshold technique to the original image, to obtain all the objects in front of the user. Intercepting the threshold based segmentation result with the region growing resulting image, we obtain candidate objects to be arms of the user. By applying a labelling algorithm to obtain each object individually, a Principal Component Analysis is computed to each one to obtain its center and orientation. Using that information, we intercept the silhouette of the arm with a line obtaining the upper point of the interception which indicates the hand position. A Kalman filter is then applied to track the hand and based on state machines to describe gestures (Start, Stop, Pause) we perform gesture recognition. We tested the proposed approach in a real case scenario with different users and we obtained an accuracy around 89,7%.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
We’re sorry, something doesn't seem to be working properly.
Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.
References
Verbal and Non-verbal Communication, pp. 223–235. Springer, Dordrecht (1991)
Abdi, H., Williams, L.J.: Principal component analysis. Wiley Interdisc. Rev. Comput. Stat. 2(4), 433–459 (2010). doi:10.1002/wics.101
Argyros, A.A., Lourakis, M.I.A.: Real-time tracking of multiple skin-colored objects with a possibly moving camera, pp. 368–379. Springer, Heidelberg (2004)
Bellmore, C., Ptucha, R., Savakis, A.: Interactive display using depth and RGB sensors for face and gesture control. In: 2011 Western New York Image Processing Workshop, pp. 1–4 (2011). doi:10.1109/WNYIPW.2011.6122883
den Bergh, M.V., Gool, L.V.: Combining RGB and ToF cameras for real-time 3D hand gesture interaction. In: 2011 IEEE Workshop on Applications of Computer Vision (WACV), pp. 66–72 (2011). doi:10.1109/WACV.2011.5711485
Cerlinca, T.I., Pentiuc, S.G.: Robust 3D hand detection for gestures recognition, pp. 259–264. Springer, Heidelberg (2012)
Chen, C.P., Chen, Y.T., Lee, P.H., Tsai, Y.P., Lei, S.: Real-time hand tracking on depth images. In: 2011 Visual Communications and Image Processing (VCIP), pp. 1–4 (2011). doi:10.1109/VCIP.2011.6115983
Gonzalez, R.C., Woods, R.E.: Digital Image Processing, 3rd edn. Prentice-Hall Inc., Upper Saddle River (2006)
Jambhulkar, K.R.: Review on sensor based hand gesture recognition system. Int. J. Res. Eng. Adv. Technol. 5(1), 33–36 (2017)
Microsoft: Kinect. https://developer.microsoft.com/pt-pt/windows/kinect. Accessed April 2017
Openni: Nite. http://openni.ru/files/nite/. Accessed April 2017
Park, S., Yu, S., Kim, J., Kim, S., Lee, S.: 3D hand tracking using Kalman filter in depth space. EURASIP J. Adv. Signal Process. 2012(1), 36 (2012)
Ramey, A., Gonzalez-Pacheco, V., Salichs, M.A.: Integration of a low-cost RGB-D sensor in a social robot for gesture recognition. In: 2011 6th ACM/IEEE International Conference on Human-Robot Interaction (HRI), pp. 229–230 (2011). doi:10.1145/1957656.1957745
Rautaray, S.S., Agrawal, A.: Vision based hand gesture recognition for human computer interaction: a survey. Artif. Intell. Rev. 43(1), 1–54 (2015)
Yang, C., Jang, Y., Beh, J., Han, D., Ko, H.: Gesture recognition using depth-based hand tracking for contactless controller application. In: 2012 IEEE International Conference on Consumer Electronics (ICCE), pp. 297–298 (2012). doi:10.1109/ICCE.2012.6161876
Acknowledgements
This work is supported in part by the Faculty of Engineering of University of Porto in collaboration with the company Follow Inspiration S.A.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG
About this paper
Cite this paper
de Sousa, P. et al. (2018). Human-Robot Interaction Based on Gestures for Service Robots. In: Tavares, J., Natal Jorge, R. (eds) VipIMAGE 2017. ECCOMAS 2017. Lecture Notes in Computational Vision and Biomechanics, vol 27. Springer, Cham. https://doi.org/10.1007/978-3-319-68195-5_76
Download citation
DOI: https://doi.org/10.1007/978-3-319-68195-5_76
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-68194-8
Online ISBN: 978-3-319-68195-5
eBook Packages: EngineeringEngineering (R0)