Abstract
Recent advances in computer vision and deep learning have lead to implementations in different industrial applications such as collaborative robotics, making robots able to perform harder tasks and giving them consciousness of their environment, easing interaction with humans. With the objective of eliminating physical barriers between humans and robots, a security system for industrial collaborative robots based on computer vision and deep learning is proposed, where an RGBD camera is used to detect and track people located inside the robot’s workspace. Detection is made with a previously trained convolutional neural network. The position of every detection is fed to the tracker, that identifies the subjects in scene and keeps record of them in case the detector fails. The detected subject’s 3D position and height are represented in a simulation of the workspace, where the robot’s speed changes depending on its distance to the manipulator following international safety guidelines. This paper shows the implementation of the detector and tracker algorithms, the subject’s 3D position, the security zones definition and the integration of the vision system with the robot and workspace. Results show the system’s ability to detect and track subjects in scene, and the robot’s capacity to change its speed depending on the subject’s location.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Halme, R.J., Lanz, M., Kämäräinen, J., Pieters, R., Latokartano, J., Hietanen, A.: Review of vision-based safety systems for human-robot collaboration. Procedia CIRP 72, 111–116 (2018). https://doi.org/10.1016/j.procir.2018.03.043
Ahmad, R., Plapper, P.: Human-Robot collaboration: twofold strategy algorithm to avoid collisions using ToF sensor. Int. J. Mater. Mech. Manuf. 4(2), 144–147 (2015). https://doi.org/10.7763/ijmmm.2016.v4.243
Lešo, M., Žilková, J., Vacek, M.: Robotic manipulator with optical safety system. In: International Conference on Electrical Drives and Power Electronics (EDPE), vol. 2015, pp. 389–393 (2015). https://doi.org/10.1109/EDPE.2015.7325326
Mohammed, A., Schmidt, B., Wang, L.: Active collision avoidance for human-robot collaboration driven by vision sensors. Int. J. Comput. Integrated Manuf. 30(9), 970–980 (2016). https://doi.org/10.1080/0951192x.2016.1268269
Morato, C., Kaipa, K.N., Zhao, B., Gupta, S.K.: Toward safe human robot collaboration by using multiple kinects based real-time human tracking. J. Comput. Inf. Sci. Eng. 14(1) (2014). https://doi.org/10.1115/1.4025810
Rybski, P., Anderson-Sprecher, P., Huber, D., Niessl, C., Simmons, R.: Sensor fusion for human safety in industrial workcells. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, vol. 2012, pp. 3612–3619 (2012). https://doi.org/10.1109/IROS.2012.6386034
Medina, A.C., Mora, J.F., Martinez, C., Barrero, N., Hernandez, W.: Safety protocol for collaborative human-robot recycling tasks. IFAC-Papers OnLine 52(13), 2008–2013 (2019). https://doi.org/10.1016/j.ifacol.2019.11.498
Info PLC: La seguridad en la colaboración inteligente hombre-robot, 27 February 2017. https://www.infoplc.net/plus-plus/tecnologia/item/104052-seguridad-colaboracion-inteligente-hombre-robot
Beaupre, M.: Collaborative Robot Technology and Applications [Slides] (2014). https://www.robotics.org/userAssets/riaUploads/file/4-KUKA_Beaupre.pdf
Al-Naji, A.A., Gibson, K., Lee, S.-H., Chahl, J.: Real time Apnoea monitoring of children using the microsoft kinect sensor: a pilot study. Sensors 17, 286 (2017). https://doi.org/10.3390/s17020286
Universal Robots: e-Series DE UNIVERSAL ROBOTS (2018). https://www.universal-robots.com/cb3/
Stewart, R., Andriluka, M., Ng, A.: End-to-End people detection in crowded scenes. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016). https://doi.org/10.1109/cvpr.2016.255
Computer Vision Annotation Tool (CVAT): Software (2020). https://cvat.org
Mallick, S.: Object Tracking using OpenCV (C++/Python), 13 February 2017. https://www.learnopencv.com/object-tracking-using-opencv-cpp-python/
Bewley, A., Ge, Z., Ott, L., Ramos, F., Upcroft, B.: Simple online and realtime tracking. In: 2016 IEEE International Conference on Image Processing (ICIP) (2016). https://doi.org/10.1109/icip.2016.7533003
Mora, J.F., Medina, A.C., Ramírez, E., Mártinez, C.V.: Human Recognition Algorithm for Industrial Collaborative Robots in Automated Waste Separations Tasks (Thesis) (2018)
(n.d). In ROS.org. tf, Static Transform Publisher - ROS Wiki (2020). http://wiki.ros.org/Parameter%20Server
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Duque-Suárez, N., Amaya-Mejía, L.M., Martinez, C., Jaramillo-Ramirez, D. (2022). Deep Learning for Safe Human-Robot Collaboration. In: Moreno, H.A., Carrera, I.G., Ramírez-Mendoza, R.A., Baca, J., Banfield, I.A. (eds) Advances in Automation and Robotics Research. LACAR 2021. Lecture Notes in Networks and Systems, vol 347. Springer, Cham. https://doi.org/10.1007/978-3-030-90033-5_26
Download citation
DOI: https://doi.org/10.1007/978-3-030-90033-5_26
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-90032-8
Online ISBN: 978-3-030-90033-5
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)