SpringerLink
Log in

Intelligent communication of two humanoid robots based on computer vison

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

The focus of this paper is on investigating the use of visual communication between two humanoid robots, in order to enhance the coordination of tasks between them. The problem continues to be an interesting and fruitful area of research from the days of using multiple robot manipulator arms for manufacturing as well as space robotics to current research in medical robotics. The approach here is to employ several off-the-shelf algorithms, software and hardware such as the NAO robot and support software, including Choregraphe, OpenCV to capture and process images, the SVM to classify objects in images, and the Python programming environment. Five robotic actions were studied and three modes. The experiments used one robot as the “viewer” and the second robot as the “subject” being analyzed. Results show that the visual communication system has an accuracy of 90% in correctly identifying the five movements. This research has shown an original solution, as a model that can enable robots to run in the complex service tasks consisting of multiple connected actions in a dynamic environment. This methodology can also let the intelligent operation of the robots serve in different scenes according to their actual requirements. This research focuses on enhancing the prototype robot vision function and development of additional value for consolidation manageable platform that increases service robots in the home environment of intelligent control capability.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Germany)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Data availability

Data sharing not applicable to this article as no datasets were generated or analyzed during the current study.

References

  1. Aldana-Murillo NG, Hayet JB, Becerra H (2018) Comparison of Local Descriptors for Humanoid Robots Localization Using a Visual Bag of Words Approach. Intell Autom Soft Comput 24(3):471–481

  2. Yan Lu, Song D (2017) Visual Navigation Using Heterogeneous Landmarks and Unsupervised Geometric Constraints.". IEEE Trans Robotics 31(3):736–749

    Google Scholar 

  3. Porzi L, Buló SR, Penate-Sanchez A, Ricci E (2017) Learning Depth-Aware Deep Representations for Robotic Perception. IEEE Robotics Autom Lett 2(2):468–475

    Article  Google Scholar 

  4. Becerra H (2014) Fuzzy Visual Control for Memory-Based Navigation Using the Trifocal Tensor. Intell Autom Soft Comput 20(2):245–262

    Article  Google Scholar 

  5. Chen MY (2018) The SLAM Algorithm for Multiple Robots Based on Parameter Estimation. Intell Autom Soft Comput 24(3):593–607

    Article  Google Scholar 

  6. Chou YC, Nakajima M (2016) Particle Filter Planar Target Tracking with a Monocular Camera for Mobile Robots. Intell Autom Soft Comput 24(3):117–125

    Google Scholar 

  7. Mukai T, Onishi M, Odashima T, Hirano S, Luo Z (Apr.2008) Development of the tactile sensor system of a human-interactive robot ‘RI-MAN. IEEE Trans Robot 24(2):505–512

    Article  Google Scholar 

  8. Takeda T, Hirata Y, Kosuge K (2007) Dance partner robot cooperative motion generation with adjustable length of dance step stride based on physical interaction, in Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst., pp. 3258–3263

  9. NAO-technical overview. http://doc.aldebaran.com/2-1/family/robots/dimensions_robot.html/22/10/2018. Accessed 22/10/2018

  10. Mohaimin Sultan Mohammad, Saha Sajib Kumar, Khan AlveMahamud, Arif Abu Shamim Mohammad, Kanagasingam Yogesan (2018) Automated method for the detection and segmentation of drusen in colour fundus image for the diagnosis of age-related macular degeneration. IET Image Process 12(6):919–927

    Article  Google Scholar 

  11. Jenkins M, Kantor G (2017) Online detection of occluded plant stalks for manipulation. 5162 – 5167

  12. Le P-H, Košecka J (2017) Dense piecewise planar RGB-D SLAM for indoor environments. 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS),4944 – 4949

  13. Kim W, Johnson-Roberson M (2017) A probabilistic framework for intrinsic image decomposition from RGB-D streams. 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS),6498 – 6505

  14. Dixit Rahul, Naskar Ruchira, Mishra Swati (2017) Blur-invariant copy-move forgery detection technique with improved detection accuracy utilising SWT-SVD. IET Image Process 11(5):301–309

    Article  Google Scholar 

  15. Zhou P, Han X, Morariu VI, Davis LS (2018) Learning Rich Features for Image Manipulation Detection. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 1053 – 1061.

  16. Rehman Yawar, Khan Jameel Ahmed, Shin Hyunchul (2018) Efficient coarser-to-fine holistic traffic sign detection for occlusion handling. IET Image Process 12(12):2229–2237

    Article  Google Scholar 

  17. Spexard T, Hanheide M, Sagerer G (Oct.2007) Human-oriented interaction with an anthropomorphic robot. IEEE Trans Robot 23(5):852–862

    Article  Google Scholar 

  18. Nigam A, Penate-Sanchez A, Agapito L (2018) Detect Globally, Label Locally: Learning Accurate 6-DOF Object Pose Estimation by Joint Segmentation and Coordinate Regression. volume: 3, Issue: 4, 3960 – 3967

  19. George L, and Mazel A (2013) Humanoid robot indoor navigation based on 2D bar codes: application to the NAO robot. Ieee-Ras International Conference on Humanoid Robots IEEE, 329–335

  20. Yoder JD and Seelinger M (2006) Visual Coordination of Heterogeneous Mobile Manipulators, Springer Tracts in Advanced Robotics: Proc of the 9th International Symposium on Experimental Robotics, vol. 21, no. 1, pp. 387–398

  21. Fleurmond R, Cadenat V (2014) Multi-Cameras Visual Servoing to Perform a Coordinated Task Using a Dual Arm Robot, Proc. of the 11th International Conference on Informatics in Control. Autom Robotics 2(2):37–44

  22. Chen ML, Kume S, Rizzi AA, Hollis RL (2000) Visually Guided Coordination for Distributed Precision Assembly. Proc IEEE Int Conf Robotics Autom 2:1651–1656

    Google Scholar 

  23. Kumar S, Kumar DK, Sharma A, McLachlan N (2004) “Classification of Visual Hand Movements Using Multiresolution Wavelet Images. Proc Int Conf Intell Sens Inform Process:373–378

  24. Rababaah H, Haroun, Shirkhodaie, A (2008) Human-Posture Classification for Intelligent Visual Surveillance Systems”, Proc. of SPIE - The International Society for Optical Engineering: Special Sessions on Food Safety, Visual Analytics, Resource-Restricted Embedded and Sensor Networks, and 3D Imaging and Display, vol. 6983, Article 69830F

  25. Amine Elforaici MEl, Chaaraoui I, Bouachir Y, Ouakrim W, Mezghani N (2018) Posture Recognition using an RGB-D Camera: Exploring 3D Body Modeling and Deep Learning Approaches, Proc. of the IEEE Life Sciences Conference, pp. 69–72

  26. Shahverdi P, Masouleh MT (2016) A Simple and Fast Geometric Kinematic Solution for Imitation of Human Arms by a NAO Humanoid Robot, Proceedings of the 4th International Conference on Robotics and Mechatronics, 26–28

  27. Shamsuddin S, Yussof H et al (2012) Humanoid Robot NAO Interacting with Autistic Children of Moderately Impaired Intelligence to Augment Communication Skills. Procedia Eng 41:1533–1538

    Article  Google Scholar 

  28. Yu YH, Ting YS, Mayer NM, et al. (2017) A new paradigm of ubiquitous home care robot using Nao and Choregraphe, International Conference on Advanced Robotics and Intelligent Systems. IEEE

  29. Becerra HM, Sagues C (2014) Visual Control of Wheeled Mobiles Robots, Springer Tracts in Advanced Robotics 103

  30. Kragic D, Miller AT, Allen PK (2001) Realtime tracking meets online grasp planning, IEEE International Conference on Robotics & Automation. IEEE

  31. Adachi J, Sato J (2010) Uncalibrated Visual Servoing from Projective Reconstruction of Control Values [J]. Electron Commun Japan 89(10):47–56

    Google Scholar 

  32. Sato T, Sato J (2015) Visual servoing from uncalibrated cameras for uncalibrated robots. Syst Comput Japan 31(14):11–19

    Article  Google Scholar 

  33. Kumar M, Kumar **dal M, Kumar M (2021) A Novel Attack on Monochrome and Greyscale Devanagari CAPTCHAs, ACM Transactions on Asian and Low-Resource Language Information Processing vo. 20, Issue 4Article no.: 56,pp. 1–30

  34. Kumar M, **dal MK, Kumar Munish (2023) An efficient technique for breaking of coloured Hindi CAPTCHA. Soft Comput 27:11661–11686

    Article  Google Scholar 

Download references

Acknowledgements

The author deeply acknowledges Mr. Zhang，Yi-**ang  initial test support at first rough model.

Author information

Authors and Affiliations

  1. School of Artificial Intelligence/School of Future Technology, Nan**g University of Information Science and Technology, Nan**g, 210044, China

    Li-Hong Juang

Authors
  1. Li-Hong Juang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Li-Hong Juang.

Ethics declarations

Conflict of interest

The authors declare that they have no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (MP4 19007 KB)

Supplementary file2 (MP4 25297 KB)

Supplementary file3 (MP4 23235 KB)

Supplementary file4 (MP4 17289 KB)

Supplementary file5 (MP4 14975 KB)

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Juang, LH. Intelligent communication of two humanoid robots based on computer vison. Multimed Tools Appl 83, 63459–63477 (2024). https://doi.org/10.1007/s11042-023-17989-w

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-17989-w

Keywords

Search

Navigation