SpringerLink
Log in

Visual servoing control of 4-DOF palletizing robotic arm for vision based sorting robot system

  • Original Paper
  • Published:
International Journal on Interactive Design and Manufacturing (IJIDeM) Aims and scope Submit manuscript

Abstract

In this paper, an image-based visual servoing is integrated into a vision-based sorting robot system to increase the flexibility and performance of the system. A novel decoupled image-based visual servoing is proposed to control a 4-DOF robot arm for gras** products. Area, orientation angle, and centroid features extracted from the image are used as input to control the velocities of the robot arm. A multi-threshold algorithm is presented to detect and classify objects and extract the image features. Due to its simplicity but efficiency, image processing and visual servoing algorithms can be implemented in real-time on a low-cost embedded computer. Furthermore, the system is also easy to set up because the robot and camera calibrations are not required. Experiment results reveal the time effectiveness and performance robustness of the system. The system can be used in industrial processes to reduce the required time and improve the performance and flexibility of the production line.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Abbreviations

VS:

visual servoing

IBVS:

Image-based visual servoing

PBVS:

Position-based visual servoing

SPVS:

Shortest path visual servoing

DOF:

degree of freedom

ROI:

Region of interest

References

  1. Cong, V.D.: Industrial robot arm controller based on programmable system-on-chip device. FME Trans. 49(4), 1025–1034 (2021)

    Article  Google Scholar 

  2. Cong, V.D., Hanh, L.D., Phuong, L.H., Duy, D.A.: Design and development of robot arm system for classification and sorting using machine vision. FME Trans. 50(1), 181–192 (2022)

    Article  Google Scholar 

  3. Ghasemi, A., Li, P., **e, W.: Adaptive switch image-based visual servoing for industrial robots. Int. J. Control Autom. Syst. 18, 1324–1334 (2020)

    Article  Google Scholar 

  4. Ren, X., Li, H., Li, Y.: Image-based visual servoing control of robot manipulators using hybrid algorithm with feature constraints. IEEE Access 8, 223495–223508 (2020)

    Article  Google Scholar 

  5. Cong, V.D., Hanh, L.D.: A new decoupled control law for image-based visual servoing control of robot manipulators. Int. J. Intell. Robotics Appl. 5, 999 (2022)

    Google Scholar 

  6. Cong, V.D., Hanh, L.D.: Evaluate control laws related to interaction matrix for image-based visual servoing. In: 2019 6th NAFOSTED Conference on Information and Computer Science (NICS), pp. 454–459, (2019)

  7. Wu, H., Andersen, T.T., Andersen, N.A., Ravn, O.: Application of visual servoing for gras** and placing operation in slaughterhouse. In: 3rd International Conference on Control, Automation and Robotics (ICCAR), pp. 457–462, (2017)

  8. Hanh, L.D., Lin, C.Y.: Autonomous gluing based on image-based visual servoing. In: International Journal of Robotics and Automation, vol. 36, no. 2, (2021)

  9. Traslosheros, A., Sebastián, J.M., Ángel, L., Roberti, F., Carelli, R.: Visual servoing for the Robotenis system: A strategy for a 3 DOF parallel robot to hit a **-Pong ball. In: 2011 50th IEEE Conference on Decision and Control and European Control Conference, pp. 5695–5701, (2011)

  10. Dewi, T., Risma, P., Yurni, O., Muslimin, S.M.: Visual Servoing Design and Control for Agriculture Robot; a Review. In: 2018 International Conference on Electrical Engineering and Computer Science (ICECOS), pp. 57–62, (2018)

  11. Chen, W., Xu, T., Liu, J., Wang, M., Zhao, D.: Picking robot visual servo control based on modified fuzzy neural network sliding mode algorithms. Electronics 8(6), 5555 (2019)

    Article  Google Scholar 

  12. Liang, X., Peng, M., Lu, J., Qin, C.: A visual servo control method for tomato cluster-picking manipulators based on a T-S fuzzy neural network. Trans. ASABE 64(2), 529–543 (2021)

    Article  Google Scholar 

  13. Mehta, S.S., Mackunis, W., Burks, T.F.: Robust Visual Servo Control in The Presence of Fruit Motion for Robotic Citrus Harvesting. Comput. Electron. Agric. 123, 362–375 (2016)

    Article  Google Scholar 

  14. Bulanon, D.M., Burr, C., DeVlieg, M., Braddock, T., Allen, B.: Development of a visual servo system for robotic fruit harvesting. AgriEngineering 3(4), 840–852 (2021)

    Article  Google Scholar 

  15. Li, C.L., Cheng, M.Y., Chang, W.C.: Dynamic performance improvement of direct image-based visual servoing in contour following. Int. J. Adv. Robot. Syst. 15(1), 558 (2018)

    Article  Google Scholar 

  16. Cong, V.D., Hanh, L.D.: Combination of two visual servoing techniques in contour following task. In: 2021 International Conference on System Science and Engineering (ICSSE), pp. 382–386, (2021)

  17. Chang, W.C., Cheng, M.Y., Tsai, H.J.: Image feature command generation of contour following tasks for SCARA robots employing Image-Based Visual Servoing-A PH-spline approach. Robotics Comput.-Integr. Manuf. 44, 53339 (2017)

    Article  Google Scholar 

  18. Cheng, C.C., Tsai, C.H.: Visual servo control for balancing a ball-plate system. Int. J. Mech. Eng. Robot. Res. 5(1), 28–32 (2016)

    Google Scholar 

  19. Shih, C., Hsu, J., Chang, C.: Visual feedback balance control of a robot manipulator and ball-beam system. J. Comput. Commun. 5, 8–18 (2017)

  20. Brill, A., Frank, J., Kapola, V.: Using inertial and visual sensing from a mounted smartphone to stabilize a ball and beam test-bed. In: Proceedings of American Control Conference, pp. 1335–1340, (2016)

  21. Malis, E., Chaumette, F., Boudet, S.: 2 1/2 d visual servoing. IEEE Trans. Robot. Autom. 15(2), 238–250 (1999)

    Article  Google Scholar 

  22. Kyrki, V., Kragic, D., Christensen, H.: New shortest-path approaches to visual servoing. In: 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 1, pp. 349–354, (2004)

  23. Corke, P., Hutchinson, S.: A new partitioned approach to image-based visual servo control. IEEE Trans. Robot. Autom. 17(4), 507–515 (2001)

    Article  Google Scholar 

  24. Cao, Q., Qingge, L., Yang, P.: Performance analysis of otsu-based thresholding algorithms: a comparative study. J. Sens. 5, 666 (2021)

    Google Scholar 

  25. Cao, X., et al.: A robust parameter-free thresholding method for image segmentation. IEEE Access 7, 3448–3458 (2019)

    Article  Google Scholar 

  26. Mukundan, R., Ramakrishman, K.R.: Moment Functions in Image Analysis: Theory and Application. World Scientific, Singapore (1998)

    Google Scholar 

  27. Chaumette, F., Hutchinson, S.: Visual servo control. I. Basic approaches. Robot. Autom. Mag. IEEE 13(4), 82–90 (2020)

    Article  Google Scholar 

  28. Grabner, H., Grabner, M., Bischof, H.: Real-time tracking via on-line boosting. In: Proceedings of British Machine Vision Conference (BMVC), pp. 47–56, (2006)

  29. Babenko, B., Yang, M.H., Belongie, S.: Robust object tracking with online multiple instance learning. IEEE Trans. Pattern Anal. Mach. Intell. 33(8), 1619–1632 (2011)

    Article  Google Scholar 

Download references

Acknowledgements

We acknowledge Ho Chi Minh City University of Technology (HCMUT), VNU-HCM for supporting this study.

Author information

Authors and Affiliations

  1. Industrial Maintenance Training Center, Ho Chi Minh City University of Technology (HCMUT), 268 Ly Thuong Kiet Street, District 10, Ho Chi Minh City, Vietnam

    Vo Duy Cong

  2. Vietnam National University Ho Chi Minh City, Linh Trung Ward, Thu Duc City, Ho Chi Minh City, Vietnam

    Vo Duy Cong

Authors
  1. Vo Duy Cong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vo Duy Cong.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cong, V.D. Visual servoing control of 4-DOF palletizing robotic arm for vision based sorting robot system. Int J Interact Des Manuf 17, 717–728 (2023). https://doi.org/10.1007/s12008-022-01077-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12008-022-01077-8

Keywords

Search

Navigation