SpringerLink
Log in

A Human Digital Twin Based Framework for Human–Robot Hand-Over Task Intention Recognition

  • Conference paper
  • First Online:
Advances in Mechanical Design (ICMD 2023)

Part of the book series: Mechanisms and Machine Science ((Mechan. Machine Science,volume 155))

Included in the following conference series:

  • 90 Accesses

Abstract

Industry 5.0, the upcoming industrial revolution, places a strong emphasis on human-centric smart manufacturing, redefining the state and role of humans in the process of human–robot collaboration (HRC). The human–robot hand-over task plays a crucial role of HRC, and finding ways to enable robots to understand human handover intentions is an urgent problem to be solved. In this study, a human digital twin-based framework for human–robot hand-over task intention recognition is proposed to enhance the execution efficiency of human–robot hand-over tasks. This framework aims to foster a seamless integration and cooperation between humans and robots, facilitating deep human–robot fusion. Then, considering the multi-scale characteristics and temporal correlation of human intention information, a feature extractor based on multi-scale convolutional neural network and bidirectional long-short-term memory (MSCNN-BiLSTM) is devised to improve the cognitive and collaboration abilities of robots. A series of optimization experiments are conducted to enhance the performance of the MSCNN-BiLSTM model. The superiority of the proposed framework is demonstrated by comparing with traditional human–robot hand-over intention recognition algorithms, which provides an approach to achieve better human–robot fusion and more efficient execution of hand-over tasks.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (Brazil)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 299.00
Price excludes VAT (Brazil)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 379.99
Price excludes VAT (Brazil)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free ship** worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Leng, J., Sha, W., Wang, B., Zheng, P., Zhuang, C., Liu, Q., Wuest, T., Mourtzis, D., Wang, L.: Industry 5.0: prospect and retrospect. J. Manuf. Syst. 65, 279–295 (2022)

    Google Scholar 

  2. Zhong, R., Hu, B., Hong, Z., Zhang, Z., Lou, S., Song, X., Feng, Y., Tan, J.: Human-robot handover task intention recognition framework by fusing human digital twin and deep domain adaptation. J. Eng. Des., 1–17 (2024)

    Article  Google Scholar 

  3. Lee, M.-L., Liu, W., Behdad, S., Liang, X., Zheng, M.: Robot-assisted disassembly sequence planning with real-time human motion prediction. IEEE Trans. Syst. Man Cybern. Syst. 53(1), 438–450 (2022)

    Article  Google Scholar 

  4. Zhong, R., Hu, B., Feng, Y., Zheng, H., Hong, Z., Lou, S., Tan, J.: Construction of human digital twin model based on multimodal data and its application in locomotion mode identification. Chin. J. Mech. Eng 36(1), 11 (2023)

    Article  Google Scholar 

  5. LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436–444 (2015)

    Article  Google Scholar 

  6. Liu, S., Wang, L., Vincent Wang, X.: Multimodal data-driven robot control for human-robot collaborative assembly. J. Manuf. Sci. Eng. 144(5), 051012 (2022)

    Article  Google Scholar 

  7. Zhang, X., Fan, J., Peng, T., Zheng, P., K. M. Lee, C., Tang, R.: A privacy-preserving and unobtrusive sitting posture recognition system via pressure array sensor and infrared array sensor for office workers. Adv. Eng. Inf. 53, 101690 (2022)

    Google Scholar 

  8. El Makrini, I., Mathijssen, G., Verhaegen, S., Verstraten, T., Vanderborght, B.: A virtual element-based postural optimization method for improved ergonomics during human-robot collaboration. IEEE Trans. Autom. Sci. Eng. 19(3), 1772–1783 (2022)

    Article  Google Scholar 

  9. Zhang, Z., Peng, G., Wang, W., Chen, Y., Jia, Y., Liu, S.: Prediction-based human-robot collaboration in assembly tasks using a learning from demonstration model. Sensors 22(11), 4279 (2022)

    Article  Google Scholar 

  10. Wang, W., Li, R., Chen, Y., Sun, Y., Jia, Y.: Predicting human intentions in human-robot hand-over tasks through multimodal learning. IEEE Trans. Autom. Sci. Eng. 19(3), 2339–2353 (2021)

    Article  Google Scholar 

  11. Liu, R., Liu, C.: Human motion prediction using adaptable recurrent neural networks and inverse kinematics. IEEE Control Syst. Lett. 5(5), 1651–1656 (2021)

    Article  Google Scholar 

  12. Zhang, J., Wang, P., Gao, R.X.: Hybrid machine learning for human action recognition and prediction in assembly. Robot. Comput. Integr. Manuf. 72, 102184 (2021)

    Article  Google Scholar 

  13. **ong, D., Zhang, D., Zhao, X., Zhao, Y.: Deep learning for EMG-based human-machine interaction: a review. IEEE/CAA J. Automatica Sinica 8(3), 512–533 (2021)

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by the National Key Research and Development Program of China (No. 2022YFB3402000), National Natural Science Foundation of China (No. 52205288, No. 52075479, and No. 52105281), and Key Research & Development Prgram of Zhejiang Province (No. 2021C01110)

Author information

Authors and Affiliations

  1. State Key Laboratory of Fluid Power and Mechatronic Systems, Zhejiang University, Hangzhou, 310027, China

    Ruirui Zhong, Bingtao Hu, Zhaoxi Hong, Zhifeng Zhang, Yixiong Feng & Jianrong Tan

  2. Engineering Research Center for Design Engineering and Digital Twin of Zhejiang Province, Hangzhou, 310027, China

    Ruirui Zhong, Bingtao Hu, Zhaoxi Hong, Zhifeng Zhang, Yixiong Feng & Jianrong Tan

Authors
  1. Ruirui Zhong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bingtao Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhaoxi Hong
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhifeng Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yixiong Feng
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jianrong Tan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bingtao Hu .

Editor information

Editors and Affiliations

  1. School of Mechanical Engineering, Zhejiang University, Hangzhou, Zhejiang, China

    Jianrong Tan

  2. School of Mechanical and Electrical Engineering, University of Electronic Science and Technology of China, Chengdu, Sichuan, China

    Yu Liu

  3. School of Mechanical and Electrical Engineering, University of Electronic Science and Technology of China, Chengdu, Sichuan, China

    Hong-Zhong Huang

  4. School of Mechanical Engineering and Automation, Beihang University, Bei**g, China

    **gjun Yu

  5. University of Electronic Science and Technology of Chin, Chengdu, Sichuan, China

    Zequn Wang

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zhong, R., Hu, B., Hong, Z., Zhang, Z., Feng, Y., Tan, J. (2024). A Human Digital Twin Based Framework for Human–Robot Hand-Over Task Intention Recognition. In: Tan, J., Liu, Y., Huang, HZ., Yu, J., Wang, Z. (eds) Advances in Mechanical Design. ICMD 2023. Mechanisms and Machine Science, vol 155. Springer, Singapore. https://doi.org/10.1007/978-981-97-0922-9_18

Download citation

  • DOI: https://doi.org/10.1007/978-981-97-0922-9_18

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-97-0921-2

  • Online ISBN: 978-981-97-0922-9

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation