SpringerLink
Log in

Augmenting Human-Machine Teaming Through Industrial AR: Trends and Challenges

  • Conference paper
  • First Online:
Systems Collaboration and Integration (ICPR1 2021)

Part of the book series: Automation, Collaboration, & E-Services ((ACES,volume 14))

Included in the following conference series:

  • 180 Accesses

Abstract

Industrial Augmented Reality (AR) is an emerging spatial computing technology which involves the use of head-mounted displays or hand-held devices such as tablets or smartphones to superimpose digital content onto the worker’s physical to foster their productivity, learning, and interactions with machines, tools, and other workers. Industrial AR has been adopted in many industries such as manufacturing, healthcare, aerospace, and defense, predominantly for training or remote assistance purposes. Yet, several technical and technological challenges remain to be addressed for industrial AR to evolve from a spatial visualization tool to a more intelligent and adaptive assistive tool that not only augments the spatial and causal reasoning of workers but can also provide them with just-in-time training and support on the job. This chapter provides some technical background on industrial AR and underscores several research and development directions which can potentially materialize this vision.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 189.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 249.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free ship** worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Brynjolfsson, E., McAfee, A.: The Second Machine Age: Work, Progress, and Prosperity in a Time of Brilliant Technologies. W W Norton & Co. (2014). Accessed 10 Mar 2021. https://psycnet.apa.org/record/2014-07087-000

  2. NAM, “NAM Manufacturers’ Outlook Survey (2020). https://www.nam.org/wp-content/uploads/2020/09/NAM-Outlook-Survey-Q3-2020.pdf. Accessed 17 Mar 2021

  3. Houseman, S.N.: Understanding the Decline of U.S. Manufacturing Employment. Upjohn Institute Working Paper 18–287, Kalamazoo, MI (2018). https://doi.org/10.17848/wp18-287

  4. Lund, S., et al.: Risk, Resilience, and Rebalancing in Global Value Chains. McKinsey & Company (2020). https://www.mckinsey.com/business-functions/operations/our-insights/risk-resilience-and-rebalancing-in-global-value-chains. Accessed 17 Mar 2021

  5. SHRM, “Preparing for an Aging Workforce: Manufacturing Industry Report,” Society for Human Resource Management (2015)

    Google Scholar 

  6. NAP, Building America’s Skilled Technical Workforce (National Academies of Sciences, Engineering, and Medicine). National Academies Press (2017). https://doi.org/10.17226/23472

  7. Giffi, C., Wellener, P., Dollar, B., Ashton Manolian, H., Monck, L., Moutray, C.: Deloitte and The Manufacturing Institute Skills Gap and Future of Work Study (2018). http://www.themanufacturinginstitute.org/~/media/E323C4D8F75A470E8C96D7A07F0A14FB/DI_2018_Deloitte_MFI_skills_gap_FoW_study.pdf

  8. PTC, “Closing the Industrial Skills Gap with Industrial Augmented Reality,” PTC (2019). https://www.ptc.com/en/resources/augmented-reality/ebook/closing-the-industrial-skills-gap-with-augmented-reality. Accessed 19 Feb 2020

  9. Conway, S.: The Total Economic Impact TM Of PTC Vuforia: Cost Savings And Business Benefits Enabled By Industrial Augmented Reality (2019). https://www.ptc.com/en/resources/augmented-reality/report/forrester-total-economic-impact

  10. Mizell, D.: Boeing’s wire bundle assembly project. In: Barfield, W., Caudell, T. (eds.), Fundamentals of Wearable Computers and Augmented Reality. CRC Press, pp. 447–467 (2001)

    Google Scholar 

  11. Schwald, B., et al.: STARMATE: using augmented reality technology for computer guided maintenance of complex mechanical elements. In: Smith, B., Chiozza, E. (eds.), E-work and E-commerce-Novel Solutions and Practices for a Global Networked Economy, pp. 196–202 (2001)

    Google Scholar 

  12. Schwald, B., Laval, B.D.: An augmented reality system for training and assistance to maintenance in the industrial context. In: International Conference in Central Europe on Computer Graphics, Visualization and Computer Vision (WSCG), pp. 425–432 (2003)

    Google Scholar 

  13. Webel, S., Bockholt, U., Engelke, T., Gavish, N., Olbrich, M., Preusche, C.: An augmented reality training platform for assembly and maintenance skills. Robot. Auton. Syst. 61(4), 398–403 (2013). https://doi.org/10.1016/j.robot.2012.09.013

    Article  Google Scholar 

  14. Syberfeldt, A., Danielsson, O., Holm, M., Wang, L.: Visual assembling guidance using augmented reality. Procedia Manufacturing 1, 98–109 (2015). https://doi.org/10.1016/j.promfg.2015.09.068

    Article  Google Scholar 

  15. Holm, M., Danielsson, O., Syberfeldt, A., Moore, P., Wang, L.: Adaptive instructions to novice shop-floor operators using augmented reality. J. Ind. Prod. Eng. 34(5), 362–374 (2017). https://doi.org/10.1080/21681015.2017.1320592

    Article  Google Scholar 

  16. Marr, B.: The Amazing Ways Honeywell Is Using Virtual And Augmented Reality To Transfer Skills To Millennials. Forbes (2018). https://www.forbes.com/sites/bernardmarr/2018/03/07/the-amazing-ways-honeywell-is-using-virtual-and-augmented-reality-to-transfer-skills-to-millennials/#586ec524536a. Accessed 19 Feb 2020

  17. Polladino, T.: Porsche Adopts Atheer’s AR Platform to Connect Mechanics with Remote Experts « Next Reality. Next Reality (2017). https://next.reality.news/news/porsche-adopts-atheers-ar-platform-connect-mechanics-with-remote-experts-0181255/. Accessed 19 Feb 2020

  18. O’Donnell, R.: How Mercedes-Benz uses augmented reality to train employees of all types. HRDive (2018). https://www.hrdive.com/news/how-mercedes-benz-uses-augmented-reality-to-train-employees-of-all-types/530425/. Accessed 19 Feb 2020

  19. Vanneste, P., Huang, Y., Park, J.Y., Cornillie, F., Decloedt, B., van den Noortgate, W.: Cognitive support for assembly operations by means of augmented reality: an exploratory study. International Journal of Human Computer Studies 143(October 2019), 102480 (2020). https://doi.org/10.1016/j.ijhcs.2020.102480

  20. Lai, Z.H., Tao, W., Leu, M.C., Yin, Z.: Smart augmented reality instructional system for mechanical assembly towards worker-centered intelligent manufacturing. J. Manuf. Syst. 55(February), 69–81 (2020). https://doi.org/10.1016/j.jmsy.2020.02.010

    Article  Google Scholar 

  21. Arbeláez, J.C., Viganò, R., Osorio-Gómez, G.: Haptic augmented reality (HapticAR) for assembly guidance. Int. J. Interact. Des. Manuf. 13(2), 673–687 (2019). https://doi.org/10.1007/s12008-019-00532-3

    Article  Google Scholar 

  22. Erkoyuncu, J.A., del Amo, I.F., Dalle Mura, M., Roy, R., Dini, G.: Improving efficiency of industrial maintenance with context aware adaptive authoring in augmented reality. CIRP Annals - Manufacturing Technol. 66(1), 465–468 (2017). https://doi.org/10.1016/j.cirp.2017.04.006

  23. Siew, C.Y., Ong, S.K., Nee, A.Y.C.: A practical augmented reality-assisted maintenance system framework for adaptive user support. Robotics and Computer-Integrated Manufacturing 59, 115–129 (2019). https://doi.org/10.1016/j.rcim.2019.03.010

    Article  Google Scholar 

  24. Urbas, U., Vrabič, R., Vukašinović, N.: Displaying product manufacturing information in augmented reality for inspection. Procedia CIRP 81, 832–837 (2019). https://doi.org/10.1016/j.procir.2019.03.208

    Article  Google Scholar 

  25. Polvi, J., et al.: Handheld guides in inspection tasks: augmented reality versus picture. IEEE Trans. Visual Comput. Graphics 24(7), 2118–2128 (2018). https://doi.org/10.1109/TVCG.2017.2709746

    Article  Google Scholar 

  26. Runji, J.M., Lin, C.Y.: Markerless cooperative augmented reality-based smart manufacturing double-check system: case of safe PCBA inspection following automatic optical inspection. Robotics and Computer-Integrated Manufacturing 64(February 2019), 101957 (2020). https://doi.org/10.1016/j.rcim.2020.101957

  27. Zhu, J., Ong, S.K., Nee, A.Y.C.: A context-aware augmented reality assisted maintenance system. Int. J. Comput. Integr. Manuf. 28(2), 213–225 (2015). https://doi.org/10.1080/0951192X.2013.874589

    Article  Google Scholar 

  28. Angrisani, L., Arpaia, P., Esposito, A., Moccaldi, N.: A wearable brain-computer interface instrument for augmented reality-based inspection in industry 4.0. IEEE Trans. Instrum. Meas. 69(4), 1530–1539 (2020). https://doi.org/10.1109/TIM.2019.2914712

    Article  Google Scholar 

  29. Gattullo, M., Scurati, G.W., Fiorentino, M., Uva, A.E., Ferrise, F., Bordegoni, M.: Towards augmented reality manuals for industry 4.0: a methodology. Robotics and Computer-Integrated Manufacturing 56(August 2018), 276–286 (2019). https://doi.org/10.1016/j.rcim.2018.10.001

  30. Wang, T., et al.: CAPturAR: An augmented reality tool for authoring human-involved context-aware applications. In: UIST 2020 - Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology, pp. 328–341 (2020). https://doi.org/10.1145/3379337.3415815

  31. Cao, Y., et al.: GhostAR: a Time-space Editor for Embodied Authoring of Human-Robot Collaborative Task with Augmented Reality (2019). Accessed 06 Feb 2021. https://doi.org/10.1145/3332165.3347902

  32. Wang, Y., Zhang, S., Wan, B., He, W., Bai, X.: Point cloud and visual feature-based tracking method for an augmented reality-aided mechanical assembly system. Int. J. Adv. Manuf. Technol. 99(9–12), 2341–2352 (2018). https://doi.org/10.1007/s00170-018-2575-8

    Article  Google Scholar 

  33. Wang, X., Ong, S.K., Nee, A.Y.C.: Multi-modal augmented-reality assembly guidance based on bare-hand interface. Adv. Eng. Inform. 30(3), 406–421 (2016). https://doi.org/10.1016/j.aei.2016.05.004

    Article  Google Scholar 

  34. Blattgerste, J., Strenge, B., Renner, P., Pfeiffer, T., Essig, K.: Comparing conventional and augmented reality instructions for manual assembly tasks. ACM International Conference Proceeding Series, Part F1285, pp. 75–82 (2017). https://doi.org/10.1145/3056540.3056547

  35. Danielsson, O., Syberfeldt, A., Holm, M., Wang, L.: Operators perspective on augmented reality as a support tool in engine assembly. Procedia CIRP 72, 45–50 (2018). https://doi.org/10.1016/j.procir.2018.03.153

    Article  Google Scholar 

  36. Mourtzis, D., Vlachou, A., Zogopoulos, V.: Cloud-based augmented reality remote maintenance through shop-floor monitoring: a product-service system approach. J. Manuf. Sci. E. T. ASME 139(6), 1–11 (2017). https://doi.org/10.1115/1.4035721

    Article  Google Scholar 

  37. Hu, S.J., Zhu, X., Wang, H., Koren, Y.: Product variety and manufacturing complexity in assembly systems and supply chains. CIRP Ann. Manuf. Technol. 57(1), 45–48 (2008). https://doi.org/10.1016/j.cirp.2008.03.138

    Article  Google Scholar 

  38. Orfi, N., Terpenny, J., Sahin-Sariisik, A.: Harnessing product complexity: step 1establishing product complexity dimensions and indicators. Engineering Economist 56(1), 59–79 (2011). https://doi.org/10.1080/0013791X.2010.549935

    Article  Google Scholar 

  39. Bujak, K.R., Radu, I., Catrambone, R., MacIntyre, B., Zheng, R., Golubski, G.: A psychological perspective on augmented reality in the mathematics classroom. Comput. Educ. 68, 536–544 (2013). https://doi.org/10.1016/j.compedu.2013.02.017

    Article  Google Scholar 

  40. Zydney, J.M., Warner, Z.: Mobile apps for science learning: review of research. Comput. Educ. 94, 1–17 (2016). https://doi.org/10.1016/j.compedu.2015.11.001

    Article  Google Scholar 

  41. Ibáñez, M.B., Delgado-Kloos, C.: Augmented reality for STEM learning: a systematic review. Comput. Educ. 123(April), 109–123 (2018). https://doi.org/10.1016/j.compedu.2018.05.002

    Article  Google Scholar 

  42. Becerra-Fernandez, I., Sabherwal, R.: Knowledge Management: Systems and Processes. Routledge (2014)

    Google Scholar 

  43. Wang, X., Ong, S.K., Nee, A.Y.C.: A comprehensive survey of augmented reality assembly research. Advances in Manufacturing 4(1), 1–22 (2016). https://doi.org/10.1007/s40436-015-0131-4

    Article  Google Scholar 

  44. Fernández del Amo I.F., Erkoyuncu, J.A., Roy, R., Palmarini, R., Onoufriou, D.: A systematic review of augmented Reality content-related techniques for knowledge transfer in maintenance applications. Computers in Industry 103, 47–71 (2018). https://doi.org/10.1016/j.compind.2018.08.007

  45. Palmarini, R., Erkoyuncu, J.A., Roy, R., Torabmostaedi, H.: A systematic review of augmented reality applications in maintenance. Robotics and Computer-Integrated Manufacturing 49(July 2017), 215–228 (2018). https://doi.org/10.1016/j.rcim.2017.06.002

  46. Masood, T., Egger, J.: Augmented reality in support of Industry 4.0—implementation challenges and success factors. Robotics and Computer-Integrated Manufacturing 58, 181–195 (2019). https://doi.org/10.1016/j.rcim.2019.02.003

    Article  Google Scholar 

  47. Egger, J., Masood, T.: Augmented reality in support of intelligent manufacturing – a systematic literature review. Computers and Industrial Engineering, 140. Elsevier Ltd, 106195 (2020). https://doi.org/10.1016/j.cie.2019.106195

  48. Koumaditis, K., Venckute, S., Jensen, F.S., Chinello, F.: Immersive training: outcomes from small scale AR/VR pilot-studies. In: 26th IEEE Conference on Virtual Reality and 3D User Interfaces, VR 2019 - Proceedings, vol. 2019-January, pp. 1894–1898 (2019). https://doi.org/10.1109/VR44988.2019.9044162

  49. Werrlich, S., Daniel, A., Ginger, A., Nguyen, P.A., Notni, G.: Comparing HMD-based and paper-based training. In: Proceedings of the 2018 IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2018, pp. 134–142 (2019). https://doi.org/10.1109/ISMAR.2018.00046

  50. Smith, E., McRae, K., Semple, G., Welsh, H., Evans, D., Blackwell, P.: Enhancing vocational training in the post-COVID era through mobile mixed reality. Sustainability 13(11), 6144 (2021). https://doi.org/10.3390/SU13116144

  51. Moghaddam, M., Wilson, N.C., Modestino, A.S., Jona, K., Marsella, S.C.: Exploring augmented reality for worker assistance versus training. Advanced Engineering Informatics 50(April), 101410 (2021). https://doi.org/10.1016/j.aei.2021.101410

  52. Masood, T., Egger, J.: Adopting augmented reality in the age of industrial digitalisation. Comput. Ind. 115, 103112 (2020). https://doi.org/10.1016/J.COMPIND.2019.07.002

    Article  Google Scholar 

  53. Werrlich, S., Nguyen, P.A., Notni, G.: Evaluating the training transfer of head-mounted display based training for assembly tasks. In: ACM International Conference Proceeding Series, pp. 297–302 (2018). https://doi.org/10.1145/3197768.3201564

  54. Brice, D., Rafferty, K., McLoone, S.: AugmenTech: the usability evaluation of an AR system for maintenance in industry. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 12243 LNCS, pp. 284–303 (2020). https://doi.org/10.1007/978-3-030-58468-9_21

  55. Wang, Z.B., Ong, S.K., Nee, A.Y.C.: Augmented reality aided interactive manual assembly design. The International Journal of Advanced Manufacturing Technol. 69(5), 1311–1321 (2013). https://doi.org/10.1007/S00170-013-5091-X

  56. Westerfield, G., Mitrovic, A., Billinghurst, M.: Intelligent augmented reality training for motherboard assembly. International Journal of Artificial Intelligence in Education 25(1), 157–172 (2014). https://doi.org/10.1007/S40593-014-0032-X

  57. Sahu, C.K., Young, C., Rai, R.: Artificial Intelligence (AI) in Augmented Reality (AR)-Assisted Manufacturing Applications: A Review (2020). https://doi.org/10.1080/00207543.2020.1859636

  58. Noroozi, O., Kirschner, P.A., Biemans, H.J.A., Mulder, M.: Promoting argumentation competence: extending from first- to second-order scaffolding through adaptive fading. Educational Psychology Review 30(1), 153–176 (2018). Springer New York LLC. https://doi.org/10.1007/s10648-017-9400-z

  59. Cabello, V.M., Lohrmann, M.E.S.: Fading scaffolds in stem: supporting students’ learning on explanations of natural phenomena. Adv. Intell. Syst. Comput. 596, 350–360 (2018). https://doi.org/10.1007/978-3-319-60018-5_34

    Article  Google Scholar 

  60. Belland, B.R., Walker, A.E., Kim, N.J., Lefler, M.: Synthesizing results from empirical research on computer-based scaffolding in STEM education: a meta-analysis. Rev. Educ. Res. 87(2), 309–344 (2017). https://doi.org/10.3102/0034654316670999

    Article  Google Scholar 

  61. Lin, T.C., Hsu, Y.S., Lin, S.S., Changlai, M.L., Yang, K.Y., Lai, T.L.: A review of empirical evidence on scaffolding for science education. Int. J. Sci. Math. Educ. 10(2), 437–455 (2012). https://doi.org/10.1007/s10763-011-9322-z

    Article  Google Scholar 

  62. Hart, S.G., Staveland, L.E.: Development of NASA-TLX (Task Load Index): results of empirical and theoretical research. Advances in Psychology 52(C), 139–183 (1988). https://doi.org/10.1016/S0166-4115(08)62386-9

  63. Radu, I.: Augmented reality in education: a meta-review and cross-media analysis. Personal and Ubiquitous Computing 18(6), 1533–1543 (2014). https://doi.org/10.1007/s00779-013-0747-y

  64. Shah, A.: Fluid Coordination of Human-Robot Teams Eytan Modiano Chairman, Department Commiltee on Graduate Theses. Massachusetts Institute of Technology, Cambridge (2011). Accessed 09 Jun 2021. https://dspace.mit.edu/handle/1721.1/63034

  65. Autor, D., Mindel, D., Reynolds, E.: The Work of the Future: Building Better Jobs in an Age of Intelligent Machines. MIT Work of the Future (2020). https://workofthefuture.mit.edu/research-post/the-work-of-the-future-building-better-jobs-in-an-age-of-intelligent-machines/. Accessed 17 Mar 2021

  66. Frechette, S.P., Jones, A.T., Fischer, B.R.: Strategy for testing conformance to geometric dimensioning & tolerancing standards. Procedia CIRP 10, 211–215 (2013). https://doi.org/10.1016/j.procir.2013.08.033

    Article  Google Scholar 

  67. Hedberg, T., Lubell, J., Fischer, L., Maggiano, L., Feeney, A.B.: Testing the digital thread in support of model-based manufacturing and inspection. J. Computing and Inf. Science in Eng. 16(2) (2016). https://doi.org/10.1115/1.4032697

Download references

Acknowledgement

This material is based upon work supported by the National Science Foundation under the Future of Work at the Human-Technology Frontier Grant No. 2128743. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation. I acknowledge the support of our expert panel and industry partners.

Author information

Authors and Affiliations

  1. Department of Mechanical and Industrial Engineering and Khoury College of Computer Sciences, Northeastern University, Boston, MA, 02115, USA

    Mohsen Moghaddam

Authors
  1. Mohsen Moghaddam
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mohsen Moghaddam .

Editor information

Editors and Affiliations

  1. Industrial Engineering and Enterprise Information, Tunghai University, Taichung, Taiwan

    Chin-Yin Huang

  2. Department of Systems Science and Industrial Engineering, State University of New York at Binghamton, New York, NY, USA

    Sang Won Yoon

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Moghaddam, M. (2023). Augmenting Human-Machine Teaming Through Industrial AR: Trends and Challenges. In: Huang, CY., Yoon, S.W. (eds) Systems Collaboration and Integration. ICPR1 2021. Automation, Collaboration, & E-Services, vol 14. Springer, Cham. https://doi.org/10.1007/978-3-031-44373-2_22

Download citation

Publish with us

Policies and ethics

Search

Navigation