Abstract
In the evolving field of industrial automation, operator awareness of robot actions and intentions is critical for safety and efficiency, especially when working in close proximity to robots. From the robot-to-human communication angle, a collaborative robot (cobot) is expected to express its internal states and monitor task progress. Various traditional communication modalities (e.g., tower light, external screen, LED ring, and sound) often fall short of conveying nuanced information, while a flexible display curved around the cobot arm using organic light-emitting diode (OLED) technology provides a potential advantage. Integrated seamlessly with the robot, this interface enhances interaction by displaying text and video, enriching communication, and positively influencing the human–robot collaboration experience. In this work, we investigate a novel integrated flexible OLED display technology used as a robotic skin-interface to improve robot-to-human communication in a real industrial setting at Volkswagen (VW), following a user-centric Double-Diamond co-design process. We first conducted a co-design workshop with six operator representatives to collect their ideas and expectations on how the robot should communicate with them. The gathered information was used to design an interface for a collaborative human-robot interaction task in motor assembly. The interface was implemented in a workcell and validated qualitatively with a small group of operators (n = 9) and quantitatively with a large group (n = 42). The validation results showed that using flexible OLED technology could improve the operators’ attitude toward the robot, increase their intention to use the robot, enhance perceived enjoyment, social influence, and trust, and reduce their anxiety.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data availability
Data are available from the corresponding author upon reasonable request.
References
Babel, F., Kraus, J., Miller, L., Kraus, M., Wagner, N., Minker, W., Baumann, M.: Small talk with a robot? the impact of dialog content, talk initiative, and gaze behavior of a social robot on trust, acceptance, and proximity. Int. J. Soc. Robot. 13(6), 1485–1498 (2021). https://doi.org/10.1007/s12369-020-00730-0
Ballard, J.A., Trent, D.M.: Idea generation and productivity: the promise of csm. Public Prod. Rev. 12(4), 373–386 (1989). https://doi.org/10.2307/3380151. (publisher: Taylor & Francis Ltd)
Beer, J.M., Prakash, A., Mitzner, T.L., Rogers, W.A.: Understanding Robot Acceptance, pp. 1–45. Georgia Institute of Technology (2011)
Berg, J., Lu, S.: Review of interfaces for industrial human-robot interaction. Curr. Robot. Rep. 1(2), 27–34 (2020). https://doi.org/10.1007/s43154-020-00005-6
Bonarini, A.: Communication in human-robot interaction. Curr. Robot. Rep. 1(4), 279–285 (2020). https://doi.org/10.1007/s43154-020-00026-1
British Standards Institute: BS EN 60073:2002: Basic and safety principles for man-machine interface, marking and identification. Coding principles for indicators and actuators. British Standards Institute. ISBN: 0580403599 (2002)
Cao, H.-L., Scholz, C., De Winter, J., Makrini, I.E., Vanderborght, B.: Investigating the role of multi-modal social cues in human-robot collaboration in industrial settings. Int. J. Soc. Robot. 15(7), 1169–1179 (2023). https://doi.org/10.1007/s12369-023-01018-9
Castro, A., Silva, F., Santos, V.: Trends of human-robot collaboration in industry contexts: handover, learning, and metrics. Sensors 21(12), 4113 (2021)
Choi, H., Swanson, N.: Understanding worker trust in industrial robots for improving workplace safety. In: Nam, C.S., Lyons, J.B. (eds.) Trust in Human-Robot Interaction, pp. 123–141. Academic Press, ??? (2021). https://doi.org/10.1016/B978-0-12-819472-0.00005-8\. https://www.sciencedirect.com/science/article/pii/B9780128194720000058
Davis, F.D.: Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Q. 13(3), 319–340 (1989). https://doi.org/10.2307/249008
Design Council: Double Diamond Framework. (2024). https://perma.cc/V3BG-VUMJ. Accessed 1 Feb 2024
Eilers, K., Nachreiner, F., Hänecke, K.: Entwicklung und überprüfung einer skala zur erfassung subjektiv erlebter anstrengung, pp. 214–224 (1986)
El Makrini, I., Merckaert, K., Lefeber, D., Vanderborght, B.: Design of a collaborative architecture for human-robot assembly tasks. In: 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 1624–1629 (2017). IEEE
El Makrini, I., Mathijssen, G., Verhaegen, S., Verstraten, T., Vanderborght, B.: A virtual element-based postural optimization method for improved ergonomics during human-robot collaboration. IEEE Trans. Autom. Sci. Eng. 19(3), 1772–1783 (2022). https://doi.org/10.1109/TASE.2022.3147702
Elprama, B., El Makrini, I., Jacobs, A.: Acceptance of collaborative robots by factory workers: a pilot study on the importance of social cues of anthropomorphic robots. In: International Symposium on Robot and Human Interactive Communication, vol. 7 (2016)
Fitzgerald, C.: Develo** baxter. In: 2013 IEEE Conference on Technologies for Practical Robot Applications (TePRA), pp. 1–6 (2013). IEEE
Frixione, M., Lombardi, A.: Street signs and ikea instruction sheets: Pragmatics and pictorial communication. Rev. Philos. Psychol. 6(1), 133–149 (2015). https://doi.org/10.1007/s13164-014-0216-1
Gallagher, M., Hares, T., Spencer, J., Bradshaw, C., Webb, I.: The nominal group technique: a research tool for general practice? Fam. Pract. 10(1), 76–81 (1993). https://doi.org/10.1093/fampra/10.1.76
Heerink, M., Kröse, B., Evers, V., Wielinga, B.: Assessing acceptance of assistive social agent technology by older adults: the Almere model. Int. J. Soc. Robot. 2(4), 361–375 (2010). https://doi.org/10.1007/s12369-010-0068-5
Hignett, S., McAtamney, L.: Rapid entire body assessment (reba). Appl. Ergon. 31(2), 201–205 (2000). https://doi.org/10.1016/S0003-6870(99)00039-3
Holleman, G.A., Hooge, I.T., Kemner, C., Hessels, R.S.: The ‘real-world approach’ and its problems: a critique of the term ecological validity. Front. Psychol. 11, 721 (2020)
Honig, S., Oron-Gilad, T.: Understanding and resolving failures in human-robot interaction: literature review and model development. Front. Psychol. 9, 861 (2018)
imk Industrial Intelligence: ema Designer. (2024). https://perma.cc/J2U5-PDAR. Accessed 1 Feb 2024
ISO: ISO/TS 15066:2016. https://perma.cc/V9EJ-N2S8. Accessed 17 July 2024
Knapp, M.L., Hall, J.A., Horgan, T.G., Knapp, M.L., Hall, J.A., Horgan, T.G.: Nonverbal Communication in Human Interaction, Eigth edition edn. Wadsworth Cengage Learning, Boston, MA (2014). OCLC: 800033348
Kumar, S.: Theories of musculoskeletal injury causation. Ergonomics 44(1), 17–47 (2001). https://doi.org/10.1080/00140130120716. (PMID: 11214897)
Li, S., Zheng, P., Liu, S., Wang, Z., Wang, X.V., Zheng, L., Wang, L.: Proactive human-robot collaboration: mutual-cognitive, predictable, and self-organising perspectives. Robot. Comput.-Integr. Manuf. 81, 102510 (2023). https://doi.org/10.1016/j.rcim.2022.102510
Lumelsky, V., Shur, M., Wagner, S.: Sensitive skin. IEEE Sens. J. 1, 41 (2001). https://doi.org/10.1109/JSEN.2001.923586
Mittendorfer, P., Cheng, G.: Humanoid multimodal tactile-sensing modules. IEEE Trans. Robot. 27(3), 401–410 (2011). https://doi.org/10.1109/TRO.2011.2106330
Nielsen norman group: heuristic evaluations: how to conduct. (2024). https://perma.cc/2DEC-CZVT. Accessed 1 Feb 2024
Nigg, B.M.: Biomechanics, load analysis and sports injuries in the lower extremities. Sports Med. (Auckland, N.Z.) 2(5), 367–379 (1985). https://doi.org/10.2165/00007256-198502050-00005
Pandey, A.K., Gelin, R.: A mass-produced sociable humanoid robot: pepper: The first machine of its kind. IEEE Robot. Autom. Mag. 25(3), 40–48 (2018)
Rettenmaier, M., Albers, D., Bengler, K.: After you?! - use of external human-machine interfaces in road bottleneck scenarios. Transport. Res. F Traffic Psychol. Behav. 70, 175–190 (2020). https://doi.org/10.1016/j.trf.2020.03.004
Robots, U.: UR+ YOUring. (2024). https://perma.cc/3KKS-A5HL. Accessed 1 Feb 2024
Scholz, C., Cao, H.-L., Makrini, I.E., Vanderborght, B.: Antropo: an open-source platform to increase the anthropomorphism of the Franka Emika collaborative robot arm. PLoS ONE 18(10), 0292078 (2023). https://doi.org/10.1371/journal.pone.0292078
Sciutti, A., Mara, M., Tagliasco, V., Sandini, G.: Humanizing human-robot interaction: on the importance of mutual understanding. IEEE Technol. Soc. Mag. 37(1), 22–29 (2018). https://doi.org/10.1109/MTS.2018.2795095
Sebanz, N., Bekkering, H., Knoblich, G.: Joint action: bodies and minds moving together. Trends Cognit. Sci. 10(2), 70–76 (2006). https://doi.org/10.1016/j.tics.2005.12.009
Simunovic, M.P.: Colour vision deficiency. Eye 24(5), 747–755 (2010). https://doi.org/10.1038/eye.2009.251. (number: 5 publisher: Nature Publishing Group)
Su, R., Park, S.H., Ouyang, X., Ahn, S.I., McAlpine, M.C.: 3d-printed flexible organic light-emitting diode displays. Sci. Adv. 8(1), 8798 (2022)
Takayama, L., Dooley, D., Ju, W.: Expressing thought: improving robot readability with animation principles. In: Proceedings of the 6th International Conference on Human-robot Interaction, pp. 69–76 (2011)
Tang, G., Webb, P., Thrower, J.: The development and evaluation of robot light skin: A novel robot signalling system to improve communication in industrial human-robot collaboration. Robot. Comput.-Integr. Manuf. 56, 85–94 (2019). https://doi.org/10.1016/j.rcim.2018.08.005
Tidwell, J., Brewer, C., Valencia, A.: Designing interfaces: patterns for effective interaction design, 3rd edn. O’Reilly, Sebastopol (2020)
Universal Robots: UR20 Cobot. (2024). https://perma.cc/HPE4-UJKT. Accessed 1 Feb 2024
University of North Carolina: lifting and material handling guidance. (2024). https://perma.cc/RV3L-T7CT. Accessed 1 Feb 2024
Venkatesh, V., Morris, M.G., Davis, G.B., Davis, F.D.: User acceptance of information technology: toward a unified view. MIS Q. 27(3), 425–478 (2003). https://doi.org/10.2307/30036540
Whitespectre: emerging UI/UX patterns in generative AI: a visual guide. (2023). https://perma.cc/6GE9-WZS2. Accessed 1 Feb 2024
Williams, M.D., Rana, N.P., Dwivedi, Y.K.: The unified theory of acceptance and use of technology (utaut): a literature review. J. Enterp. Inf. Manag. 28(3), 443–488 (2015)
Winter, J., Dodou, D.: External human-machine interfaces: gimmick or necessity? Transport. Res. Interdiscipl. Perspect. 15, 100643 (2022). https://doi.org/10.1016/j.trip.2022.100643
Yang, W., Hon, M., Yao, H., Tee, B.C.: An Atlas for Large-Area Electronic Skins: From Materials to Systems Design. Cambridge University Press (2020)
Acknowledgements
The work leading to these results received funding from the European Union’s Horizon 2020 research and innovation program as part of the SOPHIA project under Grant Agreement No. 871237, euROBIN under Grant Agreement No. 101070596, and as part of the SAFEBOT program through funding provided by imec.
Author information
Authors and Affiliations
Contributions
All authors contributed to the study’s conception and design. Material preparation, data collection, and analysis were performed by Constantin Scholz, Hoang-Long Cao, Susanne Niehaus, and Maximilian Kaufmann. The first draft of the manuscript was written by Constantin Scholz, Hoang-Long Cao, and Susanne Niehaus. All authors commented on previous versions of the manuscript. All authors have read and approved the final manuscript.
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no Conflict of interest.
Ethical statement
The authors declare that this research was conducted in full accordance with the Declaration of Helsinki and the organization’s ethical guidelines. Informed consent was obtained from all individual participants involved in the study.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Scholz, C., Cao, HL., El Makrini, I. et al. Improving robot-to-human communication using flexible display technology as a robotic-skin-interface: a co-design study. Int J Intell Robot Appl (2024). https://doi.org/10.1007/s41315-024-00343-0
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s41315-024-00343-0