Abstract
In view of the low efficiency of production and assembly in traditional industries, we use AR technology to replace traditional assembly instructions and design an industry-oriented cloud-edge intelligent assembly guidance system. Since the computing power of AR glasses cannot meet the high complexity requirements of scene understanding, we adopt the joint solution of Cloud-Edge. First, the sensor data collected by the AR glasses are streamed to the edge server using high-speed and low-latency wireless interconnection technology. Then, the product artifacts in the data scene are identified and understood through the instance segmentation network BlendMask based on deep learning. Then, the 3D pose of the object is calculated in real time by combining pose estimation and 3D reconstruction. Furthermore, an accurate 3D guidance animation is generated, and the virtual 3D model in the AR glasses is accurately superimposed on the real object to determine whether the assembly is correct in real time. Experiments show that the system effectively combines artificial intelligence and intelligent manufacturing, integrates various elements in the scene in real time to provide operators with multimodal and multidimensional immersive guidance, and corrects in time when assembly errors occur. It can not only quickly guide the operator to complete the learning of the assembly process but also assist the staff in the assembly in real time. Ultimately, it improves assembly speed and accuracy, which in turn improves enterprise productivity.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
HoÅ™ejÅ¡Ã, P., Novikov, K., Å imon, M.: A smart factory in a smart city: virtual and augmented reality in a smart assembly line. IEEE Access 8(8), 94330–94340 (2020)
Deb, M., Kannan, P.: Assembly and installation guidance by augmented reality. In: CIRED 2021-The 26th International Conference and Exhibition on Electricity Distribution, pp. 1317–1320 (2021)
Jianjun, T., Bo, Y., Junhao, G.: Exploration and practice of AR intelligent guidance technology for aircraft assembly operations. Aviat. Manufact. Technol. 62(08), 22–27 (2019)
Yue, W.: Research on Augmented Reality Fusion Technology for Product Assembly Guidance (2018)
Liu, R., Fan, X., Yin, X., et al.: Estimation method of matrix and part pose combination in AR-assisted assembly. Mech. Des. Res. 34(06), 119–125+137 (2018)
Chen, H., Sun, K., et al.: BlendMask: top-down meets bottom-up for instance segmentation. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8570–8578. IEEE, New York (2020)
Liu, Q., Wang, Y., et al.: Remote collaborative assembly and maintenance guidance based on mixed reality. Chin. J. Graph. 42(02), 216–221 (2021)
Varghese, B., Reaño, C., et al.: Accelerator virtualization in fog computing: moving from the cloud to the edge. Cloud Computing, IEEE, vol. 5, pp. 28–37. IEEE (2018)
Tian, Z., Shen, C., et al.: FCOS: fully convolutional one-stage object detection. In: International Conference on Computer Vision, ICCV, pp. 9626–9635. IEEE/CVF (2019)
Su, L., Sun, Y., et al.: A review of instance segmentation based on deep learning. J. Intell. Syst. 17(01), 16–31 (2022)
Liu, R.: Research on the pose estimation and state detection method of base parts in AR-assisted assembly. Shanghai Jiaotong University (2018)
Hui, L., Yuan, J., et al.: Superpoint network for point cloud oversegmentation. In: International Conference on Computer Vision 2021, pp. 5490–5499. IEEE/CVF (2021)
Landrieu, L., Simonovsky, M.: Large-scale point cloud semantic segmentation with superpoint graphs. In: Conference on Computer Vision and Pattern Recognition 2018, CVPR, pp. 4558–4567. IEEE/CVF, Salt Lake City (2018)
Rusu, R., Blodow, N., et al.: Fast Point Feature Histograms (FPFH) for 3D registration. In: International Conference on Robotics and Automation, pp. 3212–3217. IEEE (2009)
Peng, C.: K-means based RANSAC algorithm for ICP registration of 3D point cloud with dense outliers. In: International Conference on Consumer Electronics-Taiwan, ICCE-TW, pp. 1–2. IEEE (2021)
Li, P., Wang, R., et al.: Evaluation of the ICP algorithm in 3D point cloud registration. IEEE Access 8, 68030–68048 (2020)
Sock J., Garcia-Hernando G., et al.: Active 6D multi-object pose estimation in cluttered scenarios with deep reinforcement learning. In: International Conference on Intelligent Robots and Systems (IROS), pp. 10564–10571. IEEE/RSJ (2020)
Agati, S., Bauer, R., et al.: Augmented reality for manual assembly in Industry 4.0: gathering guidelines. In: Symposium on Virtual and Augmented Reality 2020, SVR, pp. 179–188. SVR (2020)
Bauer R., Agati S., et al.: Manual PCB assembly using augmented reality towards total quality. In: Symposium on Virtual and Augmented Reality, pp. 189–198. SVR (2020)
Liu, S., Li, M.: Multimodal GAN for energy efficiency and cloud classification in Internet of Things. Internet of Things Journal 6(4), 6034–6041 (2019)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Liu, D., Chang, Z., Ma, J., Wang, T., Li, M. (2022). Industry-Oriented Cloud Edge Intelligent Assembly Guidance System. In: Wang, Y., Zhu, G., Han, Q., Zhang, L., Song, X., Lu, Z. (eds) Data Science. ICPCSEE 2022. Communications in Computer and Information Science, vol 1629. Springer, Singapore. https://doi.org/10.1007/978-981-19-5209-8_16
Download citation
DOI: https://doi.org/10.1007/978-981-19-5209-8_16
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-19-5208-1
Online ISBN: 978-981-19-5209-8
eBook Packages: Computer ScienceComputer Science (R0)