Abstract
Transparent objects are one of the most common objects in everyday life. The poses of these objects must be estimated to pick and manipulate such objects. However, recognizing and estimating the poses of transparent objects is still a challenging task in robot vision, even after the emergence of 3D sensors. It is difficult to detect transparent objects because of the absorption and refraction of light, and the appearance of transparent objects can vary in different backgrounds. In this work, we propose a simple yet effective method for transparent object pose estimation, in which we address the problem using a synthetic dataset to train a deep neural network and estimate the pose of known transparent objects. After creating a synthetic dataset for transparent objects, we used a one-shot deep neural network to estimate the 6D pose of a known object. To the best of our knowledge, this is the first time synthetic data have been used for transparent object pose estimation. We conducted experiments on 3D printed transparent objects in a real environment as well as in a simulation environment. The results show that the proposed method can successfully estimate the pose of a transparent object even though it is only trained using synthetic data.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Chen, Y., Sun, G., Lin, H., Chen, S.: Random bin picking with multi-view image acquisition and CAD-based pose estimation. In: 2018 IEEE International Conference on Systems, pp. 2218–2223 (2018)
He, R., Rojas, J., Guan, Y.: A 3D object detection and pose estimation pipeline using RGB-D images. In: 2017 IEEE International Conference on Robotics and Biomimetics (ROBIO), pp. 1527–1532 (2017)
Wu, C., Jiang, S., Song, K.: CAD-based pose estimation for random bin-picking of multiple objects using a RGB-D camera. In: 2015 15th International Conference on Control Automation and Systems (ICCAS), pp. 1645–1649 (2015)
**ang, Y., Schmidt, T., Narayanan, V., Fox, D.: PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes. c (RSS) (2018)
Wang, C., et al.: DenseFusion: 6D object pose estimation by iterative dense fusion. In: Computer Vision and Pattern Recognition (CVPR), pp. 3343–3352 (2019)
Tremblay, J., To, T., Sundaralingam, B., ** of household objects. In: Conference on Robot Learning (CoRL), pp. 306–316 (2018)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: International Conference on Learning Representations (2015)
Guo-Hua, C., Jun-Yi, W., Ai-Jun, Z.: Transparent object detection and location based on RGB-D camera. J. Phys. Conf. Ser. (JPCS) 1183, 012011 (2019)
Dosovitskiy, A., et al.: FlowNet: learning optical flow with convolutional networks. In: 2015 IEEE International Conference on Computer Vision (ICCV), pp. 2758–2766 (2015)
Tremblay, J., To, T., Birchfield, S.: Falling things: a synthetic dataset for 3D object detection and pose estimation. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 2119–21193 (2018)
Müller, M., Casser, V., Lahoud, J., Smith, N., Ghanem, B.: Sim4CV: a photo-realistic simulator for computer vision applications. Int. J. Comput. Vis. 126, 902–919 (2018)
McCormac, J., Handa, A., Leutenegger, S., Davison, A.J.: SceneNet RGB-D: 5M photorealistic images of synthetic indoor trajectories with ground truth. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp. 2678–2687 (2016)
Ros, G., Sellart, L., Materzynska, J., Vázquez, D., López, A.: The SYNTHIA dataset: a large collection of synthetic images for semantic segmentation of urban scenes. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3234–3243 (2016)
Tobin, J., Fong, R., Ray, A., Schneider, J., Zaremba, W., Abbeel, P.: Domain randomization for transferring deep neural networks from simulation to the real world. In: 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 23–30 (2017)
Sadeghi, F., Levine, S.: CAD2RL: Real Single-Image Flight without a Single Real Image. CoRR (2016)
Tremblay, J., et al.: Training deep networks with synthetic data: bridging the reality gap by domain randomization. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 1082–10828 (2018)
Lin, T.-Y., et al.: Microsoft COCO: common objects in context. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8693, pp. 740–755. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10602-1_48
Johnson-Roberson, M., Barto, C., Mehta, R., Sridhar, S., Rosaen, K., Vasudevan, R.: Driving in the matrix: can virtual worlds replace human-generated annotations for real world tasks? In: 2017 IEEE International Conference on Robotics and Automation (ICRA), pp. 746–753 (2017)
Mayer, N., et al.: A large dataset to train convolutional networks for disparity, optical flow, and scene flow estimation. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4040–4048 (2016)
Hinterstoisser, S., Lepetit, V., Wohlhart, P., Konolige, K.: On pre-trained image features and synthetic images for deep learning. In: Leal-Taixé, L., Roth, S. (eds.) ECCV 2018. LNCS, vol. 11129, pp. 682–697. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-11009-3_42
Lysenkov, I., Rabaud, V.: Pose estimation of rigid transparent objects in transparent clutter. In: 2013 IEEE International Conference on Robotics and Automation, pp. 162–169 (2013)
Moreno-Noguer, F., Lepetit, V., Fua, P.: Accurate non-iterative O(n) solution to the PnP problem. In: 2007 IEEE 11th International Conference on Computer Vision, pp. 1–8 (2007)
Sajjan, S., et al.: Clear grasp: 3D shape estimation of transparent objects for manipulation. In: 2020 IEEE International Conference on Robotics and Automation (ICRA), pp. 3634–3642 (2020)
Liu, X., Jonschkowski, R., Angelova, A., Konolige, K.: KeyPose: multi-view 3D labeling and keypoint estimation for transparent objects. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 11599–11607 (2020)
Byambaa, M., Choimaa, L., Koutaki, G.: 6D pose estimation of transparent object from single RGB image. In: The 25th Conference of FRUCT Association, pp. 444–447 (2019)
Acknowledgement
The authors are thankful for the support from the MJEED research grant (Research profile code: J14C16) Higher Engineering Education Development Project in Mongolia.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Byambaa, M., Koutaki, G., Choimaa, L. (2022). 6D Pose Estimation of Transparent Objects Using Synthetic Data. In: Sumi, K., Na, I.S., Kaneko, N. (eds) Frontiers of Computer Vision. IW-FCV 2022. Communications in Computer and Information Science, vol 1578. Springer, Cham. https://doi.org/10.1007/978-3-031-06381-7_1
Download citation
DOI: https://doi.org/10.1007/978-3-031-06381-7_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-06380-0
Online ISBN: 978-3-031-06381-7
eBook Packages: Computer ScienceComputer Science (R0)