Abstract
Augmented reality (AR) and object tracking are active research fields and few research exists that combines both. We propose a markerless video AR system that applies video-processing tasks on the input video and virtual-data processing tasks on the virtual data and then adaptively relates both virtual and real data. The proposed system tracks a target and enables the user to select virtual data such as objects, images, or texts, for augmentation but adapted to the current state of the target. For adaptation, the virtual data is segmented to be overlaid on the target. To account for zoom, scaling, translation, and rotation of tracked object, and to make a proper alignment of virtual data, the system applies a homography estimation based on a feature point detector. Augmentation is performed on extracted keyframes of the input video. Experimental results show promising different AR applications of the proposed system, e.g., social media or industrial maintenance. No objective measures are known for video AR systems. We propose to use the image histogram to objectively measure performance by showing the histograms of the segmented virtual object before and after applying estimated homography is preserved.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Alcantarilla, P.F., Bartoli, A., Davison, A.J.: KAZE features. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7577, pp. 214–227. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-33783-3_16
Balntas, V., Lenc, K., Vedaldi, A., Mikolajczyk, K.: HPatches: a benchmark and evaluation of handcrafted and learned local descriptors. In: Proceedings of IEEE Conference on Computer Vision Pattern Recognition (2017)
Bertinetto, L., Valmadre, J., Golodetz, S., Miksik, O., Torr, P.H.S.: Staple: complementary learners for real-time tracking. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016
Dastjerdi, N.S., Valognes, J., Amer, M.: Effective keyframe extraction from RGB and RGB-D video sequences. In: International Conference on Image Processing Theory, Tools and Applications, pp. 1–5. IEEE, Montreal, Quebec, November 2017
Eisert, P., Fechteler, P., Rurainsky, J.: 3-D tracking of shoes for virtual mirror applications. In: Proceedings of IEEE Conference on Computer Vision Pattern Recognition, Anchorage, AK, pp. 1–6, 23–28 June 2008
Fischler, M., Bolles, R.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24, 381–395 (1981)
Freedman, D., Turek, M.: Illumination-invariant tracking via graph cuts. In: Proceedings IEEE Conference on CVPR, San Diego, USA, pp. 10–17, 20–25 June 2005
Fuji, T., Mitsukura, Y., Moriya, T.: Interactive augmented reality art book to promote Malaysia traditional game. IEEE Computer, Communications, and Control Technology (I4CT), Langkawi, Malaysia, pp. 203–208, September 2014
Genc, Y., Riedel, S., Souvannavong, F., Akinlar, C., Navab, N.: Marker-less tracking for AR: a learning-based approach. In: IEEE-C-ISMAR, Darmstadt, Germany, pp. 295–304, October 2002
Hadjdemetriou, E., Grossberg, M., Nayar, S.: Histogram preserving image transformations. Int. J. Comput. Vis. 45, 5–23 (2001)
Hare, S., et al.: Struck: structured output tracking with kernels. IEEE Trans. Pattern Anal. Machine Intell. 38, 2096–2109 (2016)
Khandelwal, P., Swarnalatha, P., Bisht, N., Prabu, S.: Detection of features to track objects and segmentation using GrabCut for application in marker-less augmented reality. Elsevier Procedia Comput. Sci. 58, 698–705 (2015)
Kristan, M., et al.: The sixth visual object tracking VOT2018 challenge results. In: IEEE European Conference on Computer Vision Workshops (2018)
Lehiani, Y., Maidi, M., Preda, M., Ghorbel, F.: Object identification and tracking for steady registration in mobile augmented reality. In: Proceedings of IEEE International Conference on Signal and Image Processing Applications (ICSIPA), Kuala Lumpur, Malaysia, pp. 54–59, 19–21 October 2015
Liu, T., Sun, J., Zheng, N., Tang, X., Shum, H.: Learning to detect a salient object. In: Proceedings of IEEE Conference on CVPR, USA, pp. 353–367, June 2007
Lowe, D.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60, 91–110 (2004)
Motokurumada, M., Ohta, M., Yamashita, K.: A video-based augmented reality system. In: IEEE 17th International Symposium on Consumer Electronics (ISCE), Taiwan, pp. 213–214, June 2013
Ozuysal, M., Calonder, M., Lepetit, V., Fua, P.: Fast keypoint recognition using random ferns. IEEE Trans. Pattern Anal. Machine Intell. 32, 448–461 (2010)
Park, J., Seo, D., Ku, M., Jung, I., Jeong, C.: Multiple 3D object tracking using ROI and double filtering for augmented reality. In: Proceedings of IEEE International Conference on Multimedia and Ubiquitous Engineering (MUE), Crete, Greece, pp. 317–322, 28–30 June 2011
Vacchetti, L., Lepetit, V., Fua, P.: Stable real-time 3D tracking using online and offline information. IEEE Trans. Pattern Anal. Machine Intell. 26, 1385–1391 (2004)
Verdie, Y., Yi, K., Fua, P., Lepetit, V.: Tilde: a temporally invariant learned detector. In: Proceedings of IEEE Conference on Computer Vision Pattern Recognition (2015)
Yee, T., Arshad, H., Abdullah, A.: Development of a PC-based markerless augmented reality. In: Proceedings of IEEE International Conference on Electrical Engineering and Informatics, Bali, Indonesia, pp. 49–53, 10–11 August 2015
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Valognes, J., Dastjerdi, N.S., Amer, M. (2019). Augmenting Reality of Tracked Video Objects Using Homography and Keypoints. In: Karray, F., Campilho, A., Yu, A. (eds) Image Analysis and Recognition. ICIAR 2019. Lecture Notes in Computer Science(), vol 11662. Springer, Cham. https://doi.org/10.1007/978-3-030-27202-9_21
Download citation
DOI: https://doi.org/10.1007/978-3-030-27202-9_21
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-27201-2
Online ISBN: 978-3-030-27202-9
eBook Packages: Computer ScienceComputer Science (R0)