Abstract
RGB-T object tracking is develo** rapidly in the past decade due to the complementarity of visible (RGB) and thermal infrared (T) images. However, many trackers using multi-modal information by simple feature concatenation, which ignores both the modality and channel reliability. In this paper, we propose a correlation filter-based RGB-T tracker to learn the reliability weights in terms of modality and inter-channel. Specifically, the channel regularization collaborates with the spatial regularization to jointly learn the filter and channel weights. Besides, we design a novel objective function to optimize the modality reliability weight frame by frame. Through the reliability evaluation, the useful information hidden in the modalities and channels is fully exploited. We perform extensive experiments on the RGB-T benchmark, i.e., GTOT, to verify the effectiveness of the proposed method. Experimental results show that the proposed fusion strategy can improve tracking performance.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Stephen, R.S., Alex, L.C.: Enhanced target tracking through infrared-visible image fusion. In: 14th International Conference on Information Fusion, pp. 1–8. IEEE, Chicago, IL, USA (2011)
Alex, L.C., Stephen, R.S.: Fusing concurrent visible and infrared videos for improved tracking performance. Opt. Eng. 52(1), 7004 (2013)
Lichao, Z., Martin, D., Abel, G.G., Joost, W., Fahad, S.H.: Multi-modal fusion for end-to-end RGB-T tracking. In: ICCV Workshop, pp. 2252–2261. Springer, Seoul, Korea (2019)
Chenglong, L., Chengli, Z., Yan, H., **, T., Liang, W.: Cross-modal ranking with soft consistency and noisy labels for robust RGB-T tracking. In: Proceedings of European Conference on Computer Vision, pp. 831–847. Springer, Munich, Germany (2018)
Yulong, W., Chenglong, L., **, T.: Learning soft-consistent correlation filters for RGB-T object tracking. In: PRCV 2018, pp. 259–306. Springer, Guangzhou, China (2018)
Chenglong, L., Hui, C., Shiyi, H., **aobai, L., **, T., Liang, L.: Learning collaborative sparse representation for grayscale-thermal tracking. IEEE TIP 25(12), 5743–5756 (2016)
Yi, W., Jongwoo, L., Ming-Hsuan, Y.: Online object tracking: a benchmark. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition, pp. 2411–2418. Portland Oregon (2013)
Luca, B., Jack, V., João, F.H., Andrea, V., Philip, H.S.T.: Fully-convolutional Siamese networks for object tracking. In: ECCV Workshops, pp. 850–865. Springer, Amsterdam (2016)
Martin, D., Goutam, B., Fahad, S.K., Michael, F.: ECO: Efficient convolution operators for tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6638–6646. Springer, Honolulu, Hawaii (2017)
João, F.H., Rui, C., Martins, P., Jorge B.: High-speed tracking with kernelized correlation filters. IEEE Trans. Pattern Anal. Mach. Intell. 37(3), 583–596 (2014)
Hamed, K. G., Ashton, F., Simon L.: Learning background-aware correlation filters for visual tracking. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1135–1143. Venice, Italy (2017)
Zhipeng, Z., Houwen, P.: Deeper and wider siamese networks for real-time visual tracking. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4591–4600. Long Beach, California (2019)
Kaihua, Z., Lei, Z., Qingshan, L., David, Z., Ming-Hsuan, Y.: Fast visual tracking via dense spatio-temporal context learning. In: Fleet, D., Pajdla, T., Schiele, B., Tuyte- laars, T. (eds.) ECCV 2014. LNCS, vol. 8693, pp. 127–141. Springer, Cham (2014)
Ilchae, J., Jeany, S., Mooyeol, B., Bohyung, H.: Real-time MDNet. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) Computer Vision—ECCV 2018. ECCV 2018. Lecture Notes in Computer Science, vol. 11208. Springer, Cham (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Zhang, F., Ma, S. (2022). Correlation Filter RGB-T Tracker with Modality and Channel Reliability. In: Yao, J., **ao, Y., You, P., Sun, G. (eds) The International Conference on Image, Vision and Intelligent Systems (ICIVIS 2021). Lecture Notes in Electrical Engineering, vol 813. Springer, Singapore. https://doi.org/10.1007/978-981-16-6963-7_61
Download citation
DOI: https://doi.org/10.1007/978-981-16-6963-7_61
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-16-6962-0
Online ISBN: 978-981-16-6963-7
eBook Packages: EngineeringEngineering (R0)