Abstract
Deep features can exhibit superior retrieval performance than low-level features. However, low-level features (e.g. colour and orientation) can be extracted by generally imitating the human visual perceptual system. Combining human-like low-level and deep features can harmoniously yield more discriminative representations. However, it remains challenging. To address this problem, a new representation method for image retrieval, namely the underlying importance feature histogram (UIFH), is presented in this study. Its main highlights are: (1) This new method extracts low-level features by simulating the human visual perception mechanism, such as opponent colour and orientation selectivity mechanisms. (2) Inspired by the salience evaluation mechanism, the new method can harmoniously evaluate the underlying importance information between deep and low-level features. (3) Assisting the various important information can facilitate the UIFH. It can substantially improve the discriminative power of representation. Comprehensive experiments on seven benchmark datasets demonstrated that the proposed UIFH method outperforms some recent state-of-the-art methods based on pre-trained models. The proposed UIFH method is suitable for the retrieval scenes where images have various colours and prominent orientations.
Similar content being viewed by others
Data availability
The datasets generated and/or analysed during the current study are available from the corresponding author upon reasonable request.
References
Tyagi V (2017) Content-based image retrieval: ideas, influences, and current trends. Springer, Singapore
Smeulders AWM, Worring M, Santini S et al (2000) Content-based image retrieval at the end of the early years. IEEE Trans Pattern Anal Mach Intell 22(12):1349–1380
Chen W, Liu Y, Wang W et al (2023) Deep learning for instance retrieval: a survey. IEEE Trans Pattern Anal Mach Intell 45(6):7270–7292
Zhu Y, Nachtrab G, Keyes PC et al (2018) Dynamic salience processing in paraventricular thalamus gates associative learning. Science 362(6413):423–429
Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. In: ICLR, pp 1–14
Gegenfurtner KR (2003) Cortical mechanisms of colour vision. Nat Rev Neurosci 4:563–572
Shapley R, Hawken M (2011) Color in the cortex-single-and double-opponent cells. Vis Res 51:701–717
Shapley R, Hawken M (2002) Neural mechanisms for color perception in the primary visual cortex. Curr Opin Neurobiol 12:426–432
Marĉelja S (1980) Mathematical description of the responses of simple cortical cells. J Opt Soc Am 70(11):1297–1300
Daugman JG (1980) Two-dimensional spectral analysis of cortical receptive field profiles. Vis Res 20(10):847–856
Daugman JG (1985) Uncertainty relation for resolution in space, spatial frequency, and orientation optimised by two-dimensional visual cortical filters. J Opt Soc Am A 2(7):1160–1169
Jain AK, Farrokhnia F (1991) Unsupervised texture segmentation using Gabor filters. Pattern Recognit 24:1167–1186
Kruizinga P, Petkov N (1999) Nonlinear operator for oriented texture. IEEE Trans Image Process 8(10):1395–1407
Liu GH, Yang JY (2021) Deep-seated features histogram: a novel image retrieval method. Pattern Recognit 116:107926
Liu GH, Yang JY (2019) Exploiting color volume and color difference for salient region detection. IEEE Trans Image Process 28(1):6–16
Liu GH, Yang JY, Li ZY (2015) Content-based image retrieval using computational visual attention model. Pattern Recognit 48(8):2554–2566
Yuan BH, Liu GH (2020) Image retrieval based on gradient-structures histogram. Neural Comput Appl 32(15):11717–11727
Singh C, Walia E, Kaur KP (2017) Color texture description with novel local binary patterns for effective image retrieval. Pattern Recognit 76:50–68
Dubey SR, Singh SK, Singh RK (2016) Multichannel decoded local binary patterns for content-based image retrieval. IEEE Trans Image Process 25(9):4018–4032
Saikia S, Fernández-Robles L, Alegre E et al (2021) Image retrieval based on texture using latent space representation of discrete Fourier transformed maps. Neural Comput Appl 33:13301–13316
Liu GH, Yang JY (2013) Content-based image retrieval using color difference histogram. Pattern Recognit 46(1):188–198
Clement M, Kurtz C, Wendling L (2018) Learning spatial relations and shapes for structural object description and scene recognition. Pattern Recognit 84:197–210
Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vision 60(2):91–110
Bay H, Tuytelaars T, Gool LV (2006) SURF: speeded up robust features. In: ECCV, pp 404–417
Sivic J, Zisserman A (2003) Video Google: a text retrieval approach to object matching in videos. In: ICCV, pp 1470–1477
Perronnin F, Sanchez J, Mensink T (2010) Improving the Fisher kernel for large-scale image classification. In: ECCV, pp 143–156
Jégou H, Perronnin F et al (2012) Aggregating local image descriptors into compact codes. IEEE Trans Pattern Anal Mach Intell 34(9):1704–1716
Jégou H, Zisserman A (2014) Triangulation embedding and democratic aggregation for image search. In: CVPR, pp 3310–3317
Husain SS, Bober M (2017) Improving large-scale image retrieval through robust aggregation of local descriptors. IEEE Trans Pattern Anal Mach Intell 39(9):1783–1796
Babenko A, Slesarev A, Chigorin A, Lempitsky V (2013) Neural codes for image retrieval. In: ECCV, pp 584–599
Babenko A, Lempitsky V (2015) Aggregating local deep features for image retrieval. In: ICCV, pp 1269–1277
Tolias G, Sicre R, Jégou H (2015) Particular object retrieval with integral max-pooling of CNN activations. In: ICLR, pp 1–15
Kalantidis Y, Mellina C, Osindero S (2016) Cross-dimensional weighting for aggregated deep convolutional features. In: ECCV, pp 685–701
Xu J, Wang C et al (2019) Unsupervised semantic-based aggregation of deep convolutional features. IEEE Trans Image Process 28(2):601–611
Lu F, Liu GH (2022) Image retrieval using contrastive weight aggregation histograms. Digit Signal Process 123:103457
Li J, Bo Y, Yang W et al (2021) Subspace-based multi-view fusion for instance-level image retrieval. Visual Comput 37:619–633
Liu GH, Yang JY (2023) Exploiting deep textures for image retrieval. Int J Mach Learn Cyb 14:483–493
Lu Z, Liu GH, Lu F, Zhang B (2023) Image retrieval using dual-weighted deep feature descriptor. Int J Mach Learn Cyb 14:643–653
Lu F, Liu GH (2023) Image retrieval using object semantic aggregation histogram. Cogn Comput 15:1736–1747
Arandjelovic R, Gronat P, Torii A, Pajdla T, Sivic J (2018) NetVLAD: CNN architecture for weakly supervised place recognition. IEEE Trans Pattern Anal Mach Intell 40(6):1437–1451
Radenovic F, Tolias G, Chum O (2018) Fine-tuning CNN image retrieval with no human annotation. IEEE Trans Pattern Anal Mach Intell 41(7):1655–1668
Husain SS, Ong EJ, Bober M (2021) ACTNET: end-to-end learning of feature activations and multi-stream aggregation for effective instance image retrieval. Int J Comput Vision 129:1432–1450
Bai C, Li H, Zhang J et al (2021) Unsupervised adversarial instance-level image retrieval. IEEE Trans Multimed 23:2199–2207
El-Nouby A, Neverova N, Laptev I, Jégou H (2021) Training vision transformers for image retrieval. https://doi.org/10.48550/ar**v.2102.05644
Van de Sande KEA, Gevers T, Snoek CGM (2010) Evaluating color descriptors for object and scene recognition. IEEE Trans Pattern Anal Mach Intell 32(9):1582–1596
Krizhevsky A, Sutskever I, Hinton GE (2012) ImageNet classification with deep convolutional neural networks. In: NeurIPS, pp 1097–1105
He K, Zhang X, et al (2016) Deep residual learning for image recognition. In: CVPR, pp 770–778
Ding X, Zhang X, et al (2021) RepVGG: making VGG-style ConvNets great again. In: CVPR, pp 13728–13737
Molenberghs G, Lesaffre E (1997) Non-Linear Integral Equations to Construct Bivariate Densities with Given Marginals and Dependence Function. Stat Sinica 7:713–738
UFLDL Tutorial, PCA Whitening. http://ufldl.stanford.edu/tutorial/unsupervised/PCAWhitening/. Accessed 20 Oct 2023
Philbin J, Chum O, et al (2007) Object retrieval with large vocabularies and fast spatial matching. In: CVPR, pp 1–8
Philbin J, Chum O, et al (2008) Lost in quantisation: improving particular object retrieval in large scale image databases. In: CVPR, pp 1–8
Jégou H, Douze M, Schmid C (2008) Hamming embedding and weak geometry consistency for large scale image search. In: ECCV, pp 304–317
Radenovic F, Iscen A, Tolias G, Avrithis Y, Chun O (2018) Revisiting Oxford and Paris: large-scale image retrieval benchmarking. In: CVPR, pp 5706–5715
Forcén JI, Pagola M, Barrenechea E, Bustince H (2020) Co-occurrence of deep convolutional features for image search. Image Vis Comput 97:103909
Gkelios S, Boutalis Y, Chatzichristofis SA (2021) Investigating the vision transformer model for image retrieval tasks. https://doi.org/10.48550/ar**v.2101.03771
Chum O, Philbin J, Sivic J, Isard M, Zisserman A (2007) Total recall: automatic query expansion with a generative feature model for object retrieval. In: ICCV, pp 1–8
Liu GH, Li ZY, Yang JY, Zhang D (2024) Exploiting sublimated deep features for image retrieval. Pattern Recognit 147:110076
Acknowledgements
This study is supported by National Natural Science Foundation of China (grant no. 62266008), the Foundation of Guangxi Normal University (grant no. 2021JC007) and the Foundation of Development Research Centre of Guangxi in Humanities and Social Sciences (grant no. ZXZJ202201). Here, I am very grateful to Dr. Fen Lu (My graduated doctor), because the first author (Qiao-** He) has other research tasks, the revised manuscript was completed by Dr. Fen Lu.
Author information
Authors and Affiliations
Contributions
QPH helped in conceptualisation, software, validation, writing—original draft, resources and data curation. GHL helped in methodology, writing—review & editing, supervision, revision, funding acquisition and formal analysis.
Corresponding author
Ethics declarations
Conflict of interest
Authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work submitted for publication.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
He, QP., Liu, GH. Image retrieval using underlying importance feature histogram. Neural Comput & Applic (2024). https://doi.org/10.1007/s00521-024-09735-6
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s00521-024-09735-6