Abstract
Objects and their spatial relationships are important features for human visual perception. In most existing content-based image retrieval systems, however, only global features extracted from the whole image are used. While they are easy to implement, they have limited power to model semantic-level objects and spatial relationship. To overcome this difficulty, this paper proposes a constraint-based region matching approach to image retrieval. Unlike existing region-based approaches where either individual regions are used or only first-order constraints are modeled, the proposed approach formulates the problem in a probabilistic framework and simultaneously models both first-order region properties and second-order spatial relationships for all the regions in the image. Specifically, in this paper we present a complete system that includes image segmentation, local feature extraction, first- and second-order constraints, and probabilistic regionweight estimation. Extensive experiments have been carried out on a large heterogeneous image collection with 17,000 images. The proposed approach achieves significantly better performance than the state-of-the-art approaches.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Carson, C., Belongie, S., Greenspan, H., and Malik, J. 1997. Regionbased image querying. In Proc. IEEEWorkshop on Content-Based Access of Image and Video Libraries, pp. 42–49.
Cox, I.J., Miller, M.L., Minka, T.P., Papathomas, T.V., and Yianilos, P.N. 2000. The Bayesian image retrieval system, Pichunter: Theory, implementation, and psychophysical experiments. IEEE Trans. on Image Processing, 9(3):524–524.
Deng, Y. and Manjunath, B.S. 1999. An efficient low-dimensional color indexing scheme for region-based image retrieval. In Proc. of IEEE Intl. Conf. on Acoustics, Speech, and Signal Processing (ICASSP), pp. 3017–3020.
Felzenszwalb, P.F. and Huttenlocher, D.P. 1998. Image segmentation using local variation. In Proc. of IEEE Computer Vision and Pattern Recognition, Santa Barbara, CA, USA, pp. 98–104.
Gouet, V. and Boujemaa, N. 2001. Object-based queries using color points of interest. In Proc. of IEEE Workshop on Content-Based Access of Images and Videos (CBAIVL), Kauai, Hawaii.
Jain, A.K. 1989. Fundamentals of Digital Image Processing. Prentice-Hall Press, pp. 412–413.
Kam, A.H., Ng, T.T., Kingsbury, N.G., and Fitzgerald, W.J. 2000. Content based image retrieval through object extraction and querying, 2000. In Proc. IEEE Workshop on Content-based Access of Image and Video Libraries (CBAIVL), pp. 91–95.
Kazuyoshi, H. and Fumio, M. 1995. Constraint-based approach for automatic spatial layout planning. In Proceedings of the 11th Conference on Artificial Intelligence for Applications, Los Angeles, CA, USA, pp. 38–45.
Ko, B.C., Lee, H.S., and Byun, H. 2000. Region-based image retrieval system using efficient feature description. In Proc. of 15th International Conference on Pattern Recognition, vol. 4, pp. 283–286.
Leu, J.-G. 1991. Computing a shape moments from its boundary, Pattern Recognition, 10:949–957.
Li, J., Wang, J.Z., and Wiederhold, G. 2000. IRM: Integrated region matching for image retrieval. In Proceedings ACM Multimedia 2000, Los Angeles, CA, pp. 147–156.
Ma, W.Y. and Manjunath, B.S. 1997. Netra:AToolbox for navigation Large Image Database. In Proc. of IEEE ICIP, pp. 568–571.
Moghaddam, B., Biermann, H., and Margaritis, D. 2000. Image retrieval with local and spatial queries. In Proc. IEEE ICIP 2000, vol. 2, pp. 542–545.
Niblack, W. et al. 1993. The QBIC project: Querying images by content using color, texture and shape. In SPIE Proc. Storage and Retrieval for Image and Video Databases, San Jose, vol. 1908, pp. 173–187.
Pavlidis, T. and Liow, Y.T. 1990. Integrating region growing and edge detection. IEEE Trans Pattern Analysis and Machine Intelligence, 12(3):225–133.
Pentland, A., Picard, R., and Sclaroff, S. 1996. Photobook: Contentbased manipulation of image databases. Int. J. Comput. Vis., 18(3): 233–254.
Pfefferkorn, C. 1975. A heuristic problem solving design system for equipment or furniture layouts. Communications of the ACM, 18(5):286–297.
Rodden, K., Basalaj, W., Sinclair, D., and Wood, K. 2001. Does organization by similarity assist image browsing. In Proc. ACM Compute-Human Interaction (CHI), pp. 190–197.
Rui, Y., Huang, T., and Mehrotra, S. 1997. Content-based image retrieval with relevance feedback in MARS. In Proc. IEEE Int. Conf. on Image Processing.
Rui, Y. and Huang, T.S. 2000. Optimizing learning in image retrieval. In Proc. IEEE Int. Conf. on Computer Vision and Pattern Recognition (CVPR2000), vol. 1, pp. 236–243.
Rui, Y., Huang, T.S., and Chang, S.F. 1999. Image retrieval: Current techniques, promising directions and open issues. Journal of Visual Communication and Image Representation, vol. 10, pp. 39–62.
Salton, G. and McGill, M. J. 1982. Introduction to Modern Information Retrieval. McGraw-Hill Book Company: New York.
Smeulders, A.W.M., Worring, M., and Santini, S. 2000. Contentbased image retrieval at the end of the early years. IEEE Trans. PAMI, 22(12):1349–1380.
Smith, J.R. and Chang, S.F. 1996. VisualSEEK: A Fully Automated Content-Based Image Query System. ACM Multimedia 1996, Bostan MA.
Smith, J.R. and Chang S.F. 1997. SaFe: A general framework for integrated spatial and feature image search. In IEEE FirstWorkshop on Multimedia Signal Processing, pp. 301–306.
Swain, M.J. and Ballard, D.H. 1991. Color indexing. International Journal of Computer Vision, 7(1):11–32.
Tian, Q., Wu, Y., and Huang, T.S. 2000. Combine user defined region-of-interest and spatial layout for image retrieval. In IEEE International Conference on Image Processing, Vancouver, BC, vol. 3, pp. 746–749.
Tian, Q., Sebe, N., Lew, M.S., Loupias, E., and Huang, T.S. 2001. Image Retrieval Using Wavelet-Based Salient Points. Journal of Electronic Imaging, Special Issue on Storage and Retrieval of Digital Media, 10(4):835–849.
Wang, T., Rui Y., and Hu, S.M. 2001. Optimal Adaptive Learning for Image Retrieval. In Proc. of IEEE Computer Vision and Pattern Recognition (CVPR2001), Kauai, Hawaii, pp. 1140–1147.
Wang, W., Song, Y., and Zhang, A. 2002. Semantics retrieval by content and context of image regions. In Proc. the 15th International Conference on Vision Interface, Calgary, Canada.
Xu, Y., Duygulu, P., Saber, E., Tekalp, A.M., and Yarman-Vural, F.T. 2000. Object based image retrieval based on multi-level segmentation. In Proc. IEEE ICASSP, Istanbul, Turkey, vol. 6, pp. 2019–2022.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Wang, T., Rui, Y. & Sun, JG. Constraint Based Region Matching for Image Retrieval. International Journal of Computer Vision 56, 37–45 (2004). https://doi.org/10.1023/B:VISI.0000004831.53436.88
Issue Date:
DOI: https://doi.org/10.1023/B:VISI.0000004831.53436.88