SpringerLink
Log in

Unsupervised Clustering of Natural Images in Automatic Image Annotation Systems

  • Chapter
  • First Online:
New Approaches in Intelligent Image Analysis

Part of the book series: Intelligent Systems Reference Library ((ISRL,volume 108))

Abstract

The chapter is devoted to automatic annotation of natural images joining the strengths of text-based and content-based image retrieval. The Automatic Image Annotation (AIA) is based on the semantic concept models, which are built from large number of patches receiving from a set of images. In this case, image retrieval is implemented by keywords called as Visual Words (VWs) that is similar to text document retrieval. The task involves two main stages: a low-level segmentation based on color, texture, and fractal descriptors (a shape descriptor is less useful due to great variety of visual objects and their projections in natural images) and a high-level clustering of received descriptors into the separated clusters corresponding to the VWs set. The enhanced region descriptor including color, texture (with the high order moments—skewness and kurtosis), and fractal features (fractal dimension and lacunarity) has been proposed. For the VWs generation, the unsupervised clustering is a suitable approach. The Enhanced Self-Organizing Incremental Neural Network (ESOINN) was chosen due to its main benefits as a self-organizing structure and on-line implementation. The preliminary image segmentation permitted to change a sequential order of descriptors entering in the ESOINN as the associated sets. Such approach simplified, accelerated, and decreased the stochastic variations of the ESOINN. Our experiments demonstrate acceptable results of the VWs clustering for a non-large natural image sets. Precision value of clustering achieved up to 85–90 %. Our approach show better precision values and execution time as compared with fuzzy c-means algorithm and classic ESOINN. Also issues of parallel implementation of unsupervised segmentation in OpenMP and Intel Cilk Plus environments were considered for processing of HD-quality images. Execution time has been increased on 26–32 % using the parallel computations.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
GBP 19.95
Price includes VAT (United Kingdom)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
GBP 71.50
Price includes VAT (United Kingdom)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
GBP 89.99
Price includes VAT (United Kingdom)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free ship** worldwide - see info
Hardcover Book
GBP 89.99
Price includes VAT (United Kingdom)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free ship** worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Tamura, H., Yokoya, N.: Image database systems: a survey. Pattern Recogn. 17(1), 29–43 (1984)

    Article  Google Scholar 

  2. Jain, A.K., Vailaya, A.: Image retrieval using color and shape. Pattern Recogn. 29(8), 1233–1244 (1996)

    Article  Google Scholar 

  3. Antani, S., Kasturi, R., Jain, R.: A survey on the use of pattern recognition methods for abstraction, indexing and retrieval of images and video. Pattern Recogn. 35(4), 945–965 (2002)

    Article  MATH  Google Scholar 

  4. Islam, M.M., Zhang, D., Lu, G.: Automatic categorization of image regions using dominant color based vector quantization. In: IEEE International Conference on Digital Image Computing: Techniques and Applications (DICTS’2008), pp. 191–198 (2008)

    Google Scholar 

  5. Chang, E., Goh, K., Sychay, G., Wu, G.: CBSA: content-based soft annotation for multimodal image retrieval using Bayes point machines. IEEE Trans. Circ. Syst. Video Technol. 13(1), 26–38 (2003)

    Article  Google Scholar 

  6. Cusano, C., Ciocca, G., Schettini, R.: Image annotation using SVM. In: Proceedings of SPIE, SPIE internet imaging, vol. 5304, pp. 330–338 (2004)

    Google Scholar 

  7. Carneiro, G., Chan, A.B., Moreno, P.J., Vasconcelos, N.: Supervised learning of semantic classes for image annotation and retrieval. IEEE Trans. Pattern Anal. Mach. Intell. 29(3), 394–410 (2007)

    Article  Google Scholar 

  8. Liu, Y., Zhang, D., Lu, G.: Region-based image retrieval with high-level semantics using decision tree learning. Pattern Recogn. 41(8), 2554–2570 (2008)

    Article  MATH  Google Scholar 

  9. Cavus, O., Aksoy, S.: Semantic scene classification for image annotation and retrieval. In: da Vitoria Lobo, N., Kasparis, T., Roli, F., Kwok, J.T., Georgiopoulos, M., Anagnostopoulos, G.C., Loog, M. (eds) Structural, Syntactic, and Statistical Pattern Recognition, pp. 402–410. Springer, Berlin (2008)

    Google Scholar 

  10. Rao, C., Kumar, S.S., Mohan, B.C.: Content based image retrieval using exact Legendre moments and support vector machine. Int. J. Multimedia Appl. 2(2), 69–79 (2010)

    Article  Google Scholar 

  11. Index of /imageclef/resources. IAPR TC-12 Benchmark. http://www-i6.informatik.rwth-aachen.de/imageclef/resources/iaprtc12.tgz. Accessed 20 Dec 2014

  12. Maree, R., Geurts, P., Piater, J., Wehenkel, L.: Random subwindows for robust image classification. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’2005), vol. 1, pp. 34–40 (2005)

    Google Scholar 

  13. Balasubramanian, G.P., Saber, E., Misic, V., Peskin, E., Shaw, M.: Unsupervised color image segmentation by dynamic color gradient thresholding. In: SPIE 6806, Human Vision and Electronic Imaging XIII, pp. 68061H–68061H-9 (2008)

    Google Scholar 

  14. Chan, T.F., Vese, L.A.: Active contours without edges. IEEE Trans. Image Process. 10(2), 266–277 (2001)

    Article  MATH  Google Scholar 

  15. Vanhamel, I., Pratikakis, I., Sahli, H.: Multiscale gradient watersheds of color images. IEEE Trans. Image Process. 12(6), 617–626 (2003)

    Article  MATH  Google Scholar 

  16. Makrogiannis, S., Vanhamel, I., Fotopoulos, S., Sahli, H., Cornelis, J.: Watershed-based multiscale segmentation method for color images using automated scale selection. J. Electron. Imaging 14(3), 1–16 (2005)

    Google Scholar 

  17. Jung, C.R.: Combining wavelets and watersheds for robust multiscale image segmentation. Image Vis. Comput. 25(1), 24–33 (2007)

    Article  Google Scholar 

  18. Favorskaya, M.N., Petukhov, N.Y.: Comprehensive calculation of the characteristics of landscape images. J. Opt. Technol. 77(8), 504–509 (2010)

    Article  Google Scholar 

  19. Mansouri, A.R., Mitiche, A., Vazquez, C.: Multiregion competition: a level set extension of region competition to multiple region image partitioning. Comput. Vis. Image Underst. 101(3), 137–150 (2006)

    Article  Google Scholar 

  20. Alnihou, J.: An efficient region-based approach for object recognition and retrieval based on mathematical morphology and correlation coefficient. Int. Arab. J. Inf. Technol. 5(2), 154–161 (2008)

    Google Scholar 

  21. Ugarriza, L.G., Saber, E., Vantaram, S.R., Amuso, V., Shaw, M., Bhaskar, R.: Automatic image segmentation by dynamic region growth and multi-resolution merging. IEEE Trans. Image Process. 18(10), 2275–2288 (2009)

    Article  MathSciNet  Google Scholar 

  22. Shi, J., Malik, J.: Normalized cuts and image segmentation. IEEE Trans. Pattern Anal. Intell. 22(8), 88–905 (2000)

    Google Scholar 

  23. Carson, C., Belongie, S., Greenspan, H., Malik, J.: Blobworld: image segmentation using expectation-maximization and its application to image querying. IEEE Trans. Pattern Anal. Mach. Intell. 24(8), 1026–1038 (2002)

    Article  Google Scholar 

  24. Tu, Z., Zhu, S.C.: Image segmentation by data-driven Markov chain Monte Carlo. IEEE Trans. Pattern Anal. Mach. Intell. 24(5), 657–673 (2002)

    Article  Google Scholar 

  25. Rui, S., **, W., Chua, T.S.: A novel approach to auto image annotation based on pairwise constrained clustering and semi-Naıve Bayesian model. In: 11th International Conference on Multimedia Modelling, pp. 322–327 (2005)

    Google Scholar 

  26. Alata, O., Ramananjarasoa, C.: Unsupervised textured image segmentation using 2-D quarter-plane autoregressive model with four prediction supports. Pattern Recogn. Lett. 26(8), 1069–1081 (2005)

    Article  Google Scholar 

  27. Cariou, C., Chehdi, K.: Unsupervised texture segmentation/classification using 2-D autoregressive modeling and the stochastic expectation-maximization algorithm. Pattern Recogn. Lett. 29(7), 905–917 (2008)

    Article  Google Scholar 

  28. Mezaris, V., Kompatsiaris, I., Strintzis, M.G.: An ontology approach to object-based image retrieval. In: International Conference on Image Processing (ICIP’2003), vol. 2, pp. 511–514 (2003)

    Google Scholar 

  29. Haralick, R.M., Shanmugam, K., Dinstein, I.: Textural features for image classification. IEEE Trans. SMC–3 6, 610–621 (1973)

    Google Scholar 

  30. **a, Y., Feng, D., Wang, T., Zhao, R., Zhang, Y.: Image segmentation by clustering of spatial patterns. Pattern Recogn. Lett. 28(12), 1548–1555 (2007)

    Article  Google Scholar 

  31. Hou, Y., Lun, X., Meng, W., Liu, T., Sun, X.: Unsupervised segmentation method for color image based on MRF. In: International Conference on Computational Intelligence and Natural Computing (CINC’2009), vol. 1, pp. 174–177 (2009)

    Google Scholar 

  32. Celik, T., Tjahjadi, T.: Unsupervised color image segmentation using dual-tree complex wavelet transform. Comput. Vis. Image Underst. 114(7), 813–826 (2010)

    Article  Google Scholar 

  33. Comaniciu, D., Meer, P.: Mean shift: a robust approach toward feature space analysis. IEEE Trans. Pattern Anal. Mach. Intell. 24(5), 603–619 (2002)

    Article  Google Scholar 

  34. Haykin, S.O.: Neural Networks and Learning Machines, 3edn. McMaster University, Prentice-Hall, Canada (2008)

    Google Scholar 

  35. Deng, Y., Manjunath, B.S.: Unsupervised segmentation of color-texture regions in images and video. IEEE Trans. Pattern Anal. Mach. Intell. 23(8), 800–810 (2001)

    Article  Google Scholar 

  36. Komati, K.S., Salles, E.O.T., Filho, M.S.: Unsupervised color image segmentation based on local fractal dimension and J-images. In: IEEE International Conference on Industrial Technology (ICIT’2010), pp. 303–308 (2010)

    Google Scholar 

  37. Yang, A.Y., Wright, J., Ma, Y., Sastry, S.S.: Unsupervised segmentation of natural images via lossy data compression. Comput. Vis. Image Underst. 110(2), 212–225 (2008)

    Article  Google Scholar 

  38. Nock, R., Nielsen, F.: Statistical region merging. IEEE Trans. Pattern Anal. Mach. Intell. 26(11), 1452–1458 (2004)

    Article  Google Scholar 

  39. Martınez-Uso, A., Pla, F., Garcıa-Sevilla, P.: Unsupervised color image segmentation by low-level perceptual grou**. Pattern Anal. Appl. 16(4), 581–594 (2013)

    Article  MathSciNet  Google Scholar 

  40. Kim, J.S., Hong, K.S.: Color–texture segmentation using unsupervised graph cuts. Pattern Recogn. 42(5), 735–750 (2009)

    Article  MATH  Google Scholar 

  41. Yang, Y., Han, S., Wang, T., Tao, W., Tai, X.C.: Multilayer graph cuts based unsupervised color-texture segmentation using multivariate mixed Student’s t-distribution and regional credibility merging. Pattern Recogn. 46(4), 1101–1124 (2013)

    Article  MATH  Google Scholar 

  42. Vogel, T., Nguyen, D.Q., Dittmann, J.: BlobContours: adapting Blobworld for supervised color- and texture-based image segmentation. In: Multimedia Content Analysis, Management, and Retrieval, SPIE, vol. 6073, pp. 158–169 (2006)

    Google Scholar 

  43. Unnikrishnan, R., Hebert, M.: Measures of similarity. In: IEEE Workshop on Application of Computer Vision, vol. 1, pp. 394–400 (2005)

    Google Scholar 

  44. Silakari, S., Motwani, M., Maheshwari, M.: Color image clustering using block truncation algorithm. Int. J. Comput. Sci. Issues 4(2), 31–35 (2009)

    Google Scholar 

  45. Kekre, H.B., Mirza, T.: Content based image retrieval using BTC with local average thresholding. In: International Conference on Content Based Image Retrieval (ICCBIR’2008). Niagara Falls, Canada, pp. 5–9 (2008)

    Google Scholar 

  46. Chakravarti, R., Meng, X.: A study of color histogram based image retrieval. In: IEEE 6th International Conference on Information Technology: New Generations, pp. 1323–1328 (2009)

    Google Scholar 

  47. Rao, P.S., Vamsidhar, E., Raju, G.S.V.P., Satapat, R., Varma, K.V.S.R.P.: An approach for CBIR system through multi layer neural network. Int. J. Eng. Sci. Technol. 2(4), 559–563 (2010)

    Google Scholar 

  48. Long, X., Su, D., Hu, R.: Incremental leaning algorithm for self-organizing fuzzy neural network. In: IEEE 7th International Conference on Computer Science & Education (ICCSE’2012), pp. 71–74 (2012)

    Google Scholar 

  49. Tung, W.L., Quek, C.: GenSoFNN: A generic self-organizing fuzzy neural network. IEEE Trans. Neural Netw. 13(5), 1075–1086 (2002)

    Article  Google Scholar 

  50. Malek, H., Ebadzadeh, M.M., Rahmati, M.: Three new fuzzy neural networks learning algorithms based on clustering, training error and genetic algorithm. Appl. Intell. 37(2), 280–289 (2011)

    Article  Google Scholar 

  51. Hao, P., Ding, Y., Fang, Y.: Image retrieval based on fuzzy kernel clustering and invariant moments. In: 2nd International Symposium on Intelligent Information Technology Application (IITA’2008), Shanghai, vol. 1, pp. 447–452 (2008)

    Google Scholar 

  52. Wang, H., Mohamad, D., Ismail, N.A.: Semantic Gap in CBIR: automatic objects spatial relationships semantic extraction and representation. Int. J. Image Process. 4(3), 192–204 (2010)

    Google Scholar 

  53. Swain, M.J., Ballard, D.H.: Color indexing. Int. J. Comput. Vis. 7(1), 11–32 (1991)

    Article  Google Scholar 

  54. Dubey, R.S., Choubey, R., Bhattacharjee, J.: Multi feature content based image retrieval. Int. J. Comput. Sci. Eng. 2(6), 2145–2149 (2010)

    Google Scholar 

  55. Pass, G., Zabith, R.: Histogram refinement for content-based image retrieval. In: IEEE Workshop on Applications of Computer Vision, pp. 96–102 (1996)

    Google Scholar 

  56. Huang, J., Kuamr, S., Mitra, M., Zhu, W.J., Zabih, R.: Image indexing using color correlogram. In: International Conference on Computer Vision and Pattern Recognition (CVPR’1997), San Juan, Puerto Rico, pp. 762–765 (1997)

    Google Scholar 

  57. Manjunath, B.S., Salembier, P., Sikora, T.: Introduction to MPEG-7: Multi-media Content Description Language. John Wiley & Sons Ltd., New York (2002)

    Google Scholar 

  58. Tamura, H., Mori, S., Yamawaki, T.: Texture features corresponding to visual perception. IEEE Trans. Syst. Man Cybern. 8(6), 460–473 (1978)

    Article  Google Scholar 

  59. Selvarajah, S., Kodituwakku, S.R.: Analysis and comparison of texture features for content based image retrieval. Int. J. Latest Trends Comput. 2(1), 108–113 (2011)

    Google Scholar 

  60. Haralick, R.M., Shanmugum, K., Dinstein, I.: Textural features for image classification. IEEE Trans. Syst. Man Cybern. 3(6), 610–621 (1973)

    Article  Google Scholar 

  61. Galloway, M.M.: Texture analysis using gray level run lengths. Comput. Graph. Image Process. 4(2), 172–179 (1975)

    Article  Google Scholar 

  62. Favorskaya, M., Damov, M., Zotin, A.: Intelligent texture reconstruction of missing data in video sequences using neural networks. In: Tweedale, J.W., Jain, L.C. (eds.) Advanced Techniques for Knowledge Engineering and Innovative Applications, pp. 163–176. Springer, Berlin (2013)

    Chapter  Google Scholar 

  63. Favorskaya, M., Damov, M., Zotin, A.: Accurate spatio-temporal reconstruction of missing data in dynamic scenes. Pattern Lett. Recogn. 34(14), 1694–1700 (2013)

    Article  Google Scholar 

  64. Favorskaya, M.N., Petukhov, N.Y.: Recognition of natural objects on air photographs using neural networks. Optoelectron. Instrum. Data Process. 47(3), 233–238 (2011)

    Article  Google Scholar 

  65. Mandelbrot, B.B.: The Fractal Geometry of Nature. Freeman, San Francisco (1982)

    MATH  Google Scholar 

  66. Slabaugh, G., Boyes, R., Yang, X.: Multicore Image Processing with OpenMP. IEEE Sig. Process. Mag. 27(2), 134–138 (2010)

    Article  Google Scholar 

  67. Saleem, S., Lali, I.U., Nawaz, M.S., Nauman, A.B.: Multi-core program optimization: parallel sorting algorithms in Intel Cilk Plus. Int. J. Hybrid Inf. Technol. 7(2), 151–164 (2014)

    Article  Google Scholar 

  68. Furao, S., Ogura, T., Hasegawa, O.: An enhanced self-organizing incremental neural network for online unsupervised learning. Neural Netw. 20(8), 893–903 (2007)

    Article  MATH  Google Scholar 

  69. Tangruamsub, S., Tsuboyama, M., Kawewong, A., Hasegawa, O.: Mobile robot vision-based navigation using self-organizing and incremental neural networks. In: International Joint Conference on Neural Networks (IJCNN’2009), pp. 3094–3101 (2009)

    Google Scholar 

  70. Najjar, T., Hasegawa, O.: Self-organizing incremental neural network (SOINN) as a mechanism for motor babbling and sensory-motor learning in developmental robotics. In: Rojas, I., Joya, G., Gabestany, J. (eds.) Advances in Computational Intelligence, pp. 321–330. Springer, Berlin (2013)

    Chapter  Google Scholar 

  71. De Paz, J.F., Bajo, J., Rodríguez, S., Corchado, J.M.: Computational intelligence techniques for classification in microarray analysis. In: Bichindaritz, I., Vaidya, S., Jain, A., Jain, L.C. (eds.) Computational Intelligence in Healthcare 4. Studies in Computational Intelligence, pp. 289–312. Springer, Berlin (2010)

    Chapter  Google Scholar 

  72. De Paz, J.F., Rodríguez, S., Bajo, J.: Multiagent systems in expression analysis. In: Demazeau, Y., Pavón, J., Corchado, J.M., Bajo, J. (eds.) 7th International Conference on Practical Applications of Agents and Multi-Agent Systems. Advances in Intelligent and Soft Computing, pp. 217–226. Springer, Berlin (2009)

    Chapter  Google Scholar 

  73. Ester, M., Kriegel, H.P., Sander, J., Xu, X.: Density-based algorithm for discovering clusters in large spatial databases with noise. In: 2nd International Conference on Knowledge Discovery and Data Mining (KDD‘1996), pp. 226–231. AAAI Press (1996)

    Google Scholar 

  74. Ishioka, T.: An expansion of X-means for automatically determining the optimal number of clusters. In: IASTED International conference on Computational Intelligence (CI’2005), Calgary, Alberta, Canada, pp. 91–96 (2005)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute of Informatics and Telecommunications, Siberian State Aerospace University, 31 Krasnoyarsky Rabochy, Krasnoyarsk, 660037, Russian Federation

    Margarita Favorskaya & Alexander Proskurin

  2. Bournemouth University, Fern Barrow, Poole, UK

    Lakhmi C. Jain

  3. University of Canberra, Canberra, ACT 2601, Australia

    Lakhmi C. Jain

Authors
  1. Margarita Favorskaya
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lakhmi C. Jain
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alexander Proskurin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lakhmi C. Jain .

Editor information

Editors and Affiliations

  1. Technical University of Sofia, Sofia, Bulgaria

    Roumen Kountchev

  2. University of Hyogo, Chuo-ku, Japan

    Kazumi Nakamatsu

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Favorskaya, M., Jain, L.C., Proskurin, A. (2016). Unsupervised Clustering of Natural Images in Automatic Image Annotation Systems. In: Kountchev, R., Nakamatsu, K. (eds) New Approaches in Intelligent Image Analysis. Intelligent Systems Reference Library, vol 108. Springer, Cham. https://doi.org/10.1007/978-3-319-32192-9_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-32192-9_4

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-32190-5

  • Online ISBN: 978-3-319-32192-9

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation