SpringerLink
Log in

Eye center localization using gradient and intensity information under uncontrolled environment

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Accurate localization of eyes in low-resolution facial images is a challenging problem in computer vision community. The existing techniques provides inaccurate results for eye center localization in an uncontrolled environment, e.g. low resolution, low contrast, scale, pose and illumination variations etc. This study proposes a hybrid method for accurate eye center localization which shows robustness to above-mentioned problems. The proposed hybrid method is a two stage method. In the first stage, a new operator based on gradient and intensity information is proposed to extract the coarse eye candidates. The proposed operator uses the integral image and the dot product operations that make the system computationally efficient. The likelihood of eye centers are further verified in the second stage using a convolutional neural network architecture. In the verification stage, the normalized region of interest is adopted to solve the variations of different scales. The eye pair satisfying the predefined constraints is selected as the true eye pair. The proposed method is extensively tested on various databases to check its accuracy to the uncontrolled environment. The experimental analysis suggests that the proposed hybrid method can localize the eye center more precisely and eventually shows superior performance over some of the competitive state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

Similar content being viewed by others

References

  1. Abiyev RH, Arslan M (2019) Head mouse control system for people with disabilities. Expert Syst 3:e12398

    Google Scholar 

  2. Ahmed NY (2021) Real-time accurate eye center localization for low-resolution grayscale images. J Real-Time Image Proc 18(1):193–220

    Article  Google Scholar 

  3. Ahmed M, Laskar RH (2019) Eye center localization in a facial image based on geometric shapes of iris and eyelid under natural variability. Image Vis Comput. https://doi.org/10.1016/j.imavis.2019.05.002

    Article  Google Scholar 

  4. Ahmed M, Laskar RH (2019) Eye detection and localization in a facial image based on partial geometric shape of iris and eyelid under practical scenarios. J Electron Imaging 28(3):033009. https://doi.org/10.1117/1.JEI.28.3.033009

    Article  Google Scholar 

  5. Ahmed M, Laskar RH (2021) Evaluation of accurate iris center and eye corner localization method in a facial image for gaze estimation. Multimed Syst. https://doi.org/10.1007/s00530-020-00744-8

    Article  Google Scholar 

  6. Alonso-Fernandez F, Bigun J (2016) A survey on periocular biometrics research. Pattern Recogn Lett 82:92–105

    Article  Google Scholar 

  7. Asadifard M, Shanbezadeh J (2010) Automatic adaptive center of pupil detection using face detection and CDF analysis, In: Proceedings of the International Multiconference of Engineers and Computer Scientists, vol 1, p 3

  8. Baek SJ, Choi KA, Ma C, Kim YH, Ko SJ (2013) Eyeball model-based iris center localization for visible image-based eye-gaze tracking systems. IEEE Trans Consum Electron 59(2):415–421

    Article  Google Scholar 

  9. BF Database. Dataset for face detection, Facedb - BioID, https://www.bioid.com/facedb/. Google Scholar

  10. Borza D, Darabant AS, Danescu R (2016) Real-time detection and measurement of eye features from color images. Sensors 16(7):1105

    Article  Google Scholar 

  11. Chen S, Liu C (2015) Eye detection using discriminatory Haar features and a new efficient SVM. Image Vis Comput 33:68–77

    Article  Google Scholar 

  12. Choi JH, Lee KI, Song BC (2020) Eye pupil localization algorithm using convolutional neural networks. Multimed Tools Appl 79(43):32563–32574

    Article  Google Scholar 

  13. Cootes TF, Taylor CJ, Cooper DH (1995) Graham J (1995) Active shape models-their training and application. Comput Vis Image Underst 61(1):38–59

    Article  Google Scholar 

  14. Cootes TF, Edwards GJ, Taylor CJ (2001) Active appearance models. IEEE Trans Pattern Anal Mach Intell 23(6):681–685

    Article  Google Scholar 

  15. Cristinacce D, Cootes TF (2006) Feature detection and tracking with constrained local models. In Bmvc, vol 1, no. 2, p 3

  16. Daugman J (2009) How iris recognition works. In: The essential guide to image processing, pp 715–739

  17. Drewes H (2014) Eye gaze tracking. Interactive displays: natural human interface technologies, pp 251–283

  18. Duchowski AT (2007) Eye tracking methodology. Theory and practice, p 328

  19. Farkas LG, Katic MJ (2005) Forrest CR, International anthropometric study of facial morphology in various ethnic groups/races. J Craniofac Surg 16(4):615–646

    Article  Google Scholar 

  20. Fasel I, Fortenberry B, Movellan J (2005) A generative framework for real-time object detection and classification. Comput Vis Image Underst 98(1):182–210

    Article  Google Scholar 

  21. Gao W, Cao B, Shan S, Chen X, Zhou D, Zhang X, Zhao D (2008) The CAS-PEAL large-scale Chinese face database and baseline evaluations. IEEE Trans Syst Man Cybern A Syst Hum 38(1):149–161

    Article  Google Scholar 

  22. George A, Routray A (2016) Fast and accurate algorithm for eye localization for gaze tracking in low-resolution images. IET Comput Vision 10(7):660–669

    Article  Google Scholar 

  23. Gross R, Matthews I, Cohn J, Kanade T, Baker S (2010) Multi-pie. Image Vis Comput 28(5):807–813

    Article  Google Scholar 

  24. Hamouz M, Kittler J, Kamarainen JK, Paalanen P, Kalviainen H, Matas J (2005) Feature-based affine-invariant localization of faces. IEEE Trans Pattern Anal Mach Intell 27(9):1490–1495

    Article  Google Scholar 

  25. Hassaballah M, Kandazawa T, Ido S (2010) Efficient eye detection method based on grey intensity variance and independent components analysis. IET Comput Vision 4(4):261–271

    Article  Google Scholar 

  26. Hsu WY, Chung CJ (2021) A novel eye center localization method for multiview faces. Pattern Recognit 119:108078

    Article  Google Scholar 

  27. Ito Y, Ohyama W, Wakabayashi T, Kimura F (2012) Detection of eyes by circular Hough transform and histogram of gradient. In: Pattern Recognition (ICPR), 21st IEEE International Conference, pp 1795–1798

  28. Jain AK (1989) Fundamentals of digital image processing. Prentice Hall, Englewood Cliffs

    MATH  Google Scholar 

  29. Jariwala KN, Nandi A, Dalal UD (2015) A real-time robust eye center localization using geometric eye model and edge gradients in unconstrained visual environment. Int J Comput Appl 128(1):22–27

    Google Scholar 

  30. Jesorsky O, Kirchberg KJ, Frischholz RW (2001) Robust face detection using the Hausdorff distance. In: International Conference on audio-and video-based biometric person authentication. Springer, Berlin, pp 90–95

  31. **g MQ, Chen LH (2010) A novel method for horizontal eye line detection under various environments. Int J Pattern Recognit Artif Intell 24(03):475–498

    Article  Google Scholar 

  32. Jo J, Lee SJ, Park KR, Kim IJ, Kim J (2014) Detecting driver drowsiness using feature-level fusion and user-specific classification. Expert Syst Appl 41(4):1139–1152

    Article  Google Scholar 

  33. Jo J, Choi H, Kim IJ, Kim J (2015) Single-view-based 3D facial reconstruction method robust against pose variations. Pattern Recogn 48(1):73–85

    Article  Google Scholar 

  34. Kim BS, Lee H, Kim WY (2010) Rapid eye detection method for non-glasses type 3D display on portable devices. IEEE Trans Consum Electron 56(4):2498–2505

    Article  Google Scholar 

  35. Kim H, Jo J, Toh KA, Kim J (2017) Eye detection in a facial image under pose variation based on multi-scale iris shape feature. Image Vis Comput 57:147–164

    Article  Google Scholar 

  36. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems, pp 1097–1105

  37. Kroon B, Maas S, Boughorbel S, Hanjalic A (2009) Eye localization in low and standard definition content with application to face matching. Comput Vis Image Underst 113(8):921–933

    Article  Google Scholar 

  38. LeCun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324

    Article  Google Scholar 

  39. Leo M, Cazzato D, De Marco T, Distante C (2014) Unsupervised eye pupil localization through differential geometry and local self-similarity matching. PLoS ONE 9(8):e102829

    Article  Google Scholar 

  40. Levinshtein A, Phung E, Aarabi P (2018) Hybrid eye center localization using cascaded regression and hand-crafted model fitting. Image Vis Comput 71:17–24

    Article  Google Scholar 

  41. Liu ZT, Jiang CS, Li SH, Wu M, Cao WH, Hao M (2021) Eye state detection based on weight binarization convolution neural network and transfer learning. Appl Soft Comput 109:107565

    Article  Google Scholar 

  42. Markuš N, Frljak M, Pandžić IS, Ahlberg J, Forchheimer R (2014) Eye pupil localization with an ensemble of randomized trees. Pattern Recogn 47(2):578–587

    Article  Google Scholar 

  43. Monzo D, Albiol A, Sastre J, Albiol A (2011) Precise eye localization using HOG descriptors. Mach Vis Appl 22(3):471–480

    Google Scholar 

  44. Morimoto CH, Mimica MRM (2005) Eye gaze tracking techniques for interactive applications. Comput Vis Image Underst 98(1):4–24

    Article  Google Scholar 

  45. Oliveira LS, Borges DL, Vidal FB, Chang L (2012) A fast eye localization and verification method to improve face matching in surveillance videos. IEEE International Conference on Systems, Man, and Cybernetics (SMC), pp 840–845

  46. Ponz V, Villanueva A, Cabeza R (2012) Dataset for the evaluation of eye detector for gaze estimation. In: Proceedings of the 2012 ACM Conference on Ubiquitous Computing, pp 681–684

  47. Ren Y, Wang S, Hou B, Ma J (2014) A novel eye localization method with rotation invariance. IEEE Trans Image Process 23(1):226–239

    Article  MathSciNet  Google Scholar 

  48. Savakis A, Sharma R, Kumar M (2014) Efficient eye detection using HOG-PCA descriptor. In Imaging and Multimedia Analytics in a Web and Mobile World 2014, vol 9027, p 90270J. International Society for Optics and Photonics

  49. Skodras E, Fakotakis N (2015) Precise localization of eye centers in low-resolution color images. Image Vis Comput 36:51–60

    Article  Google Scholar 

  50. Song F, Tan X, Chen S, Zhou ZH (2013) A literature survey on robust and efficient eye localization in real-life scenarios. Pattern Recogn 46(12):3157–3173

    Article  Google Scholar 

  51. Tan X, Song F, Zhou ZH, Chen S (2009) Enhanced pictorial structures for precise eye localization under uncontrolled conditions. Computer Vision and Pattern Recognition, IEEE Conference, pp 1621–1628

  52. Timm F, Barth E (2011) Accurate eye centre localisation by means of gradients. Visapp 11:125–130

    Google Scholar 

  53. Valenti R, Gevers T (2012) Accurate eye center location through invariant isocentric patterns. IEEE Trans Pattern Anal Mach Intell 34(9):1785–1798

    Article  Google Scholar 

  54. Viola P, Jones M (2001) Rapid object detection using a boosted cascade of simple features. Computer Vision and Pattern Recognition, Proceedings of the 2001 IEEE Computer Society Conference, vol 1, pp I-I

  55. Wang P, Ji Q (2007) Multi-view face and eye detection using discriminant features. Comput Vis Image Underst 105(2):99–111

    Article  Google Scholar 

  56. Wang J, Sung A, Venkateswarlu R (2003) Eye gaze estimation from a single image of one eye. In: Computer Vision, Proceedings, Ninth IEEE International Conference; pp 136–143

  57. Wang P, Green MB, Ji Q, Wayman, J (2005) Automatic eye detection and its validation. Computer Vision and Pattern Recognition, IEEE Computer Society Conference, 3, pp 164-172

  58. **a Y, Lou J, Dong J, Qi L, Li G, Yu H (2020) Hybrid regression and isophote curvature for accurate eye center localization. Multimed Tools Appl 79(1):805–824

    Article  Google Scholar 

  59. Young D, Tunley H, Samuels R (1995) Specialized Hough transform and active contour methods for real-time eye tracking. University of Sussex, Cognitive & Computing Science

  60. Zhao S, Grigat RR (2006) Robust eye detection under active infrared illumination. In Proceedings of the 18th International Conference on Pattern Recognition, IEEE Computer Society, pp 481–484

Download references

Acknowledgements

This research work has been carried out in the department of R&D, CMR College of Engineering & Technology, Hyderabad, India, and in the Speech and Image Processing Lab, NIT Silchar, India, and is supported by Visvesvaraya Ph.D. Scheme of MeitY, Government of India (Ref. No.: PhD-MLA/4(74)/2015-16).

Author information

Authors and Affiliations

  1. Department of ECE, CMR College of Engineering & Technology, Kandlakoya, Telangana, 501401, Hyderabad, India

    Manir Ahmed

  2. Department of ECE, National Institute of Technology Silchar, Silchar, 788010, Assam, India

    Rabul Hussain Laskar

Authors
  1. Manir Ahmed
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rabul Hussain Laskar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Manir Ahmed.

Additional information

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ahmed, M., Laskar, R.H. Eye center localization using gradient and intensity information under uncontrolled environment. Multimed Tools Appl 81, 7145–7168 (2022). https://doi.org/10.1007/s11042-021-11805-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-021-11805-z

Keywords

Search

Navigation