SpringerLink
Log in

Pupil detection schemes in human eye: a review

  • Special Issue Paper
  • Published:
Multimedia Systems Aims and scope Submit manuscript

Abstract

Pupil detection in a human eyeimage or video plays a key role in many applications such as eye-tracking, diabetic retinopathy screening, smart homes, iris recognition, etc. Literature reveals pupil detection faces many complications including light reflections, cataract disease, pupil constriction/dilation moments, contact lenses, eyebrows, eyelashes, hair strips, and closed eye. To cope with these challenges, research community has been struggling to devise resilient pupil localization schemes for the image/video data collected using the near-infrared (NIR) or visible spectrum (VS) illumination. This study presents a critical review of numerous pupil detection schemes taken from standard sources. This review includes pupil localization schemes based on machine learning, histogram/thresholding, Integro-differential operator (IDO), Hough transform and among others. The probable pros and cons of each scheme are highlighted. Finally, this study offers recommendations for designing a robust pupil detection system. As scope of pupil detection is very broader, therefore this review would be a great source of information for the relevant research community.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Canada)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25
Fig. 26
Fig. 27
Fig. 28

taken from IIT-Kanpur Contact Lens [74], CASIA-IrisV3-Interval [41], JLUBRIRIS [75], MMU V1.0 [37], IITD V1.0 [42] and WVU Off-axis/angle datasets [43], respectively. g–i Images taken from UBIRIS V1.0 [44], UPOL [64] and UTIRIS [76] datasets, respectively

Similar content being viewed by others

References

  1. Joshi, V.B., Raval, M.S., Gupta, D., Rege, P.P., Parulkar, S.K.: A multiple reversible watermarking technique for fingerprint authentication. Multimed. Syst. 22(3), 367–378 (2016)

    Article  Google Scholar 

  2. Bhatnagar, G., Wu, Q.M.J.: Enhancing the transmission security of biometric images using chaotic encryption. Multimed. Syst. 20(2), 203–214 (2014)

    Article  Google Scholar 

  3. Yiu, Y.-H., Aboulatta, M., Raiser, T., Ophey, L., Flanagin, V.L., ZuEulenburg, P., Ahmadi, S.-A.: DeepVOG: open-source pupil segmentation and gaze estimation in neuroscience using deep learning. J. Neurosci. Methods 324, 108301–108307 (2019)

    Article  Google Scholar 

  4. Zhang, J., Sun, G., Zheng, K., Mazhar, S.: Pupil detection based on oblique projection using a binocular camera. IEEE Access 8, 105754–105765 (2020)

    Article  Google Scholar 

  5. Rojas-Líbano, D., Wainstein, G., Carrasco, X., Aboitiz, F., Crossley, N., Ossandón, T.: A pupil size, eye-tracking and neuropsychological dataset from ADHD children during a cognitive task. Sci. Data 6(25), 1–6 (2019)

    Google Scholar 

  6. Gautam, G., Mukhopadhyay, S.: An adaptive localization of pupil degraded by eyelash occlusion and poor contrast. Multimed. Tools Appl. 78(6), 6655–6677 (2019)

    Article  Google Scholar 

  7. Donida Labati, R., Muñoz, E., Piuri, V., Ross, A., Scotti, F.: Non-ideal iris segmentation using Polar Spline RANSAC and illumination compensation. Comput. Vis. Image Understand. 188, 102787 (2019)

    Article  Google Scholar 

  8. Cazzato, D., Dominio, F., Manduchi, R., Castro, S.: Real-time gaze estimation via pupil center tracking. Paladyn J. Behav. Robot. 9, 6–18 (2018)

    Article  Google Scholar 

  9. Quadar, N., Chehri, A., Geon, G.: Visual analytics methods for eye tracking data. In: Human Centred Intelligent Systems, pp. 3–12. Springer, Singapore (2021)

    Chapter  Google Scholar 

  10. Jian, L., Li, Z., Yang, X., Wu, W., Ahmad, A., Jeon, G.: Combining unmanned aerial vehicles with artificial-intelligence technology for traffic-congestion recognition: electronic eyes in the skies to spot clogged roads. IEEE Consumer Electron. Mag. 8(3), 81–86 (2019)

    Article  Google Scholar 

  11. Kang, S., Kim, S., Lee, Y.-S., Jeon, G.: Analysis of screen resolution according to gaze estimation in the 3D space. In: Convergence and hybrid information technology, pp. 271–277. Springer, Berlin (2012)

    Chapter  Google Scholar 

  12. Li, Q., Yang, X., Wu, W., Liu, K., Jeon, G.: Pansharpening multispectral remote-sensing images with guided filter for monitoring impact of human behavior on environment. Concurr. Comput. Pract. Exp. n/a(n/a), e5074.

  13. Anisetti, M., Bellandi, V., Damiani, E., Jeon, G., Jeong, J.: An adaptable architecture for human-robot visual interaction. In: IECON 2007—33rd Annual Conference of the IEEE Industrial Electronics Society. 119–124 (2007)

  14. Ma, L., Li, H., Yu, K.: Fast iris localization algorithm on noisy images based on conformal geometric algebra. Digital Sig. Process. 100, 102682 (2020)

    Article  Google Scholar 

  15. Dutta, A., Mondal, A., Dey, N., Sen, S., Moraru, L., Hassanien, A.E.: Vision tracking: a survey of the state-of-the-art. SN Comput. Sci. 1(1), 57 (2020)

    Article  Google Scholar 

  16. Rogalska, A., Rynkiewicz, F., Daszuta, M., Guzek, K., Napieralski, P.: Blinking extraction in eye gaze system for stereoscopy movies. Open Phys. 17, 512–518 (2019)

    Article  Google Scholar 

  17. Li, Y.H., Huang, P.J., Juan, Y.: An efficient and robust iris segmentation algorithm using deep learning. Mob. Inf. Syst. 2019, 1–14 (2019)

    Google Scholar 

  18. Bowyer, K.W., Hollingsworth, K., Flynn, P.J.: Image understanding for iris biometrics: a survey. Comput. Vis. Image Underst. 110(2), 281–307 (2008)

    Article  Google Scholar 

  19. Saad, E., George, L., Tayyar, A.: Accurate and fast pupil localization using contrast stretching, seed filling and circular geometrical constraints. J. Comput. Sci. 10, 305–315 (2014)

    Article  Google Scholar 

  20. Akinci, G., Polat, E., Koçak, OM.: A video based eye detection system for bipolar disorder diagnosis. In: 2012 20th Signal Processing and Communications Applications Conference (SIU): 18–20 April 2012 1–4 (2012)

  21. Zhu, D., Moore, S.T., Raphan, T.: Robust pupil center detection using a curvature algorithm. Comput. Methods Programs Biomed. 59(3), 145–157 (1999)

    Article  Google Scholar 

  22. Talmi, K., Liu, J.: Eye and gaze tracking for visually controlled interactive stereoscopic displays. Signal Process. Image Commun. 14(10), 799–810 (1999)

    Article  Google Scholar 

  23. Lee, H., Lim, S.Y., Lee, I., Cha, J., Cho, D.-C., Cho, S.: Multi-modal user interaction method based on gaze tracking and gesture recognition. Signal Process. Image Commun. 28(2), 114–126 (2013)

    Article  Google Scholar 

  24. Skodras, E., Kanas, V.G., Fakotakis, N.: On visual gaze tracking based on a single low cost camera. Signal Process. Image Commun. 36, 29–42 (2015)

    Article  Google Scholar 

  25. Min-Allah, N., Qureshi, M.B., Jan, F., Alrashed, S., Taheri, J.: Deployment of real-time systems in the cloud environment. J. Supercomput. 77, 2069–2090 (2020)

    Article  Google Scholar 

  26. Wang, C., Muhammad, J., Wang, Y., He, Z., Sun, Z.: Towards complete and accurate iris segmentation using deep multi-task attention network for non-cooperative iris recognition. IEEE Trans. Inf. Forensics Secur. 15, 2944–2959 (2020)

    Article  Google Scholar 

  27. Ribeiro, E., Uhl, A., Alonso-Fernandez, F.: Iris super-resolution using CNNs: is photo-realism important to iris recognition? IET Biometr. 8(1), 69–78 (2019)

    Article  Google Scholar 

  28. Li, Y.H., Huang, P.J., Juan, Y.: An efficient and robust iris segmentation algorithm using deep learning. Hindawi Mobile Inform. Syst. 2019, 1–14 (2019)

    Article  Google Scholar 

  29. Jan, F., Usman, I., Khan, S.A., Malik, S.A.: A dynamic non-circular iris localization technique for non-ideal data. Comput. Electr. Eng. 40(8), 215–226 (2014)

    Article  Google Scholar 

  30. Jan, F., Usman, I.: Iris segmentation for visible wavelength and near infrared eye images. Optik Int. J. Light Electron. Opt. 125(16), 4274–4282 (2014)

    Article  Google Scholar 

  31. Jan, F., Usman, I., Agha, S.: Iris localization in frontal eye images for less constrained iris recognition systems. Digital Signal Process. 22(6), 971–986 (2012)

    Article  MathSciNet  Google Scholar 

  32. Daugman, J.G.: High confidence visual recognition of persons by a test of statistical independence. IEEE Trans. Pattern Anal. Mach. Intell. 15(11), 1148–1161 (1993)

    Article  Google Scholar 

  33. Luo, Z.: Survey of applications of pupil detection techniques in image and video processing. Int. J. Recent Technol. Eng. 2, 180–181 (2013)

    Google Scholar 

  34. Huang, X., Ti, C., Hou, Q., Tokuta, A., Yang, R.: An experimental study of pupil constriction for liveness detection. In: 2013 IEEE Workshop on Applications of Computer Vision (WACV): 15–17 Jan. 2013, 252–258 (2013)

  35. Yadav, M.R., Shivdas, S.S.: Novel method to localize the pupil in eye gaze tracking systems. Int. J. Emerg. Technol. Comput. Appl. Sci. 14, 52–57 (2014)

    Google Scholar 

  36. Andrew, D.: Eye tracking methodology: theory and practice. Springer Sci Business Media, p. 373 (2007)

  37. MMU_database: MMU database; http://pesona.mmu.edu.my/~ccteo/. Accessed 1 Jan 2021

  38. Holmqvist, K., Nystrom, M., Andersson, R., Dewhurst, R., Jarodzka, H., Weijer, V.D.: Eye tracking: a comprehensive guide to methods and measures. Oxford University Press, Oxford. 22 September 2011: p. 560 (2011)

  39. Hansen, D.W., Ji, Q.: In the eye of the beholder: a survey of models for eyes and gaze. IEEE Trans. Pattern Anal. Mach. Intell. 32(3), 478–500 (2010)

    Article  Google Scholar 

  40. Jan, F.: Segmentation and localization schemes for non-ideal iris biometric systems. Signal Process. 133, 192–212 (2017)

    Article  Google Scholar 

  41. CASIA_database: CASIA database; http://biometrics.idealtest.org/dbDetailForUser.do?id=4. Accessed 1 Jan 2021

  42. IITD_database: IITD database; http://www.iitd.ac.in/. Accessed 1 Jan 2021

  43. WVU_database: WVU database; http://www.clarkson.edu/citer/research/collections/. Accessed 1 Jan 2021

  44. UBIRIS_database: UBIRIS database; http://iris.di.ubi.pt/. Accessed 1 Jan 2021

  45. Daugman, J.: New methods in iris recognition. IEEE Trans. Syst. Man Cybernet. Part B. Cybernet. 37(5), 1167–1175 (2007)

    Article  Google Scholar 

  46. Wildes, R.P.: Iris recognition: an emerging biometric technology. Proc. IEEE 85(9), 1348–1363 (1997)

    Article  Google Scholar 

  47. Jan, F.: Development and analysis of robust iris segmentation algorithms for non ideal iris recognition system. PhD Thesis COMSATS Univeristy Islamabad (2014)

  48. Morimoto, C.H., Koons, D., Amir, A., Flickner, M.: Pupil detection and tracking using multiple light sources. Image Vis. Comput. 18(4), 331–335 (2000)

    Article  Google Scholar 

  49. Yuasa, M., Yamaguchi, O., Fukui, K.: Precise pupil contour detection based on minimizing the energy of pattern and edge. IEICE Trans. 87-D, 105–112 (2004)

    Google Scholar 

  50. Mohammed, G., Hong, B., Alkazzaz, A.: Accurate pupil features extraction based on new projection function. Comput. Informat. 29, 663–680 (2010)

    Google Scholar 

  51. Ramalingam, S.P., Mouli, C.: Pupil segmentation from IRIS images using modified peak detection algorithm. Int. J. Comput. Appl. Technol. 31, 51–56 (2011)

    Google Scholar 

  52. Aydi, W., Masmoudi, N., Kamoun, L.: Active contour without edges Vs GVF active contour for accurate pupil segmentation. Int. J. Comput. Appl. 54(4), 25–36 (2012)

    Google Scholar 

  53. Roig, A.B., Morales, M., Espinosa, J., Perez, J., Mas, D., Illueca, C.: Pupil detection and tracking for analysis of fixational eye micromovements. Optik 123(1), 11–15 (2012)

    Article  Google Scholar 

  54. Krishnamoorthi, R., Indradevi, D.: A new snake model for pupil localization using orthogonal polynomials transform. Int. J. Comput. Theor. Eng. 36–40 (2013)

  55. Markuš, N., Frljak, M., Pandžić, I.S., Ahlberg, J., Forchheimer, R.: Eye pupil localization with an ensemble of randomized trees. Pattern Recogn. 47(2), 578–587 (2014)

    Article  Google Scholar 

  56. Gwon, S., Cho, C., Lee, E.C., Lee, W., Park, K.: Robust eye and pupil detection method for gaze tracking. Int. J. Adv. Rob. Syst. 10, 1–7 (2013)

    Google Scholar 

  57. Abdullah, M., Dlay, Ss., Woo, W.L.: Fast and accurate pupil isolation based on morphology and active contour. (2014)

  58. Leo, M., Cazzato, D., De Marco, T.D., Distante, C.: Unsupervised eye pupil localization through differential geometry and local self-similarity matching. PLoS ONE 9(8), 1–12 (2014)

    Article  Google Scholar 

  59. Wang, J., Zhang, G., Shi, J.: Pupil and glint detection using wearable camera sensor and near-infrared LED array. Sensors (Basel) 15(12), 30126–30141 (2015)

    Article  Google Scholar 

  60. Javadi, A.-H., Hakimi, Z., Barati, M., Walsh, V., Tcheang, L.: SET: a pupil detection method using sinusoidal approximation. Front. Neuroeng. 8(4), 1–10 (2015)

    Google Scholar 

  61. Viola, P., Jones, M.J.: Robust real-time face detection. Int. J. Comput. Vis. 57(2), 137–154 (2004)

    Article  Google Scholar 

  62. Lee, G.J., Jang, S.W., Kim, G.Y.: Pupil center detection using edge and circle characteristics. Adv. Sci. Technol. Lett. 49, 53–58 (2014)

    Google Scholar 

  63. Lin, Z., Yu, H.: The pupil location based on the OTSU method and hough transform. Procedia Environ. Sci. 8, 352–356 (2011)

    Article  Google Scholar 

  64. UPOL_dataset: UPOL dataset; http://www.cbsr.ia.ac.cn:8080/iapr_database.jsp. Accessed 1 Jan 2021

  65. Whitelam, C., Bourlai, T.: On designing an unconstrained tri-band pupil detection system for human identification. Mach. Vis. Appl. 26, 1–19 (2015)

    Article  Google Scholar 

  66. Marco, L., Dario, D., Tommaso, D.M., Cosimo, D.: Unsupervised approach for the accurate localization of the pupils in near-frontal facial images. J. Electron. Imag. 22(3), 033033 (2013)

    Article  Google Scholar 

  67. Jan, F.: Pupil localization in image data acquired with near-infrared or visible wavelength illumination. Multimed. Tools Appl. 77, 1041–1067 (2018)

    Article  Google Scholar 

  68. Jan, F., Usman, I., Agha, S.: Reliable iris localization using Hough transform, histogram-bisection, and eccentricity. Signal Process. 93(1), 230–241 (2013)

    Article  Google Scholar 

  69. Jan, F., Usman, I., Khan, S.A.: Robust iris biometric system for visible wavelength data. Chin. Opt. Lett. 11(8), 081501 (2013)

    Article  Google Scholar 

  70. Jan, F., Usman, I., Agha, S.: Reliable iris localization using integral projection function and 2D-shape properties. Chin. Opt. Lett. 10(11), 111501–111501 (2012)

    Article  Google Scholar 

  71. Ibrahim, M.T., Khan, T.M., Khan, S.A., Khan, M.A., Guan, L.: Iris localization using local histogram and other image statistics. Opt. Lasers Eng. 50(5), 645–654 (2012)

    Article  Google Scholar 

  72. Khan, T.M., Aurangzeb Khan, M., Malik, S.A., Khan, S.A., Bashir, T., Dar, A.H.: Automatic localization of pupil using eccentricity and iris using gradient based method. Opt. Lasers Eng. 49(2), 177–187 (2011)

    Article  Google Scholar 

  73. Basit, A., Javed, M.Y.: Localization of iris in gray scale images using intensity gradient. Opt. Lasers Eng. 45(12), 1107–1114 (2007)

    Article  Google Scholar 

  74. IITK_dataset: IITK dataset; http://www.cse.iitk.ac.in/users/biometrics/. Accessed 1 Jan 2021

  75. JLUBRIRIS_database: JLUBRIRIS database; http://biis.jlu.edu.cn/; http://www.hindawi.com/journals/tswj/2014/670934/. Accessed 1 Jan 2021

  76. UTIRIS_database: UTIRIS database; https://utiris.wordpress.com/2014/03/04/university-of-tehran-iris-image-repository/. Accessed 1 Jan 2021

  77. Jan, F., Usman, I., Agha, S.: A non-circular iris localization algorithm using image projection function and gray level statistics. Optik 124, 3187–3193 (2013)

    Article  Google Scholar 

  78. Shah, S., Ross, A.: Iris segmentation using geodesic active contours. IEEE Trans. Inf. Forensics Secur. 4(4), 824–836 (2009)

    Article  Google Scholar 

  79. Ross, A., Shah, S.: Segmenting non-ideal irises using geodesic active contours. In: Biometric Consortium Conference, 2006 Biometrics Symposium: Special Session on Research at the: Sept. 19 2006-August, pp. 1–6 (2006)

Download references

Acknowledgements

We are thankful to the Biometrics Research Laboratory, Indian Institute of Technology Delhi (IITD), New Delhi, India; Malaysia Multimedia University (MMU), Department of Computer Science SOCIA Lab. Malaysia; and Chinese Academy of Sciences Institute of Automation (CASIA) for getting free access to their databases. Besides, we are also thankful to all organizations, individuals, and/or companies whose material has been used in the composition of this article.

Funding

This research was supported by the Deanship of Scientific Research (DSR), Imam Abdulrahman Bin Faisal University (IAU), Saudi Arabia, for funding under Project 2019-359-CSIT.

Author information

Authors and Affiliations

  1. Department of Computer Science, College of Computer Science and Information Technology, Imam Abdulrahman Bin Faisal University, P.O. Box 1982, Dammam, Saudi Arabia

    Nasro Min-Allah & Farmanullah Jan

  2. Management Information Systems Department, College of Applied Studies and Community Service, Imam Abdulrahman Bin Faisal University, P.O. Box 1982, Dammam, Saudi Arabia

    Saleh Alrashed

Authors
  1. Nasro Min-Allah
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Farmanullah Jan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Saleh Alrashed
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Farmanullah Jan.

Ethics declarations

Conflict of interests

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Min-Allah, N., Jan, F. & Alrashed, S. Pupil detection schemes in human eye: a review. Multimedia Systems 27, 753–777 (2021). https://doi.org/10.1007/s00530-021-00806-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00530-021-00806-5

Keywords

Search

Navigation