Abstract
Purpose
To develop a deep learning approach based on deep residual neural network (ResNet101) for the automated detection of glaucomatous optic neuropathy (GON) using color fundus images, understand the process by which the model makes predictions, and explore the effect of the integration of fundus images and the medical history data from patients.
Methods
A total of 34,279 fundus images and the corresponding medical history data were retrospectively collected from cohorts of 2371 adult patients, and these images were labeled by 8 glaucoma experts, in which 26,585 fundus images (12,618 images with GON-confirmed eyes, 1114 images with GON-suspected eyes, and 12,853 NORMAL eye images) were included. We adopted 10-fold cross-validation strategy to train and optimize our model. This model was tested in an independent testing dataset consisting of 3481 images (1524 images from NORMAL eyes, 1442 images from GON-confirmed eyes, and 515 images from GON-suspected eyes) from 249 patients. Moreover, the performance of the best model was compared with results obtained by two experts. Accuracy, sensitivity, specificity, kappa value, and area under receiver operating characteristic (AUC) were calculated. Further, we performed qualitative evaluation of model predictions and occlusion testing. Finally, we assessed the effect of integrating medical history data in the final classification.
Results
In a multiclass comparison between GON-confirmed eyes, GON-suspected eyes and NORMAL eyes, our model achieved 0.941 (95% confidence interval [CI], 0.936–0.946) accuracy, 0.957 (95% CI, 0.953–0.961) sensitivity, and 0.929 (95% CI, 0.923–0.935) specificity. The AUC distinguishing referrals (GON-confirmed and GON-suspected eyes) from observation was 0.992 (95% CI, 0.991–0.993). Our best model had a kappa value of 0.927, while the two experts’ kappa values were 0.928 and 0.925 independently. The best 2 binary classifiers distinguishing GON-confirmed/GON-suspected eyes from NORMAL eyes obtained 0.955, 0.965 accuracy, 0.977, 0.998 sensitivity, and 0.929, 0.954 specificity, while the AUC was 0.992, 0.999 respectively. Additionally, the occlusion testing showed that our model identified the neuroretinal rim region, retinal nerve fiber layer (RNFL) defect areas (superior or inferior) as the most important parts for the discrimination of GON, which evaluated fundus images in a way similar to clinicians. Finally, the results of integration of fundus images with medical history data showed a slight improvement in sensitivity and specificity with similar AUCs.
Conclusions
This approach could discriminate GON with high accuracy, sensitivity, specificity, and AUC using color fundus photographs. It may provide a second opinion on the diagnosis of glaucoma to the specialist quickly, efficiently and at low cost, and assist doctors and the public in large-scale screening for glaucoma.
Similar content being viewed by others
References
Al-Bander B, Williams BM, Al-Nuaimy W et al (2018) Dense fully convolutional segmentation of the optic disc and cup in colour fundus for glaucoma diagnosis. Symmetry-Basel 10(4):87. https://doi.org/10.3390/sym10040087
Bock R, Meier J, Nyul LG et al (2010) Glaucoma risk index: automated glaucoma detection from color fundus images. Med Image Anal 14(3):471–481. https://doi.org/10.1016/j.media.2009.12.006
Tham YC, Li X, Wong TY et al (2014) Global prevalence of glaucoma and projections of glaucoma burden through 2040: a systematic review and meta-ayalysis. Ophthalmology 121(11):2081–2090. https://doi.org/10.1016/j.ophtha.2014.05.013
Quigley HA, Broman AT (2006) The number of people with glaucoma worldwide in 2010 and 2020. Br J Ophthalmol 90(3):262–267. https://doi.org/10.1136/bjo.2005.081224
Orlando JI, Prokofyeva E, Fresno M et al (2017) Convolutional neural network transfer for automated glaucoma identification. Proceeding of SPIE 10160:UNSP 101600U. https://doi.org/10.1117/12.2255740
Rashmi P, Niladri B, Aparna R et al (2018) Deep convolutional neural network-based path classification for retinal nerve fiber layer defect detection in early glaucoma. J Med Imag 5(4):044003. https://doi.org/10.1117/1.JMI.5.4.044003
Lee CS, Baughman DM, Lee AY (2017) Deep learning is effective for classifying normal versus age-related macular degeneration optical coherence tomography images. Ophthalmol Retina 1(4):322–327. https://doi.org/10.1016/j.oret.2016.12.009
Li Z, He YF, Keel S et al (2018) Efficacy of a deep learning system for detecting glaucomatous optic neuropathy based on color fundus photographs. Ophthalmology 125(8):1199–1206. https://doi.org/10.1016/j.ophtha.2018.01.023
Phan S, Satoh S, Yoda Y et al (2019) Evaluation of deep convolutional neural networks for glaucoma detection. Jpn J Ophthalmol 63(3):276–283. https://doi.org/10.1007/s10384-019-00659-6
Shibata N, Tanito M, Mitsuashi K et al (2018) Development of a deep residual learning algorithm to screen for glaucoma from fundus photography. Sci Rep 8:14665 https://www.nature.com/articles/s41598-018-33013-w.pdf
Diaz-Pinto MS, Naranjo V et al (2019) CNNs for automatic glaucoma assessment using fundus images: an extensive validation. Biomed Eng Online 18:29. https://doi.org/10.1186/s12938-019-0649-y
Turajlic E, Karahodzic V (2017) An adaptive scheme for X-ray medical image denoising using artificial neural networks and additive white gaussian noise level estimation in SVD domain. CMBEBIH 62:36–40. https://doi.org/10.1007/978-981-10-4166-2_7
Armaly MF, Sayegh RE (1969) The cup/disc ratio: the findings of tonometry and tonography in the normal eye. Arch Ophthalmol 82:191–196. https://doi.org/10.1001/archopht.1969.00990020193008
Jonas JB, Gusek GC, Naumann GO (1988) Optic disc, cup and neuro-retinal rim size, configuration, and correlations in normal eyes. Invest Ophthalmol Vis Sci 29:1151–1158 http://iovs.arvojournals.org/data/journals/iovs/933371/1151.pdf
Deng Z, Li Y, Wang Y et al (2017) Tracking within hadronic showers in the CALICE SDHCAL prototype using a Hough transform technique. J Instrum 12:P05009 https://biblio.ugent.be/publication/8568622/file/8568804.pdf
Han DM, Liu QG, Fan WG (2018) A new image classification method using CNN transfer learning and web data augmentation. Expert Syst Appl 95:43–56. https://doi.org/10.1016/j.eswa.2017.11.028
Mahapatra D, Roy PK, Sedai S et al (2016) Retinal image quality classification using saliency maps and CNNs. Machine Learning in Medical Imaging 10019:172–179. https://doi.org/10.1007/978-3-319-47157-0_21
Yao LY, Li T, Ye YS et al (2016) Image super-resolution based on self-similarity and various patch size. ICDIP 10033(1):UNSP 100334F. https://doi.org/10.1117/12.2243771
He KM, Zhang XY, Ren SQ et al (2016) Deep residual learning for image recognition. P- ublish In CVPR 16541111:770–778. https://doi.org/10.1109/CVPR.2016.90
Paoletti ME, Haut JM, Plaza J et al (2018) Deep & dense convolutional neural network for hyperspectral image classification. Remote Sens 10(9):1454. https://doi.org/10.3390/rs10091454
Medeiros FA, Alessandro A et al (2019) From machine to machine an OCT-trained deep learning algorithm for objective quantification of glaucomatous damage in fundus photographs. Ophthalmol 126(4):513–521. https://doi.org/10.1016/j.ophtha.2018.12.033
Keel S, Wu JR, Lee PY et al (2019) Visualizing deep learning models for the detection of referable diabetic retinopathy and glaucoma. JAMA Ophthalmol 137(3):288–292. https://doi.org/10.1001/jamaophthalmol.2018.6035
Christopher M, Belghith A, Bowd C et al (2018) Performance of deep learning architectures and transfer learning for detecting glaucomatous optic neuropathy in fundus photographs. Sci Rep 8:16685 https://www.nature.com/articles/s41598-018-35044-9.pdf
Gomez-Valverde JJ, Anton A, Fatti G et al (2019) Automatic glaucoma classification using color fundus images based on convolutional neural networks and transfer learning. Biomed Opt Express 10(2):892–913. https://doi.org/10.1364/BOE.10.000892
Lima ACD, Maia LB, Pereira RMP et al (2018) Glaucoma diagnosis over eye fundus image through deep features. 25th international conference on systems, signals and image processing (IWSSIP). Accessed 22 JUN 2018. https://doi.org/10.1109/IWSSIP.2018.8439477
Coleman AL, Miglior S (2008) Risk factors for glaucoma onset and progression. Surv Ophthalmol 53(6 Suppl 1):S3–S10. https://doi.org/10.1016/j.survophthal.2008.08.006
Weinreb RN, Aung T, Medeiros FA (2014) The pathophysiology and treat of glaucoma: a review. Jama 311(18):1901–1911. https://doi.org/10.1001/jama.2014.3192
Shen DG, Wu GR, Suk H et al (2017) Deep learning in medical image analysis. Annu Rev Biomed Eng 19:221–248 http://europepmc.org/backend/ptpmcrender.fcgi?accid=PMC5479722&blobtype=pdf
Miller SE, Thapa S, Robin AL et al (2017) Glaucoma screening in Nepal: cup-to-disc estimate with standard Mydriatic fundus camera compared to portable nonmydriatic camera. Am J Ophthalmol 182:99–106. https://doi.org/10.1016/j.ajo.2017.07.010
Krishnan MMR, Faust O (2013) Automated glaucoma detection using hybrid feature extraction in retinal fundus images. J Mech Med Biol 13(01):135001. https://doi.org/10.1142/S0219519413500115
Maheshwari S, Pachori RB, Acharya UR (2017) Automated diagnosis of glaucoma using empirical wavelet transform and correntropy features extracted from fundus images. IEEE J Biomed Health Inform 21(3):803–813. https://doi.org/10.1109/JBHI.2016.2544961
Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. International Conference on Learning Representations Accessed 9 ay 2015. https://arxiv.org/pdf/1409.1556.pdf
Son J, Shin JY, Kim HD et al (2019) Development and validation of deep learning models for screening multiple abnormal findings in retinal fundus images. Ophthalmology Accessed 24 May 2019. https://doi.org/10.1016/j.ophtha.2019.05.029
Chalakkal RJ, Abdulla WH, Thulaseedharan SS et al (2019) Quality and content analysis of fundus images using deep learning. Comput Biol Med 108:317–331. https://doi.org/10.1016/j.compbiomed.2019.03.019
Ye DP, Jiang SZ, Li SY et al (2019) Faster and transferable deep learning steganalysis on GPU. J Real-Time Image Proc 16(3):623–633. https://doi.org/10.1007/s11554-019-00870-1
Tsai IL, Tsai CY, Kuo LL et al (2012) Transient changes of intraocular pressure and anterior segment configuration after diagnostic mydriasis with 1% tropicamide in children. Clin Exp Optom 95(2):166–172. https://doi.org/10.1111/j.1444-0938.2011.00677.x
Funding
This study was funded by the National Key Research and Development Program of China (2016YFF0101400), the National Natural Science Foundation of China (51675321), and the National Natural Science Foundation of China (61905144).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Ethical approval
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.
Informed consent
Informed consent was obtained from all individual participants included in the study.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Li, F., Yan, L., Wang, Y. et al. Deep learning-based automated detection of glaucomatous optic neuropathy on color fundus photographs. Graefes Arch Clin Exp Ophthalmol 258, 851–867 (2020). https://doi.org/10.1007/s00417-020-04609-8
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00417-020-04609-8