Abstract
In this study, we aimed not only to analyze model performance of the convolutional neural network (CNN)-based pigmented skin lesion (PSL) classification, but also analyze the analytic validation of the CNN-based PSL classification using unseen PSL hyperspectral dataset with an FNR. To this end, 38 hyperspectral imaging (HSI) data samples were obtained from 19 patients diagnosed with PSLs based on biopsy results. The analytic validation dataset comprised both seen and unseen PSL datasets. The seen PSL dataset included 272,677 pixels from 32 HSI data samples, and the unseen PSL dataset included 370,820 pixels from 38 HSI data samples. A snapshot-based hyperspectral camera captured the spectral (2048 × 2048 pixels) and spatial (150 spectral bands, 470–900 nm) data. A dermatologist labeled the acquired HSI data as pigmented basal cell carcinoma (BCC), melanoma, and squamous cell carcinoma (SCC) to obtain hyperspectral data for each PSL class in software. A confusion matrix and specific performance metrics were used to evaluate CNN-based PSL classification performance. The false negative ratio (FNR) for melanoma were 0.0284 ± 0.0051 and 0.4317 ± 0.0269 for seen and unseen PSL dataset, respectively. Furthermore, 49.14% of the unseen SCC hyperspectral data was predicted as BCC. We confirmed unseen SCC hyperspectral data was most commonly confused for BCC. Therefore, we confirmed the feasibility of analytic validation of the CNN-based PSL classification using unseen PSL hyperspectral dataset for clinical applications.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
H. Sung, J. Ferlay, R.L. Siegel, M. Laversanne, I. Soerjomataram, A. Jemal, F. Bray, Global cancer statistics 2020: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J. Clin. 71, 209–249 (2021). https://doi.org/10.3322/caac.21660
H. Tsao, J.M. Olazagasti, K.M. Cordoro, J.D. Brewer, S.C. Taylor, J.S. Bordeaux, M.M. Chren, A.J. Sober, C. Tegeler, R. Bhushan, W.S. Begolka, Early detection of melanoma: reviewing the ABCDEs. J. Am. Acad. Dermatol. 72, 717–723 (2015). https://doi.org/10.1016/j.jaad.2015.01.025
J.E. Gershenwald, R.A. Scolyer, K.R. Hess, V.K. Sondak, G.V. Long, M.I. Ross, A.J. Lazar, M.B. Faries, J.M. Kirkwood, G.A. McArthur, L.E. Haydu, A.M.M. Eggermont, K.T. Flaherty, C.M. Balch, J.F. Thompson, for members of the American Joint Committee on Cancer Melanoma Expert Panel and the International Melanoma Database and Discovery Platform, melanoma staging: evidence-based changes in the American Joint Committee on Cancer eighth edition cancer staging manual. CA Cancer J. Clin. 67, 472–492 (2017). https://doi.org/10.3322/caac.21409
H. Fabelo, V. Melián, B. Martínez, P. Beltrán, S. Ortega, M. Marrero, G.M. Callicó, R. Sarmiento, I. Castaño, G. Carretero, P. Almeida, A. García, J.A. Hernández, F. Godtliebsen, Dermatologic hyperspectral imaging system for skin cancer diagnosis assistance. In 2019 XXXIV Conference on Design of Circuits and Integrated Systems (DCIS); 20–22 Nov 2019; Bilbao, Spain. IEEE. p. 1
M. Combalia, N. Codella, V. Rotemberg, C. Carrera, S. Dusza, D. Gutman, B. Helba, H. Kittler, N.R. Kurtansky, K. Liopyris, M.A. Marchetti, S. Podlipnik, S. Puig, C. Rinner, P. Tschandl, J. Weber, A. Halpern, J. Malvehy, Validation of artificial intelligence prediction models for skin cancer diagnosis using dermoscopy images: the 2019 International Skin Imaging Collaboration Grand Challenge. Lancet Digit. Health. 4, e330–e339 (2022). https://doi.org/10.1016/S2589-7500(22)00021-8
American Cancer Society, Cancer Facts & Figures 2022 (American Cancer Society, Atlanta, 2022)
N. Melarkode, K. Srinivasan, S.M. Qaisar, P. Plawiak, AI-powered diagnosis of skin cancer: a contemporary review, open challenges and future research directions. Cancers 15, 1183 (2023). https://doi.org/10.3390/cancers15041183
T.J. Brinker, A. Hekler, J.S. Utikal, N. Grabe, D. Schadendorf, J. Klode, C. Berking, T. Steeb, A.H. Enk, C. von Kalle, Skin cancer classification using convolutional neural networks: systematic review. J. Med. Internet Res. 20, e11936 (2018). https://doi.org/10.2196/11936
R.C. Maron, M. Weichenthal, J.S. Utikal, A. Hekler, C. Berking, A. Hauschild, A.H. Enk, S. Haferkamp, J. Klode, D. Schadendorf, P. Jansen, T. Holland-Letz, B. Schilling, C. von Kalle, S. Fröhling, M.R. Gaiser, D. Hartmann, A. Gesierich, K.C. Kähler, U. Wehkamp, A. Karoglan, C. Bär, T.J. Brinker, Systematic outperformance of 112 dermatologists in multiclass skin cancer image classification by convolutional neural networks. Eur. J. Cancer 119, 57–65 (2019). https://doi.org/10.1016/j.ejca.2019.06.013
M.K. Hasan, S. Roy, C. Mondal, M.A. Alam, M.T.E. Elahi, A. Dutta, S.M.T.U. Raju, T. Jawad, M. Ahmad, Dermo-DOCTOR: a framework for concurrent skin lesion detection and recognition using a deep convolutional neural network with end-to-end dual encoders. Biomed. Signal Process. Control 68, 102661 (2021). https://doi.org/10.1016/j.bspc.2021.102661
A. Esteva, B. Kuprel, R.A. Novoa, J. Ko, S.M. Swetter, H.M. Blau, S. Thrun, Dermatologist-level classification of skin cancer with deep neural networks. Nature 542, 115–118 (2017). https://doi.org/10.1038/nature21056
P. Tschandl, N. Codella, B.N. Akay, G. Argenziano, R.P. Braun, H. Cabo, D. Gutman, A. Halpern, B. Helba, R. Hofmann-Wellenhof, A. Lallas, J. Lapins, C. Longo, J. Malvehy, M.A. Marchetti, A. Marghoob, S. Menzies, A. Oakley, J. Paoli, S. Puig, C. Rinner, C. Rosendahl, A. Scope, C. Sinz, H.P. Soyer, L. Thomas, I. Zalaudek, H. Kittler, Comparison of the accuracy of human readers versus machine-learning algorithms for pigmented skin lesion classification: an open, web-based, international, diagnostic study. Lancet Oncol. 20, 938–947 (2019). https://doi.org/10.1016/S1470-2045(19)30333-X
P. Tschandl, C. Rinner, Z. Apalla, G. Argenziano, N. Codella, A. Halpern, M. Janda, A. Lallas, C. Longo, J. Malvehy, J. Paoli, S. Puig, C. Rosendahl, H.P. Soyer, I. Zalaudek, H. Kittler, Human–computer collaboration for skin cancer recognition. Nat. Med. 26, 1229–1234 (2020). https://doi.org/10.1038/s41591-020-0942-0
P. Tschandl, C. Rosendahl, B.N. Akay, G. Argenziano, A. Blum, R.P. Braun, H. Cabo, J.Y. Gourhant, J. Kreusch, A. Lallas, J. Lapins, A. Marghoob, S. Menzies, N.M. Neuber, J. Paoli, H.S. Rabinovitz, C. Rinner, A. Scope, H.P. Soyer, C. Sinz, L. Thomas, I. Zalaudek, H. Kittler, Expert-level diagnosis of nonpigmented skin cancer by combined convolutional neural networks. JAMA Dermatol. 155, 58–65 (2019). https://doi.org/10.1001/jamadermatol.2018.4378
R. Leon, B. Martinez-Vega, H. Fabelo, S. Ortega, V. Melian, I. Castaño, G. Carretero, P. Almeida, A. Garcia, E. Quevedo, J.A. Hernandez, B. Clavo, G.M. Callico, Non-invasive skin cancer diagnosis using hyperspectral imaging for in-situ clinical support. J. Clin. Med. 9, 1662 (2020). https://doi.org/10.3390/jcm9061662
J. Räsänen, M. Salmivuori, I. Pölönen, M. Grönroos, N. Neittaanmäki, Hyperspectral imaging reveals spectral differences and can distinguish malignant melanoma from pigmented basal cell carcinomas: a pilot study. Acta Derm. Venereol. 101, adv00405 (2021). https://doi.org/10.2340/00015555-3755
I. Pölönen, S. Rahkonen, L. Annala, N. Neittaanmäki, Convolutional neural networks in skin cancer detection using spatial and spectral domain, Proc. SPIE 10851, Photonics in Dermatology and Plastic Surgery, 21–28 (2019) https://doi.org/10.1117/12.2509871
B. Martinez-Vega, M. Tkachenko, M. Matkabi, S. Ortega, H. Fabelo, F. Balea-Fernandez, M. La Salvia, E. Torti, F. Leporati, G.M. Callico, C. Chalopin, Evaluation of preprocessing methods on independent medical hyperspectral databases to improve analysis. Sensors 22, 8917 (2022). https://doi.org/10.3390/s22228917
L. Fulton, A. McLeod, D. Dolezel, N. Bastian, C.P. Fulton, Deep vision for breast cancer classification and segmentation. Cancers (Basel) 13, 5384 (2021). https://doi.org/10.3390/cancers13215384
L. Di Biasi, F. De Marco, A. Auriemma Citarella, M. Castrillón-Santana, P. Barra, G. Tortora, Refactoring and performance analysis of the main CNN architectures: using false negative rate minimization to solve the clinical images melanoma detection problem. BMC Bioinform. 24, 386 (2023). https://doi.org/10.1186/s12859-023-05516-5
H. Sujaini, E.Y. Ramadhan, H. Novriando, Comparing the performance of linear regression versus deep learning on detecting melanoma skin cancer using apple core ML. Bull. Electr. Eng. Inform. 10, 3110 (2021). https://doi.org/10.11591/eei.v10i6.3178
T.H. Johansen, K. Møllersen, S. Ortega, H. Fabelo, A. Garcia, G.M. Callico, F. Godtliebsen, Recent advances in hyperspectral imaging for melanoma detection. Wiley Interdiscip. R. Comput. Stat. 12, e1465 (2020). https://doi.org/10.1002/wics.1465
S. Bechelli, J. Delhommelle, Machine learning and deep learning algorithms for skin cancer classification from dermoscopic images. Bioengineering 9, 97 (2022). https://doi.org/10.3390/bioengineering9030097
J. Zhang, Y. Fan, Y. Song, J. Xu, Accuracy of Raman spectroscopy for differentiating skin cancer from normal tissue. Medicine 97, e12022 (2018). https://doi.org/10.1097/MD.0000000000012022
S.K.T. Que, F.O. Zwald, C.D. Schmults, Cutaneous squamous cell carcinoma: incidence, risk factors, diagnosis, and staging. J. Am. Acad. Dermatol. 78, 237–247 (2018). https://doi.org/10.1016/j.jaad.2017.08.059
S. Feinstein, S. Higgins, O. Ahadiat, A. Wysong, A retrospective cohort study of cutaneous squamous cell carcinoma with lymph node metastasis: risk factors and clinical course. Dermatol. Surg. 45, 772–781 (2019). https://doi.org/10.1097/DSS.0000000000001828
A. Jambusaria-Pahlajani, P.A. Kanetsky, P.S. Karia, W.T. Hwang, J.M. Gelfand, F.M. Whalen, R. Elenitsas, X. Xu, C.D. Schmults, Evaluation of AJCC tumor staging for cutaneous squamous cell carcinoma and a proposed alternative tumor staging system. JAMA Dermatol. 149, 402–410 (2013). https://doi.org/10.1001/jamadermatol.2013.2456
I. Zalaudek, J. Giacomel, K. Schmid, S. Bondino, C. Rosendahl, S. Cavicchini, A. Tourlaki, S. Gasparini, P. Bourne, J. Keir, H. Kittler, L. Eibenschutz, C. Catricalà, G. Argenziano, Dermatoscopy of facial actinic keratosis, intraepidermal carcinoma, and invasive squamous cell carcinoma: a progression model. J. Am. Acad. Dermatol. 66, 589–597 (2012). https://doi.org/10.1016/j.jaad.2011.02.011
H. Liu, S. Chen, F. Zhang, B. Shi, Z. Shi, D. Zhang, M. Yu, H. Tian, B. Yang, Seborrheic keratosis or verruca plana? A pilot study with confocal laser scanning microscopy. Skin Res. Technol. 16, 408–412 (2010). https://doi.org/10.1111/j.1600-0846.2010.00444.x
T.H. Ryu, H. Kye, J.E. Choi, H.H. Ahn, Y.C. Kye, S.H. Seo, Features causing confusion between basal cell carcinoma and squamous cell carcinoma in clinical diagnosis. Ann. Dermatol. 30, 64–70 (2018). https://doi.org/10.5021/ad.2018.30.1.64
K. Hauser, A. Kurz, S. Haggenmüller, R.C. Maron, C. von Kalle, J.S. Utikal, F. Meier, S. Hobelsberger, F.F. Gellrich, M. Sergon, A. Hauschild, L.E. French, L. Heinzerling, J.G. Schlager, K. Ghoreschi, M. Schlaak, F.J. Hilke, G. Poch, H. Kutzner, C. Berking, M.V. Heppt, M. Erdmann, S. Haferkamp, D. Schadendorf, W. Sondermann, M. Goebeler, B. Schilling, J.N. Kather, S. Fröhling, D.B. Lipka, A. Hekler, E. Krieghoff-Henning, T.J. Brinker, Explainable artificial intelligence in skin cancer recognition: a systematic review. Eur. J. Cancer 167, 54–69 (2022). https://doi.org/10.1016/j.ejca.2022.02.025
D.C. Araújo, A.A. Veloso, R.S. de Oliveira Filho, M.N. Giraud, L.J. Raniero, L.M. Ferreira, R.A. Bitar, Finding reduced Raman spectroscopy fingerprint of skin samples for melanoma diagnosis through machine learning. Artif. Intell. Med. 120, 102161 (2021). https://doi.org/10.1016/j.artmed.2021.102161
Acknowledgements
We would like to thank Editage (www.editage.co.kr) for English language editing.
Funding
This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. 2021R1G1A1003584) and Korea University (No. K1925071, No. K2310601).
Author information
Authors and Affiliations
Corresponding authors
Ethics declarations
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Heo, E.J., Park, C.G., Chang, K.H. et al. Analytic validation of convolutional neural network-based classification of pigmented skin lesions (PSLs) using unseen PSL hyperspectral data for clinical applications. J. Korean Phys. Soc. 84, 889–897 (2024). https://doi.org/10.1007/s40042-024-01069-9
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s40042-024-01069-9