SpringerLink
Log in

Effect of artificial intelligence–based computer-aided diagnosis on the screening outcomes of digital mammography: a matched cohort study

  • Breast
  • Published:
European Radiology Aims and scope Submit manuscript

Abstract

Objective

To investigate whether artificial intelligence–based computer-aided diagnosis (AI-CAD) can improve radiologists’ performance when used to support radiologists’ interpretation of digital mammography (DM) in breast cancer screening.

Methods

A retrospective database search identified 3158 asymptomatic Korean women who consecutively underwent screening DM between January and December 2019 without AI-CAD support, and screening DM between February and July 2020 with image interpretation aided by AI-CAD in a tertiary referral hospital using single reading. Propensity score matching was used to match the DM with AI-CAD group in a 1:1 ratio with the DM without AI-CAD group according to age, breast density, experience level of the interpreting radiologist, and screening round. Performance measures were compared with the McNemar test and generalized estimating equations.

Results

A total of 1579 women who underwent DM with AI-CAD were matched with 1579 women who underwent DM without AI-CAD. Radiologists showed higher specificity (96% [1500 of 1563] vs 91.6% [1430 of 1561]; p < 0.001) and lower abnormal interpretation rates (AIR) (4.9% [77 of 1579] vs 9.2% [145 of 1579]; p < 0.001) with AI-CAD than without. There was no significant difference in the cancer detection rate (CDR) (AI-CAD vs no AI-CAD, 8.9 vs 8.9 per 1000 examinations; p = 0.999), sensitivity (87.5% vs 77.8%; p = 0.999), and positive predictive value for biopsy (PPV3) (35.0% vs 35.0%; p = 0.999) according to AI-CAD support.

Conclusions

AI-CAD increases the specificity for radiologists without decreasing sensitivity as a supportive tool in the single reading of DM for breast cancer screening.

Clinical relevance statement

This study shows that AI-CAD could improve the specificity of radiologists’ DM interpretation in the single reading system without decreasing sensitivity, suggesting that it can benefit patients by reducing false positive and recall rates.

Key Points

• In this retrospective-matched cohort study (DM without AI-CAD vs DM with AI-CAD), radiologists showed higher specificity and lower AIR when AI-CAD was used to support decision-making in DM screening.

• CDR, sensitivity, and PPV for biopsy did not differ with and without AI-CAD support.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Abbreviations

AI-CAD:

Artificial intelligence-based computer-aided diagnosis

AIR:

Abnormal interpretation rate

BI-RADS:

Breast Imaging Reporting and Data System

CDR:

Cancer detection rate

DM:

Digital mammography

PPV:

Positive predictive value

References

  1. Duffy SW, Vulkan D, Cuckle H et al (2020) Effect of mammographic screening from age 40 years on breast cancer mortality (UK Age trial): final results of a randomised, controlled trial. Lancet Oncol 21:1165–1172

    Article  PubMed  PubMed Central  Google Scholar 

  2. Lauby-Secretan B, Scoccianti C, Loomis D et al (2015) Breast-cancer screening–viewpoint of the IARC Working Group. N Engl J Med 372:2353–2358

    Article  CAS  PubMed  Google Scholar 

  3. Schünemann HJ, Lerda D, Quinn C et al (2020) Breast cancer screening and diagnosis: a synopsis of the european breast guidelines. Ann Intern Med 172:46–56

    Article  PubMed  Google Scholar 

  4. Pisano ED, Gatsonis C, Hendrick E et al (2005) Diagnostic performance of digital versus film mammography for breast-cancer screening. N Engl J Med 353:1773–1783

    Article  CAS  PubMed  Google Scholar 

  5. Hong S, Song SY, Park B et al (2020) Effect of digital mammography for breast cancer screening: a comparative study of more than 8 million Korean women. Radiology 294:247–255

    Article  PubMed  Google Scholar 

  6. Prummel MV, Muradali D, Shumak R et al (2016) Digital compared with screen-film mammography: measures of diagnostic accuracy among women screened in the ontario breast screening program. Radiology 278:365–373

    Article  PubMed  Google Scholar 

  7. Kerlikowske K, Hubbard RA, Miglioretti DL et al (2011) Comparative effectiveness of digital versus film-screen mammography in community practice in the United States: a cohort study. Ann Intern Med 155:493–502

    Article  PubMed  PubMed Central  Google Scholar 

  8. Majid AS, de Paredes ES, Doherty RD, Sharma NR, Salvador X (2003) Missed breast carcinoma: pitfalls and pearls. Radiographics 23:881–895

    Article  PubMed  Google Scholar 

  9. Houssami N, Hunter K (2017) The epidemiology, radiology and biological characteristics of interval breast cancers in population mammography screening. NPJ Breast Cancer 3:12

    Article  PubMed  PubMed Central  Google Scholar 

  10. Lehman CD, Arao RF, Sprague BL et al (2017) National performance benchmarks for modern screening digital mammography: update from the Breast Cancer Surveillance Consortium. Radiology 283:49–58

    Article  PubMed  Google Scholar 

  11. Giess CS, Wang A, Ip IK, Lacson R, Pourjabbar S, Khorasani R (2019) Patient, radiologist, and examination characteristics affecting screening mammography recall rates in a large academic practice. J Am Coll Radiol 16:411–418

    Article  PubMed  Google Scholar 

  12. Wing P, Langelier MH (2009) Workforce shortages in breast imaging: impact on mammography utilization. AJR Am J Roentgenol 192:370–378

    Article  PubMed  Google Scholar 

  13. Rodriguez-Ruiz A, Lång K, Gubern-Merida A et al (2019) Stand-alone artificial intelligence for breast cancer detection in mammography: comparison with 101 radiologists. J Natl Cancer Inst 111:916–922

    Article  PubMed  PubMed Central  Google Scholar 

  14. Yoon JH, Kim EK (2021) Deep learning-based artificial intelligence for mammography. Korean J Radiol 22:1225–1239

    Article  PubMed  PubMed Central  Google Scholar 

  15. Kim HE, Kim HH, Han BK et al (2020) Changes in cancer detection and false-positive recall in mammography using artificial intelligence: a retrospective, multireader study. Lancet Digit Health 2:e138–e148

    Article  PubMed  Google Scholar 

  16. McKinney SM, Sieniek M, Godbole V et al (2020) International evaluation of an AI system for breast cancer screening. Nature 577:89–94

    Article  CAS  PubMed  Google Scholar 

  17. Rodriguez-Ruiz A, Krupinski E, Mordang JJ et al (2019) Detection of breast cancer with mammography: effect of an artificial intelligence support system. Radiology 290:305–314

    Article  PubMed  Google Scholar 

  18. Pacile S, Lopez J, Chone P, Bertinotti T, Grouin JM, Fillard P (2020) Improving breast cancer detection accuracy of mammography with the concurrent use of an artificial intelligence tool. Radiol Artif Intell 2:e190208

    Article  PubMed  PubMed Central  Google Scholar 

  19. Watanabe AT, Lim V, Vu HX et al (2019) Improved cancer detection using artificial intelligence: a retrospective evaluation of missed cancers on mammography. J Digit Imaging 32:625–637

    Article  PubMed  PubMed Central  Google Scholar 

  20. Larsen M, Aglen CF, Hoff SR, Lund-Hanssen H, Hofvind S (2022) Possible strategies for use of artificial intelligence in screen-reading of mammograms, based on retrospective data from 122,969 screening examinations. Eur Radiol 32:8238–8246

    Article  PubMed  PubMed Central  Google Scholar 

  21. Romero-Martin S, Elias-Cabot E, Raya-Povedano JL, Gubern-Merida A, Rodriguez-Ruiz A, Alvarez-Benito M (2022) Stand-alone use of artificial intelligence for digital mammography and digital breast tomosynthesis screening: a retrospective evaluation. Radiology 302:535–542

    Article  PubMed  Google Scholar 

  22. Salim M, Wåhlin E, Dembrower K et al (2020) External evaluation of 3 commercial artificial intelligence algorithms for independent assessment of screening mammograms. JAMA Oncol 6:1581–1588

    Article  PubMed  PubMed Central  Google Scholar 

  23. Freeman K, Geppert J, Stinton C et al (2021) Use of artificial intelligence for image analysis in breast cancer screening programmes: systematic review of test accuracy. BMJ 374:n1872

    Article  PubMed  PubMed Central  Google Scholar 

  24. Lee SE, Han K, Yoon JH, et al (2022) Depiction of breast cancers on digital mammograms by artificial intelligence-based computer-assisted diagnosis according to cancer characteristics. Eur Radiol 32:7400–7408. https://doi-doiorg.libproxy.samsunghospital.com/10.1007/s00330-022-08718-2

  25. Sickles EA, D’Orsi CJ, Bassett LW, Appleton CM, Berg WA, Burnside ES (2013) ACR BI-RADS® Atlas, Breast imaging reporting and data system. Reston, VA: American College of Radiology 39–48

  26. Yi C, Tang Y, Ouyang R et al (2022) The added value of an artificial intelligence system in assisting radiologists on indeterminate BI-RADS 0 mammograms. Eur Radiol 32:1528–1537

    Article  PubMed  Google Scholar 

  27. Lang K, Dustler M, Dahlblom V, Akesson A, Andersson I, Zackrisson S (2021) Identifying normal mammograms in a large screening population using artificial intelligence. Eur Radiol 31:1687–1692

    Article  PubMed  Google Scholar 

  28. Raya-Povedano JL, Romero-Martin S, Elias-Cabot E, Gubern-Merida A, Rodriguez-Ruiz A, Alvarez-Benito M (2021) AI-based strategies to reduce workload in breast cancer screening with mammography and tomosynthesis: a retrospective evaluation. Radiology 300:57–65

    Article  PubMed  Google Scholar 

  29. Yala A, Schuster T, Miles R, Barzilay R, Lehman C (2019) A Deep learning model to triage screening mammograms: a simulation study. Radiology 293:38–46

    Article  PubMed  Google Scholar 

  30. Shaw CM, Flanagan FL, Fenlon HM, McNicholas MM (2009) Consensus review of discordant findings maximizes cancer detection rate in double-reader screening mammography: Irish National Breast Screening Program experience. Radiology 250:354–362

    Article  PubMed  Google Scholar 

  31. Hofvind S, Geller BM, Rosenberg RD, Skaane P (2009) Screening-detected breast cancers: discordant independent double reading in a population-based screening program. Radiology 253:652–660

    Article  PubMed  Google Scholar 

  32. Taylor P, Potts HW (2008) Computer aids and human second reading as interventions in screening mammography: two systematic reviews to compare effects on cancer detection and recall rate. Eur J Cancer 44:798–807

    Article  PubMed  Google Scholar 

  33. Perry N, Broeders M, de Wolf C, Törnberg S, Holland R, von Karsa L (2008) European guidelines for quality assurance in breast cancer screening and diagnosis. Fourth edition–summary document. Ann Oncol 19:614–622

    Article  CAS  PubMed  Google Scholar 

  34. Dinnes J, Moss S, Melia J, Blanks R, Song F, Kleijnen J (2001) Effectiveness and cost-effectiveness of double reading of mammograms in breast cancer screening: findings of a systematic review. Breast 10:455–463

    Article  CAS  PubMed  Google Scholar 

  35. Posso MC, Puig T, Quintana MJ, Sola-Roca J, Bonfill X (2016) Double versus single reading of mammograms in a breast cancer screening programme: a cost-consequence analysis. Eur Radiol 26:3262–3271

    Article  PubMed  Google Scholar 

  36. Taplin SH, Rutter CM, Elmore JG, Seger D, White D, Brenner RJ (2000) Accuracy of screening mammography using single versus independent double interpretation. AJR Am J Roentgenol 174:1257–1262

    Article  CAS  PubMed  Google Scholar 

  37. Lang K, Hofvind S, Rodriguez-Ruiz A, Andersson I (2021) Can artificial intelligence reduce the interval cancer rate in mammography screening? Eur Radiol 31:5940–5947

    Article  PubMed  PubMed Central  Google Scholar 

  38. Boyd NF, Guo H, Martin LJ et al (2007) Mammographic density and the risk and detection of breast cancer. N Engl J Med 356:227–236

    Article  CAS  PubMed  Google Scholar 

  39. Miglioretti DL, Gard CC, Carney PA et al (2009) When radiologists perform best: the learning curve in screening mammogram interpretation. Radiology 253:632–640

    Article  PubMed  PubMed Central  Google Scholar 

  40. Sala M, Comas M, Macià F, Martinez J, Casamitjana M, Castells X (2009) Implementation of digital mammography in a population-based breast cancer screening program: effect of screening round on recall rate and cancer detection. Radiology 252:31–39

    Article  PubMed  Google Scholar 

  41. Kim YJ, Lee EH, Jun JK et al (2017) Analysis of participant factors that affect the diagnostic performance of screening mammography: a report of the Alliance for Breast Cancer Screening in Korea. Korean J Radiol 18:624–631

    Article  PubMed  PubMed Central  Google Scholar 

  42. Lee EH, Kim KW, Kim YJ et al (2016) Performance of screening mammography: a report of the Alliance for Breast Cancer Screening in Korea. Korean J Radiol 17:489–496

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Funding

The authors state that this work has not received any funding.

Author information

Authors and Affiliations

  1. Department of Radiology and Center for Imaging Science, Samsung Medical Center, Sungkyunkwan University School of Medicine, 81 Irwon-Ro, Gangnam-Gu, Seoul, 06351, Korea

    Haejung Kim, Ji Soo Choi, Eun Sook Ko, Eun Young Ko & Boo-Kyung Han

  2. Department of Digital Health, SAIHST, Sungkyunkwan University, Seoul, Korea

    Ji Soo Choi & Kyunga Kim

  3. Biomedical Statistics Center, Research Institute for Future Medicine, Samsung Medical Center, Seoul, Korea

    Kyunga Kim

  4. Department of Data Convergence & Future Medicine, Sungkyunkwan University School of Medicine, Seoul, Korea

    Kyunga Kim

Authors
  1. Haejung Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ji Soo Choi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kyunga Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Eun Sook Ko
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Eun Young Ko
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Boo-Kyung Han
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ji Soo Choi.

Ethics declarations

Guarantor

The scientific guarantor of this publication is Ji Soo Choi.

Conflict of interest

The authors of this manuscript declare no relationships with any companies, whose products or services may be related to the subject matter of the article.

Statistics and biometry

One of the authors has significant statistical expertise: Kyunga Kim.

Informed consent

Written informed consent was waived by the Institutional Review Board.

Ethical approval

This retrospective study received ethical approval from the Institutional Review Board of Samsung Medical Center (IRB File No.: SMC 2021-06-011).

Study subjects or cohorts overlap

Not applicable.

Methodology

  • retrospective

  • diagnostic or prognostic study

  • performed at one institution

Additional information

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (PDF 170 KB)

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kim, H., Choi, J.S., Kim, K. et al. Effect of artificial intelligence–based computer-aided diagnosis on the screening outcomes of digital mammography: a matched cohort study. Eur Radiol 33, 7186–7198 (2023). https://doi.org/10.1007/s00330-023-09692-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00330-023-09692-z

Keywords

Search

Navigation