Abstract
Objectives
To critically appraise methodology and reproducibility of published studies on CT radiomics of pancreatic ductal adenocarcinoma (PDAC).
Methods
PRISMA literature search of MEDLINE, PubMed, and Scopus databases was conducted from June to August 2022 relating to CT radiomics human research articles pertaining to PDAC diagnosis, treatment, and/ or prognosis, utilising Image Biomarker Standardisation Initiative-compliant (IBSI) radiomic software. Keyword search included [pancreas OR pancreatic] AND [radiomic OR [quantitative AND imaging] OR [texture AND analysis]]. Analysis included cohort size, CT protocol used, radiomic feature (RF) extraction, segmentation, and selection, software used, outcome correlation, and statistical methodology, with focus on reproducibility.
Results
Initial search yielded 1112 articles; however, only 12 articles met all inclusion/exclusion criteria. Cohort sizes ranged from 37 to 352 (median = 106, mean = 155.8). CT slice thickness varied among studies (4 using ≤ 1 mm, 5 using > 1 to 3 mm, 2 using > 3 to 5 mm, 1 not specifying). CT protocol varied (5 using a single portal-venous (pv)-phase, 5 using a pancreas protocol, 1 study using a non-contrast protocol). RF extraction and segmentation were heterogeneous (RF extraction: 5 using pv-phase, 2 using late arterial, 4 using multi-phase, 1 using non-contrast phase; RF selection: 3 pre-selected, 9 software-selected). 2D/3D RF segmentation was diverse (2D in 6, 3D in 4, 2D and 3D in 2 studies). Six different radiomics software were used. Research questions and cohort characteristics varied, ultimately leading to non-comparable outcome results.
Conclusion
The current twelve published IBSI-compliant PDAC radiomic studies show high variability and often incomplete methodology resulting in low robustness and reproducibility.
Clinical relevance statement
Radiomics research requires IBSI compliance, data harmonisation, and reproducible feature extraction methods for non-invasive imaging biomarker discoveries to be valid. This will ensure a successful clinical implementation and ultimately an improvement of patient outcomes as part of precision and personalised medicine.
Key Points
• Current state of radiomics research in pancreatic cancer shows low software compliance to the Image Biomarker Standardisation Initiative (IBSI).
• IBSI-compliant radiomics studies in pancreatic cancer are heterogeneous and not comparable, and the majority of study designs showed low reproducibility.
• Improved methodology and standardisation of practice in the emerging field of radiomics has the potential of this non-invasive imaging biomarker in the management of pancreatic cancer.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
Introduction
Pancreatic ductal adenocarcinoma (PDAC) is a malignancy with poor prognosis, frequently presenting at an advanced stage, with a 5-year survival rate of only 11% [1, 2]. Despite advancements in diagnostic and staging technologies such as CT and MR, gains in detection and outcomes have been minimal over the last decades, highlighting the need for more effective and optimised PDAC management.
Radiomics, a novel and promising higher computational method, involves extracting so-called radiomic features (RFs) from medical images that are not discernible by the human eye. The discovery of non-invasive RF-based imaging biomarkers could potentially enable better staging, and lead to improved response to treatment and overall survival as part of precision/personalised medicine [3,4,5].
The extraction of RFs from regions of interest (ROIs) in two (2D) or three dimensions (3D) is common practice in image analysis. RFs can be broadly classified into first, second, and higher order features: First-order features include shape/sphericity, voxel grey intensity, and coarse voxel distribution which can be represented in a histogram that demonstrates skewness, kurtosis, uniformity, and entropy. Second-order features describe the intensity relationships between neighbouring voxels and include characteristics such as grey-level co-occurrence matrix (GLCM) and grey-level run length matrix (GLRLM) [3, 6]. Lastly, higher order features are extracted through mathematical modulation, or filtering techniques, with the goal of supressing noise or highlighting details and patterns [7].
Despite this vast scope, compared to other solid organ cancers, such as the liver and lungs, PDAC radiomics implementation into clinical practice has been limited [11], the reproducibility of radiomic models is highly impacted by variations in scanning parameters, such as scanner model used, pixel spacing, and the contrast administration rate, as well as the manual segmentation of ROIs.
Additionally, there are shortcomings in research methodologies, such as data harmonisation and the use of software that adheres to the Image Biomarker Standardisation Initiative (IBSI) guidelines, within an already-complex clinical environment [7, 12, 13].
This review aims to provide an overview of the current state of primary research in PDAC diagnosis, treatment and prognosis, with a particular focus on radiomics and applied methodology. It highlights areas of strength and weakness in the field, with an emphasis on reproducibility and offers guidance to radiomic researchers to generate more robust results.
Material and methods
Database search (MEDLINE, PubMed, and Scopus)
A literature search of the online databases MEDLINE, PubMed, and Scopus was conducted between June and August 2022. The search formula contained [pancreas OR pancreatic] AND [radiomic OR [quantitative AND imaging] OR [texture AND analysis]].
Titles and abstracts of articles were initially screened by two raters (H.S.K (radiology subspecialist with 16 years abdominal specialty experience), J.A.M (3rd year medical graduate student)). Inclusion criteria included primary human research articles on CT radiomics in PDAC diagnosis, treatment, and/or prognosis published in English between 2017 and August 2022, and studies using non IBSI-compliant software were excluded. Final selection of articles were then analysed by three raters (H.S.K, J.A.M, M.T. (biostatistician)) (Table 1).
Data extraction
Articles were evaluated and categorised according to CT slice thickness (≤ 1 mm, > 1 to 3 mm, > 3 to 5 mm) so to align with prevalent standards and enable comparison of future radiomic investigations. Studies were further subcategorised into various RF clinical applications to address the following research questions: (1) Are there commonly identified RFs across PDAC studies that suggest trends in development of a validated imaging biomarker? (2) What is the reported cohort size, CT technical factors, and described radiomics methodology steps? Are there factors in methods that could impede reproducibility?
Our analysis did not to apply the radiomics quality score (RQS) by Lambin et al [14] given its complex structure (16 components, six key domains, score of 0–36 points) and also based on a recent extensive systematic review comprising 77 articles in high ranking medical journals that demonstrated an overall basic low RQS adherence rate at 38.7%, with many RQS components receiving a score of 0 points [15].
Results
Figure 1 illustrates the PRISMA flow diagram outlining the literature search. A total of 1112 articles were found (MEDLINE n = 584, PubMed n = 144, Scopus n = 384), with duplicates removed resulting in a total of 650 articles. The initial screening of titles and abstracts identified 49 articles that met the eligibility criteria for full-text assessment. After further exclusions, a total of 12 articles were included in this review (Table 2).
Full-text analysis revealed a lack of common RFs, highly variable methodologies, and a lack of sufficient information to ensure reproducibility (Table 3).
Cohort size and CT technique
Patient cohort sizes ranged from 37 to 352 (median = 106, mean = 155.8). Three of the 12 selected studies for review had a small cohort size ranging from 37 to 54 [12]. The 160 + pages framework places a significant emphasis on mathematical and technical aspects, while giving less attention to the clinical implementation. This is likely due to the fact that the majority of authors are of non-clinical backgrounds. As a result, clinicians have to trust and rely on IBSI-compliant radiomics software for quality assurance and reproducibility purposes. This is also underpinned in a study by Fornacon-Wood et al, which found a higher number of RFs exhibiting excellent statistical reliability when extracted using IBSI-compliant software, as opposed to non-compliant software [13]. In our initial eligibility assessment, we identified 9 of 49 studies that used non-IBSI compliant software, and as a result, these studies were excluded from our final analysis.
Statistical extrapolations and radiomic models were noted to be highly variable. For instance, Hang et al [17] sought to correlate RFs of primary PDAC tumours and liver metastases to overall survival by incorporating four texture features into a radiomics score based on a statistical significance of p < 0.1, as opposed to the commonly employed threshold of p < 0.05. Furthermore, the authors failed to provide information on whether selected RFs were considered for or actually included in the multivariable model. Similarly, Attiyeh et al [22] developed two models to predict overall survival in resectable PDAC patients, incorporating additional characteristics such as serum CA19-9 and Brennan pathology scoring. However, the chosen CT contrast phase for RF extraction as well as RF selection was not disclosed for either model or univariate analysis. Two studies that aimed to use radiomics to assess tumour grading are limited by their methodologies and reporting [23, 24]. Chang et al [24] did not disclose details regarding the significant RF selection while Tikhonova et al [23] used a p-value of < 0.1 for statistical significance and assessed contrast enhancement changes in a very small volume of interest (< 1mm3).
Abbreviations
- 2D:
-
Two dimensions
- 3D:
-
Three dimensions
- CT:
-
Computed tomography
- DECT:
-
Dual-energy computed tomography
- DFS:
-
Disease-free survival
- GLCM:
-
Grey-level co-occurrence matrix
- GLRLM:
-
Grey-level run length matrix
- GLZLM:
-
Grey-level zone length matrix
- GRLNU:
-
Grey-level non-uniformity
- IBSI:
-
Image Biomarker Standardisation Initiative
- LASSO:
-
Least absolute shrinkage and selection operator
- NGLDM:
-
Neighbourhood grey-level different matrix
- NID:
-
Neighbourhood intensity difference
- OS:
-
Overall survival
- PDAC:
-
Pancreatic ductal adenocarcinoma
- PFS:
-
Progression-free survival
- PV:
-
Portal venous
- RF:
-
Radiomic feature
- RLM:
-
Run length matrix
- ROC:
-
Receiver operating curve
- ROI:
-
Region of interest
- RQS:
-
Radiomics quality score
- SBRT:
-
Stereotactic body radiation therapy
- SMV:
-
Superior mesenteric vein
- TTP:
-
Time to progression
References
Rawla P, Sunkara T, Gaduputi V (2019) Epidemiology of pancreatic cancer: global trends, etiology and risk factors. World J Oncol 10(1):10–27
Siegel RL, Miller KD, Fuchs HE, Jemal A (2022) Cancer statistics, 2022. CA Cancer J Clin 72(1):7–33
Marti-Bonmati L, Cerdá-Alberich L, Pérez-Girbés A et al (2022) Pancreatic cancer, radiomics and artificial intelligence. Br J Radiol 95(1137):20220072
Abunahel BM, Pontre B, Kumar H, Petrov MS (2021) Pancreas image mining: a systematic review of radiomics. Eur Radiol 31(5):3447–3467
Casà C, Piras A, D’Aviero A et al (2022) The impact of radiomics in diagnosis and staging of pancreatic cancer. Ther Adv Gastrointest Endosc 15:26317745221081596
Bartoli M, Barat M, Dohan A et al (2020) CT and MRI of pancreatic tumors: an update in the era of radiomics. Jpn J Radiol 38(12):1111–1124
Rizzo S, Botta F, Raimondi S et al (2018) Radiomics: the facts and the challenges of image analysis. Eur Radiol Exp 2(1):36
Bibault JE, **ng L, Giraud P et al (2020) Radiomics: a primer for the radiation oncologist. Cancer Radiother 24(5):403–410
Park HJ, Park B, Lee SS (2020) Radiomics and deep learning: hepatic applications. Korean J Radiol 21(4):387–401
Yang F, Zhang J, Zhou L et al (2022) CT-based radiomics signatures can predict the tumor response of non-small cell lung cancer patients treated with first-line chemotherapy and targeted therapy. Eur Radiol 32(3):1538–1547
Yamashita R, Perrin T, Chakraborty J, Chou JF, Horvat N, Koszalka MA et al (2020) Radiomic feature reproducibility in contrast-enhanced CT of the pancreas is affected by variabilities in scan parameters and manual segmentation. Eur Radiol 30(1):195–205
Zwanenburg A, Vallieres M, Abdalah MA et al (2020) The image biomarker standardization initiative: standardized quantitative radiomics for high-throughput image-based phenoty**. Radiology 295(2):328–338
Fornacon-Wood I, Mistry H, Ackermann CJ et al (2020) Reliability and prognostic value of radiomic features are highly dependent on choice of feature extraction platform. Eur Radiol 30(11):6241–6250
Lambin P, Leijenaar RTH, Deist TM, Peerlings J, de Jong EEC, van Timmeren J et al (2017) Radiomics: the bridge between medical imaging and personalized medicine. Nat Rev Clin Oncol 14(12):749–762
Park JE, Kim D, Kim HS et al (2020) Quality of science and reporting of radiomics in oncologic studies: room for improvement according to radiomics quality score and TRIPOD statement. Eur Radiol 30(1):523–536
Chen F, Zhou Y, Qi X, **a W, Zhang R, Zhang J et al (2021) CT texture analysis for the presurgical prediction of superior mesenteric-portal vein invasion in pancreatic ductal adenocarcinoma: comparison with CT imaging features. Clin Radiol 76(5):358–366
Hang J, Xu K, Yin R et al (2021) Role of CT texture features for predicting outcome of pancreatic cancer patients with liver metastases. J Cancer 12(8):2351–2358
Gregucci F, Fiorentino A, Mazzola R et al (2022) Radiomic analysis to predict local response in locally advanced pancreatic cancer treated with stereotactic body radiation therapy. Radiol Med 127(1):100–107
Healy GM, Salinas-Miranda E, Jain R et al (2022) Pre-operative radiomics model for prognostication in resectable pancreatic adenocarcinoma with external validation. Eur Radiol 32(4):2492–2505
Salinas-Miranda E, Khalvati F, Namdar K et al (2021) Validation of prognostic radiomic features from resectable pancreatic ductal adenocarcinoma in patients with advanced disease undergoing chemotherapy. Can Assoc Radiol J 72(4):605–613
Cozzi L, Comito T, Fogliata A et al (2019) Computed tomography based radiomic signature as predictive of survival and local control after stereotactic body radiation therapy in pancreatic carcinoma. PLoS One 14(1):1–11
Attiyeh MA, Chakraborty J, Doussot A et al (2018) Survival prediction in pancreatic ductal adenocarcinoma by quantitative computed tomography image analysis. Ann Surg Oncol 25(4):1034–1042
Tikhonova VS, Karmazanovsky GG, Kondratyev EV et al (2022) Radiomics model-based algorithm for preoperative prediction of pancreatic ductal adenocarcinoma grade. Eur Radiol 33:1152–1161
Chang N, Cui L, Luo Y, Chang Z, Yu B, Liu Z (2020) Development and multicenter validation of a CT-based radiomics signature for discriminating histological grades of pancreatic ductal adenocarcinoma. Quant Imaging Med Surg 10(3):692–702
Shi H, Wei Y, Cheng S et al (2021) Survival prediction after upfront surgery in patients with pancreatic ductal adenocarcinoma: radiomic, clinic-pathologic and body composition analysis. Pancreatology 21(4):731–737
Khalvati F, Zhang Y, Baig S et al (2019) Prognostic value of CT radiomic features in resectable pancreatic ductal adenocarcinoma. Sci Rep 9(1):5449
Cai X, Gao F, Qi Y et al (2020) Pancreatic adenocarcinoma: quantitative CT features are correlated with fibrous stromal fraction and help predict outcome after resection. Eur Radiol 30(9):5158–5169
Almeida LS, Teixeira CJ, Campos CV et al (2022) Low birth weight intensifies changes in markers of hepatocarcinogenesis induced by fructose consumption in rats. Metabolites 12(10):886
Steyerberg EW (2019) Clinical prediction models: a practical approach to development, validation, and updating / Ewout W. Steyerberg. Second edition. ed: Springer
Al-Hawary M (2016) Role of imaging in diagnosing and staging pancreatic cancer. J Natl Compr Canc Netw 14(5 Suppl):678–680
Tempero MA, Malafa MP, Al-Hawary M et al (2021) Pancreatic adenocarcinoma, Version 2.2021, NCCN Clinical Practice Guidelines in Oncology. J Natl Compr Canc Netw 19(4):439–57
Almeida RR, Lo GC, Patino M, Bizzo B, Canellas R, Sahani DV (2018) Advances in pancreatic CT imaging. AJR Am J Roentgenol 211(1):52–66
Noda Y, Tochigi T, Parakh A, Joseph E, Hahn PF, Kambadakone A (2021) Low keV portal venous phase as a surrogate for pancreatic phase in a pancreatic protocol dual-energy CT: feasibility, image quality, and lesion conspicuity. Eur Radiol 31(9):6898
He L, Huang Y, Ma Z, Liang C, Liang C, Liu Z (2016) Effects of contrast-enhancement, reconstruction slice thickness and convolution kernel on the diagnostic performance of radiomics signature in solitary pulmonary nodule. Sci Rep 6:34921
van Timmeren JE, Cester D, Tanadini-Lang S, Alkadhi H, Baessler B (2020) Radiomics in medical imaging-"how-to" guide and critical reflection. Insights Imaging 11(1):91
Acknowledgements
We would like to acknowledge the support of our families and friends and of Cancer Imaging director Prof. Stephen Stuckey at Peter MacCallum Cancer Centre, Melbourne, Victoria, Australia.
Funding
Open Access funding enabled and organised by CAUL and its Member Institutions. The authors state that this work has not received any funding.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Guarantor
The scientific guarantor of this publication is Dr. Hyun Soo Ko.
Conflict of interest
The authors of this manuscript declare no relationships with any companies, whose products or services may be related to the subject matter of the article.
Statistics and biometry
One of the authors is a licenced academic biostatistician.
Informed consent
Written informed consent was not required for this study because it did not involve any living (human, animal, plant) subjects.
Ethical approval
Institutional Review Board approval was not required because it involved a systematic analysis of publically available data/publications.
Study subjects or cohorts overlap
NA
Methodology
• retrospective
• observational
• systematic analysis performed at one institution
Additional information
Publisher's note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Belinda Lee and Hyun Soo Ko are joint senior authors.
Supplementary Information
Below is the link to the electronic supplementary material.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Malcolm, J.A., Tacey, M., Gibbs, P. et al. Current state of radiomic research in pancreatic cancer: focusing on study design and reproducibility of findings. Eur Radiol 33, 6659–6669 (2023). https://doi.org/10.1007/s00330-023-09653-6
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00330-023-09653-6