Abstract
Automated disease classification could significantly improve the accuracy of prostate cancer diagnosis on MRI, which is a difficult task even for trained experts. Convolutional neural networks (CNNs) have shown some promising results for disease classification on multi-parametric MRI. However, CNNs struggle to extract robust global features about the anatomy which may provide important contextual information for further improving classification accuracy. Here, we propose a novel multi-scale hybrid CNN/transformer architecture with the ability of better contextualising local features at different scales. In our application, we found this to significantly improve performance compared to using CNNs. Classification accuracy is even further improved with a stacked ensemble yielding promising results for binary classification of prostate lesions into clinically significant or non-significant.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Albawi, S., Mohammed, T.A., Al-Zawi, S.: Understanding of a convolutional neural network. In: 2017 International Conference on Engineering and Technology (ICET), pp. 1–6. IEEE (2017)
Aldoj, N., Lukas, S., Dewey, M., Penzkofer, T.: Semi-automatic classification of prostate cancer on multi-parametric MR imaging using a multi-channel 3d convolutional neural network. Eur. Radiol. 30(2), 1243–1253 (2020)
Arshad, M.A., et al.: Discovery of pre-therapy 2-deoxy-2-18 f-fluoro-d-glucose positron emission tomography-based radiomics classifiers of survival outcome in non-small-cell lung cancer patients. Eur. J. Nucl. Med. Mol. Imaging 46(2), 455–466 (2019)
Bass, E., et al.: A systematic review and meta-analysis of the diagnostic accuracy of biparametric prostate MRI for prostate cancer in men at risk. Prostate Cancer and Prostatic Diseases, pp. 1–16 (2020)
Benesty, J., Chen, J., Huang, Y., Cohen, I.: Pearson correlation coefficient. In: Cohen, I., Huang, Y., Chen, J., Benesty, J. (eds.) Noise Reduction in Speech Processing, pp. 1–4. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-00296-0
Brizmohun Appayya, M., et al.: National implementation of multi-parametric magnetic resonance imaging for prostate cancer detection-recommendations from a UK consensus meeting. BJU Int. 122(1), 13–25 (2018)
Castillo, T., et al.: Automated classification of significant prostate cancer on MRI: a systematic review on the performance of machine learning applications. Cancers 12(6), 1606 (2020)
Cordonnier, J.B., Loukas, A., Jaggi, M.: On the relationship between self-attention and convolutional layers. ar**v preprint ar**v:1911.03584 (2019)
Dosovitskiy, A., et al.: An image is worth 16x16 words: transformers for image recognition at scale. ar**v preprint ar**v:2010.11929 (2020)
He, K., Zhang, X., Ren, S., Sun, J.: Delving deep into rectifiers: surpassing human-level performance on ImageNet classification. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1026–1034 (2015)
Hendrycks, D., Gimpel, K.: Gaussian error linear units (GELUs). ar**v preprint ar**v:1606.08415 (2016)
Ishioka, J., et al.: Computer-aided diagnosis of prostate cancer on magnetic resonance imaging using a convolutional neural network algorithm. BJU Int. 122(3), 411–417 (2018)
Khalvati, F., Wong, A., Haider, M.A.: Automated prostate cancer detection via comprehensive multi-parametric magnetic resonance imaging texture feature models. BMC Med. Imaging 15(1), 1–14 (2015)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. ar**v preprint ar**v:1412.6980 (2014)
Litjens, G., Debats, O., Barentsz, J., Karssemeijer, N., Huisman, H.: Computer-aided detection of prostate cancer in MRI. IEEE Trans. Med. Imaging 33(5), 1083–1092 (2014)
Rizzo, S., et al.: Radiomics: the facts and the challenges of image analysis. Eur. Radiol. Exp. 2(1), 1–8 (2018). https://doi.org/10.1186/s41747-018-0068-z
Song, Y., et al.: Computer-aided diagnosis of prostate cancer using a deep convolutional neural network from multiparametric MRI. J. Magn. Reson. Imaging 48(6), 1570–1577 (2018)
St, L., Wold, S., et al.: Analysis of variance (ANOVA). Chemom. Intell. Lab. Syst. 6(4), 259–272 (1989)
Stoyanova, R., et al.: Prostate cancer radiomics and the promise of radiogenomics. Transl. Cancer Res. 5(4), 432 (2016)
Vaswani, A., et al.: Attention is all you need. ar**v preprint ar**v:1706.03762 (2017)
Wang, Z., Liu, C., Cheng, D., Wang, L., Yang, X., Cheng, K.T.: Automated detection of clinically significant prostate cancer in MP-MRI images based on an end-to-end deep neural network. IEEE Trans. Med. Imaging 37(5), 1127–1139 (2018)
**e, S., Girshick, R., Dollar, P., Tu, Z., He, K.: Aggregated residual transformations for deep neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017
Yang, X., et al.: Co-trained convolutional neural networks for automated detection of prostate cancer in multi-parametric MRI. Med. Image Anal. 42, 212–227 (2017)
Yushkevich, P.A., Gao, Y., Gerig, G.: ITK-snap: an interactive tool for semi-automatic segmentation of multi-modality biomedical images. In: 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 3342–3345. IEEE (2016)
Acknowledgments
This work was supported and funded by Cancer Research UK (CRUK) (C309/A28804).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Santhirasekaram, A., Pinto, K., Winkler, M., Aboagye, E., Glocker, B., Rockall, A. (2021). Multi-scale Hybrid Transformer Networks: Application to Prostate Disease Classification. In: Syeda-Mahmood, T., et al. Multimodal Learning for Clinical Decision Support. ML-CDS 2021. Lecture Notes in Computer Science(), vol 13050. Springer, Cham. https://doi.org/10.1007/978-3-030-89847-2_2
Download citation
DOI: https://doi.org/10.1007/978-3-030-89847-2_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-89846-5
Online ISBN: 978-3-030-89847-2
eBook Packages: Computer ScienceComputer Science (R0)
