Abstract
Echocardiography is one of the main imaging modalities used to assess the cardiovascular health of patients. Among the many analyses performed on echocardiography, segmentation of left ventricle is crucial to quantify the clinical measurements like ejection fraction. However, segmentation of left ventricle in 3D echocardiography remains a challenging and tedious task. In this paper, we propose a multi-frame attention network to improve the performance of segmentation of left ventricle in 3D echocardiography. The multi-frame attention mechanism allows highly correlated spatiotemporal features in a sequence of images that come after a target image to be used to augment the performance of segmentation. Experimental results shown on 51 in vivo porcine 3D+time echocardiography images show that utilizing correlated spatiotemporal features significantly improves the performance of left ventricle segmentation when compared to other standard deep learning-based medical image segmentation models.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Ahn, S.S., Ta, K., Lu, A., Stendahl, J.C., Sinusas, A.J., Duncan, J.S.: Unsupervised motion tracking of left ventricle in echocardiography. In: Medical Imaging 2020: Ultrasonic Imaging and Tomography, vol. 11319, p. 113190Z. International Society for Optics and Photonics (2020)
Chen, C., et al.: Deep learning for cardiac image segmentation: a review. Front. Cardiovasc. Med. 7, 25 (2020)
Compas, C.B., et al.: Radial basis functions for combining shape and speckle tracking in 4d echocardiography. IEEE Trans. Med. Imaging 33(6), 1275–1289 (2014)
Dong, S., et al.: Deep atlas network for efficient 3d left ventricle segmentation on echocardiography. Med. Image Anal. 61, 101638 (2020)
Dong, S., Luo, G., Wang, K., Cao, S., Li, Q., Zhang, H.: A combined fully convolutional networks and deformable model for automatic left ventricle segmentation based on 3d echocardiography. BioMed Res. Int. 2018 (2018)
Ghorbani, A., et al.: Deep learning interpretation of echocardiograms. NPJ Dig. Med. 3(1), 1–10 (2020)
Huang, X., et al.: Contour tracking in echocardiographic sequences via sparse representation and dictionary learning. Med. Image Anal. 18(2), 253–271 (2014)
Leclerc, S., et al.: Deep learning for segmentation using an open large-scale dataset in 2d echocardiography. IEEE Trans. Med. Imaging 38(9), 2198–2210 (2019)
Liu, F., Wang, K., Liu, D., Yang, X., Tian, J.: Deep pyramid local attention neural network for cardiac structure segmentation in two-dimensional echocardiography. Med. Image Anal. 67, 10187 (2021)
Lu, X., et al.: See more, know more: unsupervised video object segmentation with co-attention siamese networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3623–3632 (2019)
Luong, M.T., et al.: Effective approaches to attention-based neural machine translation. ar**v preprint ar**v:1508.04025 (2015)
Ouyang, D., et al.: Video-based ai for beat-to-beat assessment of cardiac function. Nature 580(7802), 252–256 (2020)
Papademetris, X., Sinusas, A.J., Dione, D.P., Constable, R.T., Duncan, J.S.: Estimation of 3-d left ventricular deformation from medical images using biomechanical models. IEEE Trans. Med. Imaging 21(7), 786–800 (2002)
Parajuli, N., et al.: Flow network tracking for spatiotemporal and periodic point matching: Applied to cardiac motion analysis. Med. Image Anal. 55, 116–135 (2019)
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Schlemper, J., et al.: Attention gated networks: learning to leverage salient regions in medical images. Med. Image Anal. 53, 197–207 (2019)
Shi, P., Sinusas, A.J., Constable, R.T., Ritman, E., Duncan, J.S.: Point-tracked quantitative analysis of left ventricular surface motion from 3-d image sequences. IEEE Trans. Med. Imaging 19(1), 36–50 (2000)
Stendahl, J.C., et al.: Regional myocardial strain analysis via 2d speckle tracking echocardiography: validation with sonomicrometry and correlation with regional blood flow in the presence of graded coronary stenoses and dobutamine stress. Cardiovasc. Ultrasound 18(1), 1–16 (2020)
Ta, K., Ahn, S.S., Lu, A., Stendahl, J.C., Sinusas, A.J., Duncan, J.S.: A semi-supervised joint learning approach to left ventricular segmentation and motion tracking in echocardiography. In: 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI), pp. 1734–1737. IEEE (2020)
Taha, A.A., Hanbury, A.: Metrics for evaluating 3d medical image segmentation: analysis, selection, and tool. BMC Med. Imaging 15(1), 1–28 (2015)
Virani, S.S., et al.: Heart disease and stroke statistics-2020 update: a report from the American heart association. Circulation 141, E139–E596 (2020)
Wu, L., et al.: Deep coattention-based comparator for relative representation learning in person re-identification. IEEE Trans. Neural Netw. Learn. Syst. 32, 722–735 (2020)
Wu, Y., He, K.: Group normalization. In: Proceedings of the European conference on computer vision (ECCV), pp. 3–19 (2018)
Xu, H., Saenko, K.: Ask, attend and answer: exploring question-guided spatial attention for visual question answering. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9911, pp. 451–466. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46478-7_28
Xu, K., et al.: Show, attend and tell: neural image caption generation with visual attention. In: International Conference on Machine Learning, pp. 2048–2057 (2015)
Zhang, J., et al.: Fully automated echocardiogram interpretation in clinical practice: feasibility and diagnostic accuracy. Circulation 138(16), 1623–1635 (2018)
Acknowledgments
We would like to thank the technical assistance provided by the staff of the Yale Translational Research Imaging Center. This work was supported in part by the following grants: R01HL121226, R01HL137365, F30HL158154, and Medical Scientist Training Program Grant T32GM007205.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Ahn, S.S., Ta, K., Thorn, S., Langdon, J., Sinusas, A.J., Duncan, J.S. (2021). Multi-frame Attention Network for Left Ventricle Segmentation in 3D Echocardiography. In: de Bruijne, M., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2021. MICCAI 2021. Lecture Notes in Computer Science(), vol 12901. Springer, Cham. https://doi.org/10.1007/978-3-030-87193-2_33
Download citation
DOI: https://doi.org/10.1007/978-3-030-87193-2_33
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-87192-5
Online ISBN: 978-3-030-87193-2
eBook Packages: Computer ScienceComputer Science (R0)
