Abstract
Spatial data augmentation is a standard technique for regularizing deep segmentation networks that are tasked with localizing medical abnormalities. However, a typical spatial augmentation scheme is built upon ad hoc selections of spatial transformation parameters which are not determined by the data set and therefore may not capture spatial variations in the data. For segmentation networks trained in the low-data regime, these ad hoc transformation techniques often fail to encourage better generalization. To address this problem, we propose a variational autoencoder framework for spatial data augmentation. We show how this framework provides a natural, data-driven approach to probabilistic, instance-specific spatial augmentation. Further, we observe that U-Nets trained on data augmented using this framework compare favorably with U-Nets trained using standard spatial augmentation methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Balakrishnan, G., Zhao, A., Sabuncu, M.R., Guttag, J., Dalca, A.V.: VoxelMorph: a learning framework for deformable medical image registration. IEEE Trans. Med. Imaging 38(8), 1788–1800 (2019). https://doi.org/10.1109/TMI.2019.2897538. https://ieeexplore.ieee.org/document/8633930/
Bepler, T., Zhong, E.D., Kelley, K., Brignole, E., Berger, B.: Explicitly disentangling image content from translation and rotation with spatial-VAE. In: Advances in Neural Information Processing Systems, pp. 15409–15419 (2019). http://arxiv.org/abs/1909.11663
Detlefsen, N.S., Hauberg, S.: Explicit disentanglement of appearance and perspective in generative models. In: Advances in Neural Information Processing Systems, pp. 1016–1026 (2019). http://arxiv.org/abs/1906.11881
Hauberg, S., Freifeld, O., Lindbo Larsen, A.B., Fisher, J.W., Hansen, L.K.: Dreaming more data: class-dependent distributions over diffeomorphisms for learned data augmentation. In: Proceedings of 19th International Conference on Artificial Intelligence and Statistics, pp. 342–350 (2016)
Isensee, F., Jaeger, P.F., Kohl, S.A.A., Petersen, J., Maier-Hein, K.H.: nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation. Nat. Methods 18(2), 203–211 (2021). https://doi.org/10.1038/s41592-020-01008-z. https://doi.org/10.1038/s41592-020-01008-zhttp://www.nature.com/articles/s41592-020-01008-z
Jaderberg, M., Simonyan, K., Zisserman, A., Kavukcuoglu, K.: Spatial transformer networks. In: Advances in Neural Information Processing Systems, 2015-January, pp. 2017–2025 (2015)
Kingma, D.P., Welling, M.: Auto-encoding variational bayes. In: 2nd International Conference on Learning Representations, ICLR 2014 - Conference Track Proceedings (ML), pp. 1–14 (2014)
Kuijf, H.J., et al.: Standardized assessment of automatic segmentation of white matter hyperintensities and results of the WMH segmentation challenge. IEEE Trans. Med. Imaging 38(11), 2556–2568 (2019). https://doi.org/10.1109/TMI.2019.2905770
Locatello, F., et al.: A commentary on the unsupervised learning of disentangled representations. In: AAAI 2020–34th AAAI Conference on Artificial Intelligence, pp. 13681–13684 (2020). https://doi.org/10.1609/aaai.v34i09.7120. http://arxiv.org/abs/2007.14184
Locatello, F., et al.: Challenging common assumptions in the unsupervised learning of disentangled representations. In: 36th International Conference on Machine Learning, ICML 2019, pp. 7247–7283 (2019)
Olut, S., Shen, Z., Xu, Z., Gerber, S., Niethammer, M.: Adversarial data augmentation via deformation statistics. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12374, pp. 643–659. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58526-6_38
Orbes, M., et al.: PADDIT: probabilistic augmentation of data using diffeomorphic image transformation. In: Angelini, E.D., Landman, B.A. (eds.) Medical Imaging 2019 Image Processing, vol. 10949, p. 27. SPIE (2019). https://doi.org/10.1117/12.2512520
Qin, C., Shi, B., Liao, R., Mansi, T., Rueckert, D., Kamen, A.: Unsupervised deformable registration for multi-modal images via disentangled representations. In: Chung, A.C.S., Gee, J.C., Yushkevich, P.A., Bao, S. (eds.) IPMI 2019. LNCS, vol. 11492, pp. 249–261. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-20351-1_19
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Schwöbel, P., Warburg, F., Jørgensen, M., Madsen, K.H., Hauberg, S.: Probabilistic Spatial Transformer Networks. ar**v (2020). http://arxiv.org/abs/2004.03637
Shorten, C., Khoshgoftaar, T.M.: A survey on image data augmentation for deep learning. J. Big Data 6(1), 1–48 (2019). https://doi.org/10.1186/s40537-019-0197-0
Shu, Z., Sahasrabudhe, M., Alp Güler, R., Samaras, D., Paragios, N., Kokkinos, I.: Deforming autoencoders: unsupervised disentangling of shape and appearance. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11214, pp. 664–680. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01249-6_40
Sudre, C.H., Li, W., Vercauteren, T., Ourselin, S., Jorge Cardoso, M.: Generalised dice overlap as a deep learning loss function for highly unbalanced segmentations. In: Cardoso, M.J., et al. (eds.) DLMIA/ML-CDS -2017. LNCS, vol. 10553, pp. 240–248. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-67558-9_28
Tang, Z., Chen, K., Pan, M., Wang, M., Song, Z.: An augmentation strategy for medical image processing based on statistical shape model and 3D thin plate spline for deep learning. IEEE Access 7, 133111–133121 (2019). https://doi.org/10.1109/ACCESS.2019.2941154
Uzunova, H., Handels, H., Ehrhardt, J.: Guided filter regularization for improved disentanglement of shape and appearance in diffeomorphic autoencoders. In: Proceedings of Fourth Conference on Medical Imaging with Deep Learning, pp. 774–786. PMLR (2021). https://proceedings.mlr.press/v143/uzunova21a.html%7D
Wyburd, M.K., Dinsdale, N.K., Namburete, A.I.L., Jenkinson, M.: TEDS-Net: enforcing diffeomorphisms in spatial transformers to guarantee topology preservation in segmentations. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12901, pp. 250–260. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87193-2_24
Zhang, H., Cisse, M., Dauphin, Y.N., Lopez-Paz, D.: MixUp: beyond empirical risk minimization. In: 6th International Conference on Learning Representations, ICLR 2018 - Conference Track Proceedings, pp. 1–13 (2018)
Zhao, A., Balakrishnan, G., Durand, F., Guttag, J.V., Dalca, A.V.: Data augmentation using learned transformations for one-shot medical image segmentation. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8535–8545. IEEE (2019). https://doi.org/10.1109/CVPR.2019.00874. https://ieeexplore.ieee.org/document/8953991/
Acknowledgement
This work is supported by Innovation Fund Denmark.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Middleton, J., Bauer, M., Johansen, J., Nielsen, M., Sommer, S., Pai, A. (2023). Instance-Specific Augmentation of Brain MRIs with Variational Autoencoders. In: Fragemann, J., Li, J., Liu, X., Tsaftaris, S.A., Egger, J., Kleesiek, J. (eds) Medical Applications with Disentanglements. MAD 2022. Lecture Notes in Computer Science, vol 13823. Springer, Cham. https://doi.org/10.1007/978-3-031-25046-0_5
Download citation
DOI: https://doi.org/10.1007/978-3-031-25046-0_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-25045-3
Online ISBN: 978-3-031-25046-0
eBook Packages: Computer ScienceComputer Science (R0)
