Abstract
Purpose
Manual annotations for training deep learning models in auto-segmentation are time-intensive. This study introduces a hybrid representation-enhanced sampling strategy that integrates both density and diversity criteria within an uncertainty-based Bayesian active learning (BAL) framework to reduce annotation efforts by selecting the most informative training samples.
Methods
The experiments are performed on two lower extremity datasets of MRI and CT images, focusing on the segmentation of the femur, pelvis, sacrum, quadriceps femoris, hamstrings, adductors, sartorius, and iliopsoas, utilizing a U-net-based BAL framework. Our method selects uncertain samples with high density and diversity for manual revision, optimizing for maximal similarity to unlabeled instances and minimal similarity to existing training data. We assess the accuracy and efficiency using dice and a proposed metric called reduced annotation cost (RAC), respectively. We further evaluate the impact of various acquisition rules on BAL performance and design an ablation study for effectiveness estimation.
Results
In MRI and CT datasets, our method was superior or comparable to existing ones, achieving a 0.8% dice and 1.0% RAC increase in CT (statistically significant), and a 0.8% dice and 1.1% RAC increase in MRI (not statistically significant) in volume-wise acquisition. Our ablation study indicates that combining density and diversity criteria enhances the efficiency of BAL in musculoskeletal segmentation compared to using either criterion alone.
Conclusion
Our sampling method is proven efficient in reducing annotation costs in image segmentation tasks. The combination of the proposed method and our BAL framework provides a semi-automatic way for efficient annotation of medical image datasets.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Loureiro A, Mills PM, Barrett RS (2013) Muscle weakness in hip osteoarthritis: a systematic review. Arthritis Care Res 65(3):340–352
Uemura K, Takao M, Sakai T, Nishii T, Sugano N (2016) Volume increases of the gluteus maximus, gluteus medius, and thigh muscles after hip arthroplasty. J Arthroplast 31(4):906–912
Ogawa T, Takao M, Otake Y, Yokota F, Hamada H, Sakai T, Sato Y, Sugano N (2020) Validation study of the CT-based cross-sectional evaluation of muscular atrophy and fatty degeneration around the pelvis and the femur. J Orthop Sci 25(1):139–144
Yagi M, Taniguchi M, Tateuchi H, Hirono T, Fukumoto Y, Yamagata M, Nakai R, Yamada Y, Kimura M, Ichihashi N (2022) Age-and sex-related differences of muscle cross-sectional area in iliocapsularis: a cross-sectional study. BMC Geriatr 22(1):435
Sourati J, Gholipour A, Dy JG, Kurugol S, Warfield SK (2018) Active deep learning with fisher information for patch-wise semantic segmentation. In: Deep learning in medical image analysis and multimodal learning for clinical decision support: 4th international workshop. Springer, DLMIA, pp 83–91
Settles B, Craven M (2008) An analysis of active learning strategies for sequence labeling tasks. In: proceedings of the 2008 conference on EMNLP, pp 1070–1079
Budd S, Robinson EC, Kainz B (2021) A survey on active learning and human-in-the-loop deep learning for medical image analysis. Med Image Anal 71:102062
Gal Y, Ghahramani Z (2016) Dropout as a Bayesian approximation: representing model uncertainty in deep learning. In: International conference on machine learning, PMLR, pp 1050–1059
Gal Y, Islam R, Ghahramani Z (2017) Deep Bayesian active learning with image data. In: International conference on machine learning. PMLR, pp 1183–1192
Lakshminarayanan B, Pritzel A, Blundell C (2017) Simple and scalable predictive uncertainty estimation using deep ensembles. Adv Neural Inf Process Syst 30
Gaillochet M, Desrosiers C, Lombaert H (2023) Active learning for medical image segmentation with stochastic batches. Med Image Anal 102958
Yang L, Zhang Y, Chen J, Zhang S, Chen DZ (2017) Suggestive annotation: a deep active learning framework for biomedical image segmentation. In: MICCAI. Springer, Quebec City, QC, Canada, pp 399–407
Hiasa Y, Otake Y, Takao M, Ogawa T, Sugano N, Sato Y (2019) Automated muscle segmentation from clinical CT using Bayesian u-net for personalized musculoskeletal modeling. IEEE Trans Med Imaging 39(4):1030–1040
Ozdemir F, Peng Z, Fuernstahl P, Tanner C, Goksel O (2021) Active learning for segmentation based on Bayesian sample queries. Knowl-Based Syst 214:106531
Smailagic A, Costa P, Young Noh H, Walawalkar D, Khandelwal K, Galdran A, Mirshekari M, Fagert J, Xu S, Zhang P, Campilho A (2018) Medal: accurate and robust deep active learning for medical image analysis. In: ICMLA. IEEE, pp 481–488
Nath V, Yang D, Landman BA, Xu D, Roth HR (2020) Diminishing uncertainty within the training pool: active learning for medical image segmentation. IEEE Trans Med Imaging 40(10):2534–2547
Li X, **a M, Jiao J, Zhou S, Chang C, Wang Y, Guo Y (2023) HAL-IA: a hybrid active learning framework using interactive annotation for medical image segmentation. Med Image Anal 102862
Liu P, Wang L, Ranjan R, He G, Zhao L (2022) A survey on active deep learning: from model driven to data driven. ACM Comput Surv (CSUR) 54(10s):1–34
Amagata D (2023) Diversity maximization in the presence of outliers. In: Proceedings of the AAAI conference on artificial intelligence, pp 12338–12345
Fukumoto Y, Taniguchi M, Hirono T, Yagi M, Yamagata M, Nakai R, Asai T, Yamada Y, Kimura M, Ichihashi N (2022) Influence of ultrasound focus depth on the association between echo intensity and intramuscular adipose tissue. Muscle Nerve 66(5):568–575
Rosset A, Spadola L, Ratib O (2004) OsiriX: an open-source software for navigating in multidimensional DICOM images. J Digit Imaging 17:205–216
Kikinis R, Pieper SD, Vosburgh KG (2013) 3d slicer: a platform for subject-specific image analysis, visualization, and clinical support. In: Intraoperative imaging and image-guided therapy. Springer, pp 277–289
Lin TY, Goyal P, Girshick R, He K, Dollár P (2017) Focal loss for dense object detection. In: Proceedings of the IEEE international conference on computer vision, pp 2980–2988
Chen JA, Niu W, Ren B, Wang Y, Shen X (2023) Survey: exploiting data redundancy for optimization of deep learning. ACM Comput Surv 55(10):1–38
Yuan J, Hou X, **ao Y, Cao D, Guan W, Nie L (2019) Multi-criteria active deep learning for image classification. Knowl-Based Syst 172:86–94
Nath V, Yang D, Roth HR, Xu D (2022) Warm start active learning with proxy labels and selection via semi-supervised fine-tuning. In: MICCAI. Springer, Singapore, pp 297–308
Acknowledgements
This work was funded by MEXT/JSPS KAKENHI (19H01176, 20H04550, 20K19376, 21H03303, 21K16655, 21K18080).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Below is the link to the electronic supplementary material.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Li, G., Otake, Y., Soufi, M. et al. Hybrid representation-enhanced sampling for Bayesian active learning in musculoskeletal segmentation of lower extremities. Int J CARS (2024). https://doi.org/10.1007/s11548-024-03065-7
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s11548-024-03065-7