TY - GEN
T1 - Lesion-wise evaluation for effective performance monitoring of small object segmentation
AU - Groothuis, Irme
AU - Sudre, Carole H.
AU - Ingala, Silvia
AU - Barnes, Jo
AU - Gispert, Juan Domingo
AU - Sørensen, Lauge
AU - Pai, Akshay
AU - Nielsen, Mads
AU - Ourselin, Sebastien
AU - Cardoso, M. Jorge
AU - Barkhof, Frederik
AU - Modat, Marc
N1 - Funding Information: This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Sk lodowska-Curie grant agreement No 721820. Funding Information: The authors would like to thank the ADNI and ALFA consortia for providing access to their data. We gratefully acknowledge the support of NVIDIA Corporation with the donation of the P6000 GPU used for this research. Publisher Copyright: © 2021 SPIE. Copyright: Copyright 2021 Elsevier B.V., All rights reserved.
PY - 2021
Y1 - 2021
N2 - Object detection in medical images using deep learning is a challenging task, due to the imbalance often present in the data. Deep learning algorithms require large amount of balanced data to achieve optimal performance, as well as close monitoring and ne-tuning of hyper parameters. For most applications, such performance monitoring is done by simply feeding unseen data trough the network, and then using the loss function for evaluation. In the case of small or sparse objects, the loss function might not able to describe the features needed, but such features can be hard to capture in a loss function. In this paper we introduce a lesion-wise whole volume validation tool, which allows more a more accurate performance monitoring of segmentation of small and sparse objects. We showcase the efficacy of our tool by applying it to the task of microbleed segmentation, and compare the behaviour of lesionwise-whole volume validation compared to well known segmentation loss functions. Microbleeds are visible as small (less than 10 mm), ovoid hypo-intensities on T2∗-weighted and susceptibility weighted magnetic resonance images. Detection of microbleeds is clinically relevant, as microbleeds can indicate the risk of recurrent stroke, and are used as imaging biomarker for various neurodegenerative diseases. Manual detection or segmentation is time consuming and error prone, and suffers from high inter- and intraobserver variability. Due to the sparsity and small size of the lesions, the data is severely imbalanced.
AB - Object detection in medical images using deep learning is a challenging task, due to the imbalance often present in the data. Deep learning algorithms require large amount of balanced data to achieve optimal performance, as well as close monitoring and ne-tuning of hyper parameters. For most applications, such performance monitoring is done by simply feeding unseen data trough the network, and then using the loss function for evaluation. In the case of small or sparse objects, the loss function might not able to describe the features needed, but such features can be hard to capture in a loss function. In this paper we introduce a lesion-wise whole volume validation tool, which allows more a more accurate performance monitoring of segmentation of small and sparse objects. We showcase the efficacy of our tool by applying it to the task of microbleed segmentation, and compare the behaviour of lesionwise-whole volume validation compared to well known segmentation loss functions. Microbleeds are visible as small (less than 10 mm), ovoid hypo-intensities on T2∗-weighted and susceptibility weighted magnetic resonance images. Detection of microbleeds is clinically relevant, as microbleeds can indicate the risk of recurrent stroke, and are used as imaging biomarker for various neurodegenerative diseases. Manual detection or segmentation is time consuming and error prone, and suffers from high inter- and intraobserver variability. Due to the sparsity and small size of the lesions, the data is severely imbalanced.
KW - Data imbalance
KW - Evaluation
KW - Microbleed
UR - http://www.scopus.com/inward/record.url?scp=85103674563&partnerID=8YFLogxK
U2 - https://doi.org/10.1117/12.2580734
DO - https://doi.org/10.1117/12.2580734
M3 - Conference contribution
VL - 11596
T3 - Progress in Biomedical Optics and Imaging - Proceedings of SPIE
BT - Medical Imaging 2021
A2 - Isgum, Ivana
A2 - Landman, Bennett A.
PB - SPIE
T2 - Medical Imaging 2021: Image Processing
Y2 - 15 February 2021 through 19 February 2021
ER -