Comparing Training Strategies Using Multi-Assessor Segmentation Labels for Barrett’s Neoplasia Detection

In medical imaging, segmentation ground truths generally suffer from large inter-observer variability. When multiple observers are used, simple fusion techniques are typically employed to combine multiple delineations into one consensus ground truth. However, in this process, potentially valuable information is discarded and it is yet unknown what strategy leads to optimal segmentation results. In this work, we compare several ground-truth types to train a U-net and apply it to the clinical use case of Barrett’s neoplasia detection. To this end, we have invited 14 international Barrett’s experts to delineate 2,851 neoplastic images derived from 812 patients into a higher- and lower-likelihood neoplasia areas. Five different ground-truths techniques along with four different training losses are compared with each other using the Area-under-the-curve (AUC) value for Barrett’s neoplasia detection. The value used to generate this curve is the maximum pixel value in the raw segmentation map, and the histologically proven ground truth of the image. The experiments show that random sampling of the four neoplastic areas together with a compound loss Binary Cross-entropy and DICE yields the highest value of 94.12%, while fusion-based ground truth clearly performs lower. The results show that researchers should incorporate measures for uncertainty in their design of networks.