Affordable Access

Semantic segmentation of 3D medical images with deep learning

  • Petit, Olivier
Publication Date
Dec 17, 2021
External links


Deep Learning has recently shown impressive results in computer vision. Especially with theConvolutional Neural Networks (ConvNets) which have redefined the state of the art in many applications such asmedical image segmentation. In this thesis we address problems in the task of abdominal organ segmentation usingdeep learning models. In the first part, we address the issue of training deep ConvNets on partially labeled data.Professionals often focus on specific anatomical regions leading to heterogeneous datasets with partially labeledimages. Training a model directly on such data leads to very poor results. Thus, we propose a training schemethat leverages all the labels without being affected by the missing ones. Moreover, an iterative scheme relabelsthe missing organs of the training set which further improves the segmentation model. The second part aims atusing spatial prior about the position of the organs to improve the detection of structures and reduce outliersin the segmentation. ConvNets by construction, does not capture absolute spatial information. However, medicalimages are very structured and there are conventions about the expected position of organs. Thus, we propose a 3Dspatial prior that captures the spatial position of organs and then explicitly biases the model through a prior-drivenactivation function. Finally, we propose to use Transformers to model long range dependencies between anatomicalstructures in a segmentation model used for organ segmentation. ConvNets do not capture such interactionsbecause of the receptive field which is often limited. Using dense attention introduced in Transformers allows toconnect every pixel with each other and thus to model complex interactions on different parts of the input image.We propose U-Transformer and show that it improves the quality of the segmentation on various datasets.

Report this publication


Seen <100 times