Multi-Task Learning with a Fully Convolutional Network for Rectum and Rectal Cancer Segmentation

In a rectal cancer treatment planning, the location of rectum and rectal cancer plays an important role. The aim of this study is to propose a fully automatic method to segment both rectum and rectal cancer with axial T2-weighted magnetic resonance images. We present a fully convolutional network for multi-task learning to segment both rectum and rectal cancer. Moreover, we propose an assessment method based on bias-variance decomposition to visualize and measure the regional model robustness of a segmentation network. In addition, we suggest a novel augmentation method which can improve the segmentation performance and reduce the training time. Our proposed method not only is computationally efficient due to its fully convolutional nature but also outperforms the current state-of-the-art in rectal cancer segmentation. It also shows high accuracy in rectum segmentation, for which no previous studies exist. We conclude that rectum information benefits the training of rectal cancer segmentation model, especially concerning model variance.