We address the generalization behavior of deep neural networks in the context of brain tumor segmentation. While current topologies show an increasingly complex structure, the overall benchmark performance does improve negligibly. In our experiments, we demonstrate that a well trained U-Net shows the best generalization behavior and is sufficient to solve this segmentation problem. We illustrate why extensions of this model cannot only be pointless but even harmful in a realistic scenario. Also, we suggest two simple modifications (that do not alter the topology) to further improve its generalization performance.