In this paper we present a method for building boundary refinement and regularization in satellite images using a fully convolutional neural network trained with a combination of adversarial and regularized losses. Compared to a pure Mask R-CNN model, the overall algorithm can achieve equivalent performance in terms of accuracy and completeness. However, unlike Mask R-CNN that produces irregular footprints, our framework generates regularized and visually pleasing building boundaries which are beneficial in many applications.