Self-Supervised Large Scale Point Cloud Completion for Archaeological Site Restoration

Point cloud completion helps restore partial incomplete point clouds suffering occlusions. Current self-supervised methods fail to give high fidelity completion for large objects with missing surfaces and unbalanced distribution of available points. In this paper, we present a novel method for restoring large-scale point clouds with limited and imbalanced ground-truth. Using rough boundary annotations for a region of interest, we project the original point clouds into a multiple-center-of-projection (MCOP) image, where fragments are projected to images of 5 channels (RGB, depth, and rotation). Completion of the original point cloud is reduced to inpainting the missing pixels in the MCOP images. Due to lack of complete structures and an unbalanced distribution of existing parts, we develop a self-supervised scheme which learns to infill the MCOP image with points resembling existing "complete" patches. Special losses are applied to further enhance the regularity and consistency of completed MCOP images, which is mapped back to 3D to form final restoration. Extensive experiments demonstrate the superiority of our method in completing 600+ incomplete and unbalanced archaeological structures in Peru.

Image Stitching Based on Planar Region Consensus

Image stitching for two images without a global transformation between them is notoriously difficult. In this paper, noticing the importance of planar structure under perspective geometry, we propose a new image stitching method which stitches images by allowing for the alignment of a set of matched dominant planar regions. Clearly different from previous methods resorting to plane segmentation, the key to our approach is to utilize rich semantic information directly from RGB images to extract planar image regions with a deep Convolutional Neural Network (CNN). We specifically design a new module to make fully use of existing semantic segmentation networks to accommodate planar segmentation. To train the network, a dataset for planar region segmentation is contributed. With the planar region knowledge, a set of local transformations can be obtained by constraining matched regions, enabling more precise alignment in the overlapping area. We also use planar knowledge to estimate a transformation field over the whole image. The final mosaic is obtained by a mesh-based optimization framework which maintains high alignment accuracy and relaxes similarity transformation at the same time. Extensive experiments with quantitative comparisons show that our method can deal with different situations and outperforms the state-of-the-arts on challenging scenes.