Natural Image Stitching Using Depth Maps

Natural image stitching (NIS) aims to create one natural-looking mosaic from two overlapping images that capture a same 3D scene from different viewing positions. Challenges inevitably arise when the scene is non-planar and the camera baseline is wide, since parallax becomes not negligible in such cases. In this paper, we propose a novel NIS method using depth maps, which generates natural-looking mosaics against parallax in both overlapping and non-overlapping regions. Firstly, we estimate a pixel-to-pixel transformation based on feature matches and their depth values. Then, we draw a triangulation of the target image and estimate multiple local homographies, one per triangle, based on the locations of their vertices and the rectified depth values. Finally, the warping image is composited by the backward mapping of piece-wise homographies. Experimental results demonstrate that the proposed method not only provides accurate alignment in the overlapping regions, but also virtual naturalness in the non-overlapping region.