Local Structure Matching Driven by Joint-Saliency-Structure Adaptive Kernel Regression

For nonrigid image registration, matching the particular structures (or the outliers) that have missing correspondence and/or local large deformations, can be more difficult than matching the common structures with small deformations in the two images. Most existing works depend heavily on the outlier segmentation to remove the outlier effect in the registration. Moreover, these works do not handle simultaneously the missing correspondences and local large deformations. In this paper, we defined the nonrigid image registration as a local adaptive kernel regression which locally reconstruct the moving image's dense deformation vectors from the sparse deformation vectors in the multi-resolution block matching. The kernel function of the kernel regression adapts its shape and orientation to the reference image's structure to gather more deformation vector samples of the same structure for the iterative regression computation, whereby the moving image's local deformations could be compliant with the reference image's local structures. To estimate the local deformations around the outliers, we use joint saliency map that highlights the corresponding saliency structures (called Joint Saliency Structures, JSSs) in the two images to guide the dense deformation reconstruction by emphasizing those JSSs' sparse deformation vectors in the kernel regression. The experimental results demonstrate that by using local JSS adaptive kernel regression, the proposed method achieves almost the best performance in alignment of all challenging image pairs with outlier structures compared with other five state-of-the-art nonrigid registration algorithms.

Scale Selection of Adaptive Kernel Regression by Joint Saliency Map for Nonrigid Image Registration

Joint saliency map (JSM) [1] was developed to assign high joint saliency values to the corresponding saliency structures (called Joint Saliency Structures, JSSs) but zero or low joint saliency values to the outliers (or mismatches) that are introduced by missing correspondence or local large deformations between the reference and moving images to be registered. JSM guides the local structure matching in nonrigid registration by emphasizing these JSSs' sparse deformation vectors in adaptive kernel regression of hierarchical sparse deformation vectors for iterative dense deformation reconstruction. By designing an effective superpixel-based local structure scale estimator to compute the reference structure's structure scale, we further propose to determine the scale (the width) of kernels in the adaptive kernel regression through combining the structure scales to JSM-based scales of mismatch between the local saliency structures. Therefore, we can adaptively select the sample size of sparse deformation vectors to reconstruct the dense deformation vectors for accurately matching the every local structures in the two images. The experimental results demonstrate better accuracy of our method in aligning two images with missing correspondence and local large deformation than the state-of-the-art methods.