DD_RoTIR: Dual-Domain Image Registration via Image Translation and Hiearchical Feature-matching

Microscopy images acquired by multiple camera lenses or sensors in biological experiments offer a comprehensive understanding of the objects from diverse aspects. However, setups for multiple microscopes raise the possibility of misalignment of identical target features through different modalities. Thus, multimodal image registration is essential. In this work, we employed previous successes in biological image translation (XAcGAN) and mono-modal image registration (RoTIR) and created a deep-learning-based model, Dual-Domain RoTIR (DD_RoTIR), to address the challenges. However, it is believed that GAN-based translation models are inadequate for multimodal image registration. We facilitated the registration utilizing the feature-matching algorithm based on Transformers and rotation equivariant networks. Furthermore, hierarchical feature-matching was employed as multimodal image registration is more challenging. Results show the DD_RoTIR model presents good applicability and robustness in multiple microscopy image datasets.