MDT-A2G: Exploring Masked Diffusion Transformers for Co-Speech Gesture Generation

Recent advancements in the field of Diffusion Transformers have substantially improved the generation of high-quality 2D images, 3D videos, and 3D shapes. However, the effectiveness of the Transformer architecture in the domain of co-speech gesture generation remains relatively unexplored, as prior methodologies have predominantly employed the Convolutional Neural Network (CNNs) or simple a few transformer layers. In an attempt to bridge this research gap, we introduce a novel Masked Diffusion Transformer for co-speech gesture generation, referred to as MDT-A2G, which directly implements the denoising process on gesture sequences. To enhance the contextual reasoning capability of temporally aligned speech-driven gestures, we incorporate a novel Masked Diffusion Transformer. This model employs a mask modeling scheme specifically designed to strengthen temporal relation learning among sequence gestures, thereby expediting the learning process and leading to coherent and realistic motions. Apart from audio, Our MDT-A2G model also integrates multi-modal information, encompassing text, emotion, and identity. Furthermore, we propose an efficient inference strategy that diminishes the denoising computation by leveraging previously calculated results, thereby achieving a speedup with negligible performance degradation. Experimental results demonstrate that MDT-A2G excels in gesture generation, boasting a learning speed that is over 6$\times$ faster than traditional diffusion transformers and an inference speed that is 5.7$\times$ than the standard diffusion model.

OverlapNetVLAD: A Coarse-to-Fine Framework for LiDAR-based Place Recognition

Place recognition is a challenging yet crucial task in robotics. Existing 3D LiDAR place recognition methods suffer from limited feature representation capability and long search times. To address these challenges, we propose a novel coarse-to-fine framework for 3D LiDAR place recognition that combines Birds' Eye View (BEV) feature extraction, coarse-grained matching, and fine-grained verification. In the coarse stage, our framework leverages the rich contextual information contained in BEV features to produce global descriptors. Then the top-\textit{K} most similar candidates are identified via descriptor matching, which is fast but coarse-grained. In the fine stage, our overlap estimation network reuses the corresponding BEV features to predict the overlap region, enabling meticulous and precise matching. Experimental results on the KITTI odometry benchmark demonstrate that our framework achieves leading performance compared to state-of-the-art methods. Our code is available at: \url{https://github.com/fcchit/OverlapNetVLAD}.