Abstract:Recently, developing unified medical image segmentation models gains increasing attention, especially with the advent of the Segment Anything Model (SAM). SAM has shown promising binary segmentation performance in natural domains, however, transferring it to the medical domain remains challenging, as medical images often possess substantial inter-category overlaps. To address this, we propose the SEmantic-Guided SAM (SEG-SAM), a unified medical segmentation model that incorporates semantic medical knowledge to enhance medical segmentation performance. First, to avoid the potential conflict between binary and semantic predictions, we introduce a semantic-aware decoder independent of SAM's original decoder, specialized for both semantic segmentation on the prompted object and classification on unprompted objects in images. To further enhance the model's semantic understanding, we solicit key characteristics of medical categories from large language models and incorporate them into SEG-SAM through a text-to-vision semantic module, adaptively transferring the language information into the visual segmentation task. In the end, we introduce the cross-mask spatial alignment strategy to encourage greater overlap between the predicted masks from SEG-SAM's two decoders, thereby benefiting both predictions. Extensive experiments demonstrate that SEG-SAM outperforms state-of-the-art SAM-based methods in unified binary medical segmentation and task-specific methods in semantic medical segmentation, showcasing promising results and potential for broader medical applications.
Abstract:Controllable person image generation aims to generate a person image conditioned on reference images, allowing precise control over the person's appearance or pose. However, prior methods often distort fine-grained textural details from the reference image, despite achieving high overall image quality. We attribute these distortions to inadequate attention to corresponding regions in the reference image. To address this, we thereby propose learning flow fields in attention (Leffa), which explicitly guides the target query to attend to the correct reference key in the attention layer during training. Specifically, it is realized via a regularization loss on top of the attention map within a diffusion-based baseline. Our extensive experiments show that Leffa achieves state-of-the-art performance in controlling appearance (virtual try-on) and pose (pose transfer), significantly reducing fine-grained detail distortion while maintaining high image quality. Additionally, we show that our loss is model-agnostic and can be used to improve the performance of other diffusion models.
Abstract:In this paper, we study the neural tangent kernel (NTK) for general partial differential equations (PDEs) based on physics-informed neural networks (PINNs). As we all know, the training of an artificial neural network can be converted to the evolution of NTK. We analyze the initialization of NTK and the convergence conditions of NTK during training for general PDEs. The theoretical results show that the homogeneity of differential operators plays a crucial role for the convergence of NTK. Moreover, based on the PINNs, we validate the convergence conditions of NTK using the initial value problems of the sine-Gordon equation and the initial-boundary value problem of the KdV equation.
Abstract:We introduce MarDini, a new family of video diffusion models that integrate the advantages of masked auto-regression (MAR) into a unified diffusion model (DM) framework. Here, MAR handles temporal planning, while DM focuses on spatial generation in an asymmetric network design: i) a MAR-based planning model containing most of the parameters generates planning signals for each masked frame using low-resolution input; ii) a lightweight generation model uses these signals to produce high-resolution frames via diffusion de-noising. MarDini's MAR enables video generation conditioned on any number of masked frames at any frame positions: a single model can handle video interpolation (e.g., masking middle frames), image-to-video generation (e.g., masking from the second frame onward), and video expansion (e.g., masking half the frames). The efficient design allocates most of the computational resources to the low-resolution planning model, making computationally expensive but important spatio-temporal attention feasible at scale. MarDini sets a new state-of-the-art for video interpolation; meanwhile, within few inference steps, it efficiently generates videos on par with those of much more expensive advanced image-to-video models.
Abstract:This paper presents, for the first time, the concept of \textit{polarforming} for wireless communications. Polarforming refers to a novel technique that enables dynamic adjustment of antenna polarization using reconfigurable polarized antennas (RPAs). It can fully leverage polarization diversity to improve the performance of wireless communication systems by aligning the effective polarization state of the incoming electromagnetic (EM) wave with the antenna polarization. To better demonstrate the benefits of polarforming, we propose a general RPA-aided system that allows for tunable antenna polarization. A wavefront-based channel model is developed to properly capture depolarization behaviors in both line-of-sight (LoS) and non-line-of-sight (NLoS) channels. Based on this model, we provide a detailed description of transmit and receive polarforming on planes of polarization (PoPs). We also evaluate the performance gains provided by polarforming under stochastic channel conditions. Specifically, we derive a closed-form expression for the relative signal-to-noise ratio (SNR) gain compared to conventional fixed-polarization antenna (FPA) systems and approximate the cumulative distribution function (CDF) for the RPA system. Our analysis reveals that polarforming offers a diversity gain of two, indicating full utilization of polarization diversity for dual-polarized antennas. Furthermore, extensive simulation results validate the effectiveness of polarforming and exhibit substantial improvements over conventional FPA systems. The results also indicate that polarforming not only can combat depolarization effects caused by wireless channels but also can overcome channel correlation when scattering is insufficient.
Abstract:This letter investigates a movable antenna (MA)-aided full-duplex (FD) satellite communication system, where the satellite, equipped with both transmit and receive MAs, serves multiple uplink (UL) and downlink (DL) user terminals (UTs) in FD mode. Specifically, we formulate a multiobjective optimization problem to minimize the UL and DL transmit powers under imperfect channel state information (CSI) conditions. To jointly optimize the MA positions and transmit powers, we propose a two-loop particle swarm optimization (PSO) algorithm based on a multiobjective optimization framework. Simulation results demonstrate that flexible adjustments of MA positions can effectively reduce the total UL and DL transmit powers, while also alleviating the burden on self-interference (SI) cancellation modules.
Abstract:This paper proposes a secure wire-line telephone prototype that leverages physical layer security (PLS) techniques to protect communications from wiretapping. The system generates artificial noise (AN) in both directions over a telephone line and utilizes a telephone hybrid circuit to achieve effective AN cancellation. We conduct a thorough analysis of the secrecy capacity and evaluate the system's performance through both simulations and practical experiments. The results demonstrate that the proposed scheme significantly enhances communication security while preserving the integrity of legitimate signals, making it a robust and viable solution for secure telephone systems.
Abstract:This letter investigates movable antenna (MA)-aided downlink (DL) multiuser communication systems under the near-field channel condition, in which both the base station (BS) and the users are equipped with MAs to fully exploit the degrees of freedom (DoFs) in antenna position optimization by leveraging the wireless channel variation in spatial regions of large size. The objective is to minimize the transmit power by jointly optimizing the beamformers and the MA positions while satisfying the minimum-achievable-rate requirement for each user. We propose a two-loop dynamic neighborhood pruning particle swarm optimization (DNPPSO) algorithm that significantly reduces computational complexity while effectively maintaining the performance of the standard particle swarm optimization (PSO) algorithm. Simulation results validate the effectiveness and advantages of the proposed scheme in power-saving for multiuser communications.
Abstract:Panoptic Scene Graph Generation (PSG) aims to segment objects and recognize their relations, enabling the structured understanding of an image. Previous methods focus on predicting predefined object and relation categories, hence limiting their applications in the open world scenarios. With the rapid development of large multimodal models (LMMs), significant progress has been made in open-set object detection and segmentation, yet open-set relation prediction in PSG remains unexplored. In this paper, we focus on the task of open-set relation prediction integrated with a pretrained open-set panoptic segmentation model to achieve true open-set panoptic scene graph generation (OpenPSG). Our OpenPSG leverages LMMs to achieve open-set relation prediction in an autoregressive manner. We introduce a relation query transformer to efficiently extract visual features of object pairs and estimate the existence of relations between them. The latter can enhance the prediction efficiency by filtering irrelevant pairs. Finally, we design the generation and judgement instructions to perform open-set relation prediction in PSG autoregressively. To our knowledge, we are the first to propose the open-set PSG task. Extensive experiments demonstrate that our method achieves state-of-the-art performance in open-set relation prediction and panoptic scene graph generation. Code is available at \url{https://github.com/franciszzj/OpenPSG}.
Abstract:In this paper, we investigate physical layer security (PLS) for full-duplex (FD) multi-user systems. To simultaneously protect uplink (UL) and downlink (DL) transmissions and ensure efficient use of time-frequency resources, we consider a base station (BS) that operates in FD mode and enables to emit the artificial noise (AN). Conventional fixed-position antennas (FPAs) at the BS struggle to fully exploit spatial degrees of freedom (DoFs). Therefore, we propose a new paradigm for secure FD multi-user systems, where multiple transmit and receive movable antennas (MAs) are deployed at the BS to serve UL and DL users and effectively counter the cooperative interception by multiple eavesdroppers (Eves). Specifically, the MA positions, the transmit, receive, and AN beamformers at the BS, and the UL powers are jointly optimized to maximize the sum of secrecy rates (SSR). To solve the challenging non-convex optimization problem with highly coupled variables, we propose an alternating optimization (AO) algorithm. This algorithm decomposes the original problem into three sub-problems, which are iteratively solved by the proposed multi-velocity particle swarm optimization (MVPSO) and successive convex approximation (SCA). Simulation results demonstrate that the proposed scheme for MA-aided secure FD multi-user systems can significantly enhance security performance compared to conventional FPA systems.