Abstract:Beamforming design has been extensively investigated in integrated sensing and communication (ISAC) systems. The use of movable antennas has proven effective in enhancing the design of beamforming. Although some studies have explored joint optimization of transmit beamforming matrices and antenna positions in bistatic scenarios, there is a gap in the literature regarding monostatic full-duplex (FD) systems. To fill this gap, we propose an algorithm that jointly optimizes the beamforming and antenna positions at both the transmitter and the receiver in a monostatic FD system. In an FD system, suppressing self-interference is crucial. This interference can be significantly reduced by carefully designing transmit and receive beamforming matrices. To further enhance the suppression, we derive a formulation of self-interference characterized by antenna position vectors. This enables the strategic positioning of movable antennas to further mitigate interference. Our approach optimizes the weighted sum of communication capacity and mutual information by simultaneously optimizing beamforming and antenna positions for both tranceivers. Specifically, we propose a coarse-to-fine grained search algorithm (CFGS) to find optimal antenna positions. Numerical results demonstrate that our proposed algorithm provides significant improvements for the MA system compared to conventional fixed-position antenna systems.