Beam Management in Ultra-dense mmWave Network via Federated Reinforcement Learning: An Intelligent and Secure Approach

Deploying ultra-dense networks that operate on millimeter wave (mmWave) band is a promising way to address the tremendous growth on mobile data traffic. However, one key challenge of ultra-dense mmWave network (UDmmN) is beam management due to the high propagation delay, limited beam coverage as well as numerous beams and users. In this paper, a novel systematic beam control scheme is presented to tackle the beam management problem which is difficult due to the nonconvex objective function. We employ double deep Q-network (DDQN) under a federated learning (FL) framework to address the above optimization problem, and thereby fulfilling adaptive and intelligent beam management in UDmmN. In the proposed beam management scheme based on FL (BMFL), the non-rawdata aggregation can theoretically protect user privacy while reducing handoff cost. Moreover, we propose to adopt a data cleaning technique in the local model training for BMFL, with the aim to further strengthen the privacy protection of users while improving the learning convergence speed. Simulation results demonstrate the performance gain of our proposed scheme.

Joint Optimization of Active and Passive Beamforming in Multi-IRS Aided mmWave Communications

Intelligent reflecting surface (IRS) has been considered as a promising technology to alleviate the blockage effect and enhance coverage in millimeter wave (mmWave) communication. To explore the impact of IRS on the performance of mmWave communication, we investigate a multi-IRS assisted mmWave communication network and formulate a sum rate maximization problem by jointly optimizing the active and passive beamforming and the set of IRSs for assistance. The optimization problem is intractable due to the lack of convexity of the objective function and the binary nature of the IRS selection variables. To tackle the complex non-convex problem, an alternating iterative approach is proposed. In particular, utilizing the fractional programming method to optimize the active and passive beamforming and the optimization of IRS selection is solved by enumerating. Simulation results demonstrate the performance gain of our proposed approach.