User Association and Coordinated Beamforming in Cognitive Aerial-Terrestrial Networks: A Safe Reinforcement Learning Approach

Cognitive aerial-terrestrial networks (CATNs) offer a solution to spectrum scarcity by sharing spectrum between aerial and terrestrial networks. However, aerial users (AUs) experience significant interference from numerous terrestrial base stations (BSs). To alleviate such interference, we investigate a user association and coordinated beamforming (CBF) problem in CATN, where the aerial network serves as the primary network sharing its spectrum with the terrestrial network. Specifically, we maximize the sum rate of the secondary terrestrial users (TUs) under the interference temperature constraints of the AUs. Traditional iterative optimization schemes are impractical due to their high computational complexity and information exchange overhead. Although deep reinforcement learning (DRL) based schemes can address these challenges, their performance is sensitive to the weights of the weighted penalty terms for violating constraints in the reward function. Motivated by these issues, we propose a safe DRL-based user association and CBF scheme for CATN, eliminating the need for training multiple times to find the optimal penalty weight before actual deployment. Specifically, the CATN is modeled as a networked constrained partially observable Markov game. Each TU acts as an agent to choose its associated BS, and each BS acts as an agent to decide its beamforming vectors, aiming to maximize the reward while satisfying the safety constraints introduced by the interference constraints of the AUs. By exploiting a safe DRL algorithm, the proposed scheme incurs lower deployment expenses than the penalty-based DRL schemes since only one training is required before actual deployment. Simulation results show that the proposed scheme can achieve a higher sum rate of TUs than a two-stage optimization scheme while the average received interference power of the AUs is generally below the threshold.

Hierarchical Cognitive Spectrum Sharing in Space-Air-Ground Integrated Networks

In space-air-ground integrated networks (SAGINs), cognitive spectrum sharing has been regarded as a promising solution to improve spectrum efficiency by enabling a secondary network to access the spectrum of a primary network. However, different networks in SAGIN may have different quality of service (QoS) requirements, which can not be well satisfied with the traditional cognitive spectrum sharing architecture. For example, the aerial network typically has high QoS requirements, which however may not be met when it acts as a secondary network. To address this issue, in this paper, we propose a hierarchical cognitive spectrum sharing architecture (HCSSA) for SAGINs, where the secondary networks are divided into a preferential one and an ordinary one. Specifically, the aerial and terrestrial networks can access the spectrum of the satellite network under the condition that the caused interference to the satellite terminal is below a certain threshold. Besides, considering that the aerial network has a higher priority than the terrestrial network, we aim to use a rate constraint to ensure the performance of the aerial network. Subject to these two constraints, we consider a sum-rate maximization for the terrestrial network by jointly optimizing the transmit beamforming vectors of the aerial and terrestrial base stations. To solve this non-convex problem, we propose a penalty-based iterative beamforming (PIBF) scheme that uses the penalty method and the successive convex approximation technique. Moreover, we also develop three low-complexity schemes by optimizing the normalized beamforming vectors and power control. Finally, we provide extensive numerical simulations to compare the performance of the proposed PIBF scheme and the low-complexity schemes. The results also demonstrate the advantages of the proposed HCSSA compared with the traditional cognitive spectrum sharing architecture.