Abstract:This paper proposes a novel framework that leverages large language models (LLMs) to automate curriculum design, thereby enhancing the application of reinforcement learning (RL) in mobile networks. As mobile networks evolve towards the 6G era, managing their increasing complexity and dynamic nature poses significant challenges. Conventional RL approaches often suffer from slow convergence and poor generalization due to conflicting objectives and the large state and action spaces associated with mobile networks. To address these shortcomings, we introduce curriculum learning, a method that systematically exposes the RL agent to progressively challenging tasks, improving convergence and generalization. However, curriculum design typically requires extensive domain knowledge and manual human effort. Our framework mitigates this by utilizing the generative capabilities of LLMs to automate the curriculum design process, significantly reducing human effort while improving the RL agent's convergence and performance. We deploy our approach within a simulated mobile network environment and demonstrate improved RL convergence rates, generalization to unseen scenarios, and overall performance enhancements. As a case study, we consider autonomous coordination and user association in mobile networks. Our obtained results highlight the potential of combining LLM-based curriculum generation with RL for managing next-generation wireless networks, marking a significant step towards fully autonomous network operations.
Abstract:A simultaneous transmitting and reflecting reconfigurable intelligent surface (STAR-RIS) aided integrated sensing and communication (ISAC) dual-secure communication system is studied in this paper. The sensed target and legitimate users (LUs) are situated on the opposite sides of the STAR-RIS, and the energy splitting and time switching protocols are applied in the STAR-RIS, respectively. The long-term average security rate for LUs is maximized by the joint design of the base station (BS) transmit beamforming and receive filter, along with the STAR-RIS transmitting and reflecting coefficients, under guarantying the echo signal-to-noise ratio thresholds and rate constraints for the LUs. Since the channel information changes over time, conventional convex optimization techniques cannot provide the optimal performance for the system, and result in excessively high computational complexity in the exploration of the long-term gains for the system. Taking continuity control decisions into account, the deep deterministic policy gradient and soft actor-critic algorithms based on off-policy are applied to address the complex non-convex problem. Simulation results comprehensively evaluate the performance of the proposed two reinforcement learning algorithms and demonstrate that STAR-RIS is remarkably better than the two benchmarks in the ISAC system.
Abstract:Wireless sensing and wireless energy are enablers to pave the way for smart transportation and a greener future. In this paper, an intelligent reflecting surface (IRS) assisted integrated sensing and wireless power transfer (ISWPT) system is investigated, where the transmitter in transportation infrastructure networks sends signals to sense multiple targets and simultaneously to multiple energy harvesting devices (EHDs) to power them. In light of the performance tradeoff between energy harvesting and sensing, we propose to jointly optimize the system performance via optimizing the beamforming and IRS phase shift. However, the coupling of optimization variables makes the formulated problem non-convex. Thus, an alternative optimization approach is introduced and based on which two algorithms are proposed to solve the problem. Specifically, the first one involves a semi-definite program technique, while the second one features a low-complexity optimization algorithm based on successive convex approximation and majorization minimization. Our simulation results validate the proposed algorithms and demonstrate the advantages of using IRS to assist wireless power transfer in ISWPT systems.
Abstract:Massive connectivity for extra-large multi-input multi-output (XL-MIMO) systems is a challenging issue due to the prohibitive cost and the near-field non-stationary channels. In this paper, we propose an uplink grant-free massive access scheme for XL-MIMO systems, in which a mixed-analog-to-digital converters (ADC) architecture is adopted to strike the right balance between access performance and energy cost. By exploiting the spatial-domain structured sparsity and the piecewise angular-domain cluster sparsity of massive access channels, a compressive sensing (CS)-based two-stage orthogonal approximate message passing algorithm is proposed to efficiently solve the joint activity detection and channel estimation problem. Particularly, high-precision quantized measurements are leveraged to perform accurate hyper-parameter estimation, thereby facilitating the activity detection. Moreover, we adopt subarray-wise estimation strategy to overcome the severe angular-domain energy dispersion problem which is caused by the spatial non-stationarity of near-field XL-MIMO channels. Simulation results verify the superiority of our proposed algorithm over state-of-the-art CS algorithms for massive access based on XL-MIMO with mixed-ADC architectures.
Abstract:This paper proposes a novel transmission policy for an intelligent reflecting surface (IRS) assisted wireless powered sensor network (WPSN). An IRS is deployed to enhance the performance of wireless energy transfer (WET) and wireless information transfer (WIT) by intelligently adjusting phase shifts of each reflecting elements. To achieve its self-sustainability, the IRS needs to collect energy from energy station to support its control circuit operation. Our proposed policy for the considered WPSN is called IRS assisted harvest-then-transmit time switching, which is able to schedule the transmission time slots by switching between energy collection and energy reflection modes. We study the achievable sum throughput of the proposed transmission policy and investigate a joint design of the transmission time slots, the power allocation, as well as the discrete phase shifts of the WET and WIT. This formulates the problem as a mixed-integer non-linear program, which is NP-hard and non-convex. We first relax it to one with continuous phase shifts, and then propose a two-step approach and decompose the original problem into two sub-problems. We solve the first sub-problem with respect to the phase shifts of the WIT in terms of closed-form expression. For the second sub-problem, we consider a special case without the circuit power of each sensor node, the Lagrange dual method and the KKT conditions are applied to derive the optimal closed-form transmission time slots, power allocation, and phase shift of the WET. Then we generalise the case with the circuit power of each sensor node, which can be solved via employing a semi-definite programming relaxation. The optimal discrete phase shifts can be obtained by quantizing the continuous values. Numerical results demonstrate the effectiveness of the proposed policy and validate the beneficial role of the IRS in comparison to the benchmark schemes.