Reinforcement Learning-Based Downlink Transmit Precoding for Mitigating the Impact of Delayed CSI in Satellite Systems

The integration of low earth orbit (LEO) satellites with terrestrial communication networks holds the promise of seamless global connectivity. The efficiency of this connection, however, depends on the availability of reliable channel state information (CSI). Due to the large space-ground propagation delays, the estimated CSI is outdated. In this paper we consider the downlink of a satellite operating as a base station in support of multiple mobile users. The estimated outdated CSI is used at the satellite side to design a transmit precoding (TPC) matrix for the downlink. We propose a deep reinforcement learning (DRL)-based approach to optimize the TPC matrices, with the goal of maximizing the achievable data rate. We utilize the deep deterministic policy gradient (DDPG) algorithm to handle the continuous action space, and we employ state augmentation techniques to deal with the delayed observations and rewards. We show that the DRL agent is capable of exploiting the time-domain correlations of the channels for constructing accurate TPC matrices. This is because the proposed method is capable of compensating for the effects of delayed CSI in different frequency bands. Furthermore, we study the effect of handovers in the system, and show that the DRL agent is capable of promptly adapting to the environment when a handover occurs.

Tackling Delayed CSI in a Distributed Multi-Satellite MIMO Communication System

In this study, we explore the integration of satellites with ground-based communication networks. Specifically, we analyze downlink data transmission from a constellation of satellites to terrestrial users and address the issue of delayed channel state information (CSI). The satellites cooperate in data transmission within a cluster to create a unified, distributed massive multiple input, multiple output (MIMO) system. The CSI used for this process is inherently outdated, particularly due to the delay from the most distant satellite in the cluster. Therefore, in this paper, we develop a precoding strategy that leverages the long-term characteristics of CSI uncertainty to compensate for the undesirable impact of these unavoidable delays. Our proposed method is computationally efficient and particularly effective in lower frequency bands. As such, it holds significant promise for facilitating the integration of satellite and terrestrial communication, especially within frequency bands of up to 1 GHz.