Abstract:With the advancement of the Industrial Internet of Things (IIoT), IIoT services now exhibit diverse Quality of Service (QoS) requirements in terms of delay, determinacy, and security, which pose significant challenges for alignment with existing network resources. Reconfigurable Intelligent Surface (RIS), a key enabling technology for IIoT, not only optimizes signal propagation and enhances network performance but also ensures secure communication and deterministic delays to mitigate threats such as data leakage and eavesdropping. In this paper, we conduct a deterministic delay analysis under a specified decoding error rate for RIS-assisted IIoT communication systems using Stochastic Network Calculus (SNC). We propose an on-demand joint strategy to maximize delay determinacy while guaranteeing secure transmission performance. This is achieved by jointly optimizing the transmit power, channel blocklength (CBL) at the user end, and the phase shift matrix at the RIS. Furthermore, we introduce a State Interdependence-Driven Parameterized Deep Q-Network (SID-PDQN) algorithm to intelligently enforce on-demand performance guarantees. Simulation results demonstrate that the proposed SID-PDQN algorithm significantly enhances network performance compared to baseline methods such as DQN, Dueling-DQN, and DDPG.
Abstract:We characterize a joint CLT of the number of pulls and the sample mean reward of the arms in a stochastic two-armed bandit environment under UCB algorithms. Several implications of this result are in place: (1) a nonstandard CLT of the number of pulls hence pseudo-regret that smoothly interpolates between a standard form in the large arm gap regime and a slow-concentration form in the small arm gap regime, and (2) a heuristic derivation of the sample bias up to its leading order from the correlation between the number of pulls and sample means. Our analysis framework is based on a novel perturbation analysis, which is of broader interest on its own.