Joint Age-State Belief is All You Need: Minimizing AoII via Pull-Based Remote Estimation

Age of incorrect information (AoII) is a recently proposed freshness and mismatch metric that penalizes an incorrect estimation along with its duration. Therefore, keeping track of AoII requires the knowledge of both the source and estimation processes. In this paper, we consider a time-slotted pull-based remote estimation system under a sampling rate constraint where the information source is a general discrete-time Markov chain (DTMC) process. Moreover, packet transmission times from the source to the monitor are non-zero which disallows the monitor to have perfect information on the actual AoII process at any time. Hence, for this pull-based system, we propose the monitor to maintain a sufficient statistic called {\em belief} which stands for the joint distribution of the age and source processes to be obtained from the history of all observations. Using belief, we first propose a maximum a posteriori (MAP) estimator to be used at the monitor as opposed to existing martingale estimators in the literature. Second, we obtain the optimality equations from the belief-MDP (Markov decision process) formulation. Finally, we propose two belief-dependent policies one of which is based on deep reinforcement learning, and the other one is a threshold-based policy based on the instantaneous expected AoII.

Multi-Threshold AoII-Optimum Sampling Policies for CTMC Information Sources

We study push-based sampling and transmission policies for a status update system consisting of a general finite-state continuous-time Markov chain (CTMC) information source with known dynamics, with the goal of minimizing the average age of incorrect information (AoII). The problem setting we investigate involves an exponentially distributed delay channel for transmissions and a constraint on the average sampling rate. We first show that the optimum sampling and transmission policy is a 'multi-threshold policy', where the thresholds depend on both the estimation value and the state of the original process, and sampling and transmission need to be initiated when the instantaneous AoII exceeds the corresponding threshold, called the estimation- and state-aware transmission (ESAT) policy. Subsequently, we formulate the problem of finding the thresholds as a constrained semi-Markov decision process (CSMDP) and the Lagrangian approach. Additionally, we propose two lower complexity sub-optimum policies, namely the estimation-aware transmission (EAT) policy, and the single-threshold (ST) policy, for which it is possible to obtain these thresholds for CTMCs with relatively larger number of states. The underlying CSMDP formulation relies on the 'multi-regime phase-type' (MRPH) distribution which is a generalization of the well-known phase-type distribution, which allows us to obtain the distribution of time until absorption in a CTMC whose transition rates change with respect to time in a piece-wise manner. The effectiveness of the proposed ESAT, EAT and ST sampling and transmission policies are shown through numerical examples, along with comparisons with a baseline scheme that transmits packets according to a Poisson process in out-of-sync periods.

AoII-Optimum Sampling of CTMC Information Sources Under Sampling Rate Constraints

We consider a sensor that samples an $N$-state continuous-time Markov chain (CTMC)-based information source process, and transmits the observed state of the source, to a remote monitor tasked with timely tracking of the source process. The mismatch between the source and monitor processes is quantified by age of incorrect information (AoII), which penalizes the mismatch as it stays longer, and our objective is to minimize the average AoII under an average sampling rate constraint. We assume a perfect reverse channel and hence the sensor has information of the estimate while initiating a transmission or preempting an ongoing transmission. First, by modeling the problem as an average cost constrained semi-Markov decision process (CSMDP), we show that the structure of the problem gives rise to an optimum threshold policy for which the sensor initiates a transmission once the AoII exceeds a threshold depending on the instantaneous values of both the source and monitor processes. However, due to the high complexity of obtaining the optimum policy in this general setting, we consider a relaxed problem where the thresholds are allowed to be dependent only on the estimate. We show that this relaxed problem can be solved with a novel CSMDP formulation based on the theory of absorbing MCs, with a computational complexity of $\mathcal{O}(N^4)$, allowing one to obtain optimum policies for general CTMCs with over a hundred states.

Modeling AoII in Push- and Pull-Based Sampling of Continuous Time Markov Chains

Age of incorrect information (AoII) has recently been proposed as an alternative to existing information freshness metrics for real-time sampling and estimation problems involving information sources that are tracked by remote monitors. Different from existing metrics, AoII penalizes the incorrect information by increasing linearly with time as long as the source and the monitor are de-synchronized, and is reset when they are synchronized back. While AoII has generally been investigated for discrete time information sources, we develop a novel analytical model in this paper for push- and pull-based sampling and transmission of a continuous time Markov chain (CTMC) process. In the pull-based model, the sensor starts transmitting information on the observed CTMC only when a pull request from the monitor is received. On the other hand, in the push-based scenario, the sensor, being aware of the AoII process, samples and transmits when the AoII process exceeds a random threshold. The proposed analytical model for both scenarios is based on the construction of a discrete time MC (DTMC) making state transitions at the embedded epochs of synchronization points, using the theory of absorbing CTMCs, and in particular phase-type distributions. For a given sampling policy, analytical models to obtain the mean AoII and the average sampling rate are developed. Numerical results are presented to validate the analytical model as well as to provide insight on optimal sampling policies under sampling rate constraints.