Data Similarity-Based One-Shot Clustering for Multi-Task Hierarchical Federated Learning

We address the problem of cluster identity estimation in a hierarchical federated learning setting in which users work toward learning different tasks. To overcome the challenge of task heterogeneity, users need to be grouped in a way such that users with the same task are in the same group, conducting training together, while sharing the weights of feature extraction layers with the other groups. Toward that end, we propose a one-shot clustering algorithm that can effectively identify and group users based on their data similarity. This enables more efficient collaboration and sharing of a common layer representation within the federated learning system. Our proposed algorithm not only enhances the clustering process, but also overcomes challenges related to privacy concerns, communication overhead, and the need for prior knowledge about learning models or loss function behaviors. We validate our proposed algorithm using various datasets such as CIFAR-10 and Fashion MNIST, and show that it outperforms the baseline in terms of accuracy and variance reduction.

Age and Value of Information Optimization for Systems with Multi-Class Updates

Received samples of a stochastic process are processed by a server for delivery as updates to a monitor. Each sample belongs to a class that specifies a distribution for its processing time and a function that describes how the value of the processed update decays with age at the monitor. The class of a sample is identified when the processed update is delivered. The server implements a form of M/G/1/1 blocking queue; samples arriving at a busy server are discarded and samples arriving at an idle server are subject to an admission policy that depends on the age and class of the prior delivered update. For the delivered updates, we characterize the average age of information (AoI) and average value of information (VoI). We derive the optimal stationary policy that minimizes the convex combination of the AoI and (negative) VoI. It is shown that the policy has a threshold structure, in which a new sample is allowed to arrive to the server only if the previous update's age and value difference surpasses a certain threshold that depends on the specifics of the value function and system statistics.

Private Status Updating with Erasures: A Case for Retransmission Without Resampling

A status updating system is considered in which a source updates a destination over an erasure channel. The utility of the updates is measured through a function of their age-of-information (AoI), which assesses their freshness. Correlated with the status updates is another process that needs to be kept private from the destination. Privacy is measured through a leakage function that depends on the amount and time of the status updates received: stale updates are more private than fresh ones. Different from most of the current AoI literature, a post-sampling waiting time is introduced in order to provide a privacy cover at the expense of AoI. More importantly, it is also shown that, depending on the leakage budget and the channel statistics, it can be useful to retransmit stale status updates following erasure events without resampling fresh ones.

Delay Sensitive Hierarchical Federated Learning with Stochastic Local Updates

The impact of local averaging on the performance of federated learning (FL) systems is studied in the presence of communication delay between the clients and the parameter server. To minimize the effect of delay, clients are assigned into different groups, each having its own local parameter server (LPS) that aggregates its clients' models. The groups' models are then aggregated at a global parameter server (GPS) that only communicates with the LPSs. Such setting is known as hierarchical FL (HFL). Different from most works in the literature, the number of local and global communication rounds in our work is randomly determined by the (different) delays experienced by each group of clients. Specifically, the number of local averaging rounds are tied to a wall-clock time period coined the sync time $S$, after which the LPSs synchronize their models by sharing them with the GPS. Such sync time $S$ is then reapplied until a global wall-clock time is exhausted.

Timely Multi-Process Estimation with Erasures

We consider a multi-process remote estimation system observing $K$ independent Ornstein-Uhlenbeck processes. In this system, a shared sensor samples the $K$ processes in such a way that the long-term average sum mean square error (MSE) is minimized. The sensor operates under a total sampling frequency constraint $f_{\max}$ and samples the processes according to a Maximum-Age-First (MAF) schedule. The samples from all processes consume random processing delays, and then are transmitted over an erasure channel with probability $\epsilon$. Aided by optimal structural results, we show that the optimal sampling policy, under some conditions, is a \emph{threshold policy}. We characterize the optimal threshold and the corresponding optimal long-term average sum MSE as a function of $K$, $f_{\max}$, $\epsilon$, and the statistical properties of the observed processes.

Timely Updating with Intermittent Energy and Data for Multiple Sources over Erasure Channels

A status updating system is considered in which multiple data sources generate packets to be delivered to a destination through a shared energy harvesting sensor. Only one source's data, when available, can be transmitted by the sensor at a time, subject to energy availability. Transmissions are prune to erasures, and each successful transmission constitutes a status update for its corresponding source at the destination. The goal is to schedule source transmissions such that the collective long-term average age-of-information (AoI) is minimized. AoI is defined as the time elapsed since the latest successfully-received data has been generated at its source. To solve this problem, the case with a single source is first considered, with a focus on threshold waiting policies, in which the sensor attempts transmission only if the time until both energy and data are available grows above a certain threshold. The distribution of the AoI is fully characterized under such a policy. This is then used to analyze the performance of the multiple sources case under maximum-age-first scheduling, in which the sensor's resources are dedicated to the source with the maximum AoI at any given time. The achievable collective long-term average AoI is derived in closed-form. Multiple numerical evaluations are demonstrated to show how the optimal threshold value behaves as a function of the system parameters, and showcase the benefits of a threshold-based waiting policy with intermittent energy and data arrivals.

Timely Status Updating Over Erasure Channels Using an Energy Harvesting Sensor: Single and Multiple Sources

A status updating system is considered in which data from multiple sources are sampled by an energy harvesting sensor and transmitted to a remote destination through an erasure channel. The goal is to deliver status updates of all sources in a timely manner, such that the cumulative long-term average age-of-information (AoI) is minimized. The AoI for each source is defined as the time elapsed since the generation time of the latest successful status update received at the destination from that source. Transmissions are subject to energy availability, which arrives in units according to a Poisson process, with each energy unit capable of carrying out one transmission from only one source. The sensor is equipped with a unit-sized battery to save the incoming energy. A scheduling policy is designed in order to determine which source is sampled using the available energy. The problem is studied in two main settings: no erasure status feedback, and perfect instantaneous feedback.

Timely Private Information Retrieval

We introduce the problem of \emph{timely} private information retrieval (PIR) from $N$ non-colluding and replicated servers. In this problem, a user desires to retrieve a message out of $M$ messages from the servers, whose contents are continuously updating. The retrieval process should be executed in a timely manner such that no information is leaked about the identity of the message. To assess the timeliness, we use the \emph{age of information} (AoI) metric. Interestingly, the timely PIR problem reduces to an AoI minimization subject to PIR constraints under \emph{asymmetric traffic}. We explicitly characterize the optimal tradeoff between the PIR rate and the AoI metric (peak AoI or average AoI) for the case of $N=2$, $M=3$. Further, we provide some structural insights on the general problem with arbitrary $N$, $M$.

Download Cost of Private Updating

We consider the problem of privately updating a message out of $K$ messages from $N$ replicated and non-colluding databases. In this problem, a user has an outdated version of the message $\hat{W}_\theta$ of length $L$ bits that differ from the current version $W_\theta$ in at most $f$ bits. The user needs to retrieve $W_\theta$ correctly using a private information retrieval (PIR) scheme with the least number of downloads without leaking any information about the message index $\theta$ to any individual database. To that end, we propose a novel achievable scheme based on \emph{syndrome decoding}. Specifically, the user downloads the syndrome corresponding to $W_\theta$, according to a linear block code with carefully designed parameters, using the optimal PIR scheme for messages with a length constraint. We derive lower and upper bounds for the optimal download cost that match if the term $\log_2\left(\sum_{i=0}^f \binom{L}{i}\right)$ is an integer. Our results imply that there is a significant reduction in the download cost if $f < \frac{L}{2}$ compared with downloading $W_\theta$ directly using classical PIR approaches without taking the correlation between $W_\theta$ and $\hat{W}_\theta$ into consideration.

Timely Transmissions Using Optimized Variable Length Coding

A status updating system is considered in which a variable length code is used to transmit messages to a receiver over a noisy channel. The goal is to optimize the codewords lengths such that successfully-decoded messages are timely. That is, such that the age-of-information (AoI) at the receiver is minimized. A hybrid ARQ (HARQ) scheme is employed, in which variable-length incremental redundancy (IR) bits are added to the originally-transmitted codeword until decoding is successful. With each decoding attempt, a non-zero processing delay is incurred. The optimal codewords lengths are analytically derived utilizing a sequential differential optimization (SDO) framework. The framework is general in that it only requires knowledge of an analytical expression of the positive feedback (ACK) probability as a function of the codeword length.