Bringing Multi-Modal Multi-Task Federated Foundation Models to Education Domain: Prospects and Challenges

Multi-modal multi-task (M3T) foundation models (FMs) have recently shown transformative potential in artificial intelligence, with emerging applications in education. However, their deployment in real-world educational settings is hindered by privacy regulations, data silos, and limited domain-specific data availability. We introduce M3T Federated Foundation Models (FedFMs) for education: a paradigm that integrates federated learning (FL) with M3T FMs to enable collaborative, privacy-preserving training across decentralized institutions while accommodating diverse modalities and tasks. Subsequently, this position paper aims to unveil M3T FedFMs as a promising yet underexplored approach to the education community, explore its potentials, and reveal its related future research directions. We outline how M3T FedFMs can advance three critical pillars of next-generation intelligent education systems: (i) privacy preservation, by keeping sensitive multi-modal student and institutional data local; (ii) personalization, through modular architectures enabling tailored models for students, instructors, and institutions; and (iii) equity and inclusivity, by facilitating participation from underrepresented and resource-constrained entities. We finally identify various open research challenges, including studying of (i) inter-institution heterogeneous privacy regulations, (ii) the non-uniformity of data modalities' characteristics, (iii) the unlearning approaches for M3T FedFMs, (iv) the continual learning frameworks for M3T FedFMs, and (v) M3T FedFM model interpretability, which must be collectively addressed for practical deployment.

An Uncertainty Quantification Framework for Deep Learning-Based Automatic Modulation Classification

Deep learning has been shown to be highly effective for automatic modification classification (AMC), which is a pivotal technology for next-generation cognitive communications. Yet, existing deep learning methods for AMC often lack robust mechanisms for uncertainty quantification (UQ). This limitation restricts their ability to produce accurate and reliable predictions in real-world environments, where signals can be perturbed as a result of several factors such as interference and low signal-to-noise ratios (SNR). To address this problem, we propose a deep ensemble approach that leverages multiple convolutional neural networks (CNNs) to generate predictive distributions, as opposed to point estimates produced by standard deep learning models, which produce statistical characteristics that quantify the uncertainty associated with each prediction. We validate our approach using real-world AMC data, evaluating performance through multiple UQ metrics in a variety of signal environments. Our results show that our proposed ensemble-based framework captures uncertainty to a greater degree compared to previously proposed baselines in multiple settings, including in-distribution samples, out-of-distribution samples, and low SNR signals. These findings highlight the strong UQ capabilities of our ensemble-based AMC approach, paving the way for more robust deep learning-based AMC.

Robust Eavesdropping in the Presence of Adversarial Communications for RF Fingerprinting

Deep learning is an effective approach for performing radio frequency (RF) fingerprinting, which aims to identify the transmitter corresponding to received RF signals. However, beyond the intended receiver, malicious eavesdroppers can also intercept signals and attempt to fingerprint transmitters communicating over a wireless channel. Recent studies suggest that transmitters can counter such threats by embedding deep learning-based transferable adversarial attacks in their signals before transmission. In this work, we develop a time-frequency-based eavesdropper architecture that is capable of withstanding such transferable adversarial perturbations and thus able to perform effective RF fingerprinting. We theoretically demonstrate that adversarial perturbations injected by a transmitter are confined to specific time-frequency regions that are insignificant during inference, directly increasing fingerprinting accuracy on perturbed signals intercepted by the eavesdropper. Empirical evaluations on a real-world dataset validate our theoretical findings, showing that deep learning-based RF fingerprinting eavesdroppers can achieve classification performance comparable to the intended receiver, despite efforts made by the transmitter to deceive the eavesdropper. Our framework reveals that relying on transferable adversarial attacks may not be sufficient to prevent eavesdroppers from successfully fingerprinting transmissions in next-generation deep learning-based communications systems.

FLAME: A Federated Learning Approach for Multi-Modal RF Fingerprinting

Authorization systems are increasingly relying on processing radio frequency (RF) waveforms at receivers to fingerprint (i.e., determine the identity) of the corresponding transmitter. Federated learning (FL) has emerged as a popular paradigm to perform RF fingerprinting in networks with multiple access points (APs), as they allow effective deep learning-based device identification without requiring the centralization of locally collected RF signals stored at multiple APs. Yet, FL algorithms that operate merely on in-phase and quadrature (IQ) time samples incur high convergence rates, resulting in excessive training rounds and inefficient training times. In this work, we propose FLAME: an FL approach for multimodal RF fingerprinting. Our framework consists of simultaneously representing received RF waveforms in multiple complimentary modalities beyond IQ samples in an effort to reduce training times. We theoretically demonstrate the feasibility and efficiency of our methodology and derive a convergence bound that incurs lower loss and thus higher accuracies in the same training round in comparison to single-modal FL-based RF fingerprinting. Extensive empirical evaluations validate our theoretical results and demonstrate the superiority of FLAME with with improvements of up to 30% in comparison to multiple considered baselines.

The Transition from Centralized Machine Learning to Federated Learning for Mental Health in Education: A Survey of Current Methods and Future Directions

Research has increasingly explored the application of artificial intelligence (AI) and machine learning (ML) within the mental health domain to enhance both patient care and healthcare provider efficiency. Given that mental health challenges frequently emerge during early adolescence -- the critical years of high school and college -- investigating AI/ML-driven mental health solutions within the education domain is of paramount importance. Nevertheless, conventional AI/ML techniques follow a centralized model training architecture, which poses privacy risks due to the need for transferring students' sensitive data from institutions, universities, and clinics to central servers. Federated learning (FL) has emerged as a solution to address these risks by enabling distributed model training while maintaining data privacy. Despite its potential, research on applying FL to analyze students' mental health remains limited. In this paper, we aim to address this limitation by proposing a roadmap for integrating FL into mental health data analysis within educational settings. We begin by providing an overview of mental health issues among students and reviewing existing studies where ML has been applied to address these challenges. Next, we examine broader applications of FL in the mental health domain to emphasize the lack of focus on educational contexts. Finally, we propose promising research directions focused on using FL to address mental health issues in the education sector, which entails discussing the synergies between the proposed directions with broader human-centered domains. By categorizing the proposed research directions into short- and long-term strategies and highlighting the unique challenges at each stage, we aim to encourage the development of privacy-conscious AI/ML-driven mental health solutions.

Mitigating Evasion Attacks in Federated Learning-Based Signal Classifiers

There has been recent interest in leveraging federated learning (FL) for radio signal classification tasks. In FL, model parameters are periodically communicated from participating devices, which train on local datasets, to a central server which aggregates them into a global model. While FL has privacy/security advantages due to raw data not leaving the devices, it is still susceptible to adversarial attacks. In this work, we first reveal the susceptibility of FL-based signal classifiers to model poisoning attacks, which compromise the training process despite not observing data transmissions. In this capacity, we develop an attack framework that significantly degrades the training process of the global model. Our attack framework induces a more potent model poisoning attack to the global classifier than existing baselines while also being able to compromise existing server-driven defenses. In response to this gap, we develop Underlying Server Defense of Federated Learning (USD-FL), a novel defense methodology for FL-based signal classifiers. We subsequently compare the defensive efficacy, runtimes, and false positive detection rates of USD-FL relative to existing server-driven defenses, showing that USD-FL has notable advantages over the baseline defenses in all three areas.

How Potent are Evasion Attacks for Poisoning Federated Learning-Based Signal Classifiers?

There has been recent interest in leveraging federated learning (FL) for radio signal classification tasks. In FL, model parameters are periodically communicated from participating devices, training on their own local datasets, to a central server which aggregates them into a global model. While FL has privacy/security advantages due to raw data not leaving the devices, it is still susceptible to several adversarial attacks. In this work, we reveal the susceptibility of FL-based signal classifiers to model poisoning attacks, which compromise the training process despite not observing data transmissions. In this capacity, we develop an attack framework in which compromised FL devices perturb their local datasets using adversarial evasion attacks. As a result, the training process of the global model significantly degrades on in-distribution signals (i.e., signals received over channels with identical distributions at each edge device). We compare our work to previously proposed FL attacks and reveal that as few as one adversarial device operating with a low-powered perturbation under our attack framework can induce the potent model poisoning attack to the global classifier. Moreover, we find that more devices partaking in adversarial poisoning will proportionally degrade the classification performance.

Defending Adversarial Attacks on Deep Learning Based Power Allocation in Massive MIMO Using Denoising Autoencoders

Recent work has advocated for the use of deep learning to perform power allocation in the downlink of massive MIMO (maMIMO) networks. Yet, such deep learning models are vulnerable to adversarial attacks. In the context of maMIMO power allocation, adversarial attacks refer to the injection of subtle perturbations into the deep learning model's input, during inference (i.e., the adversarial perturbation is injected into inputs during deployment after the model has been trained) that are specifically crafted to force the trained regression model to output an infeasible power allocation solution. In this work, we develop an autoencoder-based mitigation technique, which allows deep learning-based power allocation models to operate in the presence of adversaries without requiring retraining. Specifically, we develop a denoising autoencoder (DAE), which learns a mapping between potentially perturbed data and its corresponding unperturbed input. We test our defense across multiple attacks and in multiple threat models and demonstrate its ability to (i) mitigate the effects of adversarial attacks on power allocation networks using two common precoding schemes, (ii) outperform previously proposed benchmarks for mitigating regression-based adversarial attacks on maMIMO networks, (iii) retain accurate performance in the absence of an attack, and (iv) operate with low computational overhead.

A Neural Network-Prepended GLRT Framework for Signal Detection Under Nonlinear Distortions

Many communications and sensing applications hinge on the detection of a signal in a noisy, interference-heavy environment. Signal processing theory yields techniques such as the generalized likelihood ratio test (GLRT) to perform detection when the received samples correspond to a linear observation model. Numerous practical applications exist, however, where the received signal has passed through a nonlinearity, causing significant performance degradation of the GLRT. In this work, we propose prepending the GLRT detector with a neural network classifier capable of identifying the particular nonlinear time samples in a received signal. We show that pre-processing received nonlinear signals using our trained classifier to eliminate excessively nonlinear samples (i) improves the detection performance of the GLRT on nonlinear signals and (ii) retains the theoretical guarantees provided by the GLRT on linear observation models for accurate signal detection.

A Deep Ensemble-based Wireless Receiver Architecture for Mitigating Adversarial Interference in Automatic Modulation Classification

Deep learning-based automatic modulation classification (AMC) models are susceptible to adversarial attacks. Such attacks inject specifically crafted (non-random) wireless interference into transmitted signals to induce erroneous classification predictions. Furthermore, adversarial interference is transferable in black box environments, allowing an adversary to attack multiple deep learning models with a single perturbation crafted for a particular classification model. In this work, we propose a novel wireless receiver architecture to mitigate the effects of adversarial interference in various black box attack environments. We begin by evaluating the architecture uncertainty environment, where we show that adversarial attacks crafted to fool specific AMC DL architectures are not directly transferable to different DL architectures. Next, we consider the domain uncertainty environment, where we show that adversarial attacks crafted on time domain and frequency domain features to not directly transfer to the altering domain. Using these insights, we develop our Assorted Deep Ensemble (ADE) defense, which is an ensemble of deep learning architectures trained on time and frequency domain representations of received signals. Through evaluation on two wireless signal datasets under different sources of uncertainty, we demonstrate that our ADE obtains substantial improvements in AMC classification performance compared with baseline defenses across different adversarial attacks and potencies.