Optimizing Value of Learning in Task-Oriented Federated Meta-Learning Systems

Federated Learning (FL) has gained significant attention in recent years due to its distributed nature and privacy preserving benefits. However, a key limitation of conventional FL is that it learns and distributes a common global model to all participants, which fails to provide customized solutions for diverse task requirements. Federated meta-learning (FML) offers a promising solution to this issue by enabling devices to finetune local models after receiving a shared meta-model from the server. In this paper, we propose a task-oriented FML framework over non-orthogonal multiple access (NOMA) networks. A novel metric, termed value of learning (VoL), is introduced to assess the individual training needs across devices. Moreover, a task-level weight (TLW) metric is defined based on task requirements and fairness considerations, guiding the prioritization of edge devices during FML training. The formulated problem, to maximize the sum of TLW-based VoL across devices, forms a non-convex mixed-integer non-linear programming (MINLP) challenge, addressed here using a parameterized deep Q-network (PDQN) algorithm to handle both discrete and continuous variables. Simulation results demonstrate that our approach significantly outperforms baseline schemes, underscoring the advantages of the proposed framework.

Joint Secrecy Rate Achieving and Authentication Enhancement via Tag-based Encoding in Chaotic UAV Communication Environment

Secure communication is crucial in many emerging systems enabled by unmanned aerial vehicle (UAV) communication networks. To protect legitimate communication in a chaotic UAV environment, where both eavesdropping and jamming become straightforward from multiple adversaries with line-of-sight signal propagation, a new reliable and integrated physical layer security mechanism is proposed in this paper for a massive multiple-input-multiple-output (MIMO) UAV system. Particularly, a physical layer fingerprint, also called a tag, is first embedded into each message for authentication purpose. We then propose to reuse the tag additionally as a reference to encode each message to ensure secrecy for confidentiality enhancement at a low cost. Specifically, we create a new dual-reference symmetric tag generation mechanism by inputting an encoding-insensitive feature of plaintext along with the key into a hash function. At a legitimate receiver, an expected tag, reliable for decoding, can be symmetrically regenerated based on the received ciphertext, and authentication can be performed by comparing the regenerated reference tag to the received tag. However, an illegitimate receiver can only receive the fuzzy tag which can not be used to decode the received message. Additionally, we introduce artificial noise (AN) to degrade eavesdropping to further decrease message leakage. To verify the efficiency of our proposed tag-based encoding (TBE) scheme, we formulate two optimization problems including ergodic sum secrecy rate maximization and authentication fail probability minimization. The power allocation solutions are derived by difference-of-convex (DC) programming and the Lagrange method, respectively. The simulation results demonstrate the superior performance of the proposed TBE approach compared to the prior AN-aided tag embedding scheme.

Knowledge Sharing-enabled Semantic Rate Maximization for Multi-cell Task-oriented Hybrid Semantic-Bit Communication Networks

In task-oriented semantic communications, the transmitters are designed to deliver task-related semantic information rather than every signal bit to receivers, which alleviates the spectrum pressure by reducing network traffic loads. Effective semantic communications depend on the perfect alignment of shared knowledge between transmitters and receivers, however, the alignment of knowledge cannot always be guaranteed in practice. To tackle this challenge, we propose a novel knowledge sharing-enabled task-oriented hybrid semantic and bit communications mechanism, where a mobile device (MD) can proactively share and upload the task-related mismatched knowledge to associated small base station (SBS). The traditional bit communications can be adopted as an aid to transmit the rest data related to unshared mismatched knowledge to guarantee the effective execution of target tasks. Considering the heterogeneous transceivers in multi-cell networks, target task demands, and channel conditions, an optimization problem is formulated to maximize the generalized effective semantic transmission rate of all MDs by jointly optimizing knowledge sharing, semantic extraction ratio, and SBS association, while satisfying the semantic accuracy requirements and delay tolerances of MD target tasks. The formulated mixed integer nonlinear programming problem is decomposed into multiple subproblems equivalently. An optimum algorithm is proposed and another efficient algorithm is further developed using hierarchical class partitioning and monotonic optimization. Simulation results demonstrate the validity and superior performance of proposed solutions.

Stackelberg Game Based Performance Optimization in Digital Twin Assisted Federated Learning over NOMA Networks

Despite the advantage of preserving data privacy, federated learning (FL) still suffers from the straggler issue due to the limited computing resources of distributed clients and the unreliable wireless communication environment. By effectively imitating the distributed resources, digital twin (DT) shows great potential in alleviating this issue. In this paper, we leverage DT in the FL framework over non-orthogonal multiple access (NOMA) network to assist FL training process, considering malicious attacks on model updates from clients. A reputationbased client selection scheme is proposed, which accounts for client heterogeneity in multiple aspects and effectively mitigates the risks of poisoning attacks in FL systems. To minimize the total latency and energy consumption in the proposed system, we then formulate a Stackelberg game by considering clients and the server as the leader and the follower, respectively. Specifically, the leader aims to minimize the energy consumption while the objective of the follower is to minimize the total latency during FL training. The Stackelberg equilibrium is achieved to obtain the optimal solutions. We first derive the strategies for the followerlevel problem and include them in the leader-level problem which is then solved via problem decomposition. Simulation results verify the superior performance of the proposed scheme.

Collaborative Knowledge Sharing-empowered Effective Semantic Rate Maximization for Two-tier Semantic-Bit Communication Networks

Effective task-oriented semantic communications relies on perfect knowledge alignment between transmitters and receivers for accurate recovery of task-related semantic information, which can be susceptible to knowledge misalignment and performance degradation in practice. To tackle this issue, continual knowledge updating and sharing are crucial to adapt to evolving task and user related demands, despite the incurred resource overhead and increased latency. In this paper, we propose a novel collaborative knowledge sharing-empowered semantic transmission mechanism in a two-tier edge network, exploiting edge cooperations and bit communications to address KB mismatch. By deriving a generalized effective semantic transmission rate (GESTR) that considers both semantic accuracy and overhead, we formulate a mixed integer nonlinear programming problem to maximize GESTR of all mobile devices by optimizing knowledge sharing decisions, extraction ratios, and BS/subchannel allocations, subject to task accuracy and delay requirements. The joint optimum solution can be obtained by proposed fractional programming based branch and bound algorithm and modified Kuhn-Munkres algorithm efficiently. Simulation results demonstrate the superior performance of proposed solution, especially in low signal-to-noise conditions.

Power-Efficient Optimization for Coexisting Semantic and Bit-Based Users in NOMA Networks

Semantic communication focuses on transmitting the meaning of data, aiming for efficient, relevant communication, while non-orthogonal multiple access (NOMA) enhances spectral efficiency by allowing multiple users to share the same spectrum. Integrating semantic users into a NOMA network with bit-based users improves both transmission and spectrum efficiency. However, the performance metric for semantic communication differs significantly from that of traditional communication, posing challenges in simultaneously meeting individual user demands and minimizing transmission power, especially in scenarios with coexisting semantic and bit-based users. Furthermore, the different hardware architectures of semantic and bit-based users complicate the implementation of successive interference cancellation (SIC). To address these challenges, in this paper, we propose a clustered framework to mitigate the complexity of SIC and two multiple access (MA) schemes, e.g., pure cluster-based NOMA (P-CNOMA) and hybrid cluster-based NOMA (H-CNOMA), to minimize the total transmission power. The P-CNOMA scheme can achieve the minimum transmission power, but may not satisfy the high quality of service (QoS) requirement. In contrast, H-CNOMA addresses these issues with a slight increase in power and a reduced semantic rate. These two schemes complement each other, enabling an adaptive MA selection mechanism that adapts to specific network conditions and user requirements.

Vinci: A Real-time Embodied Smart Assistant based on Egocentric Vision-Language Model

We introduce Vinci, a real-time embodied smart assistant built upon an egocentric vision-language model. Designed for deployment on portable devices such as smartphones and wearable cameras, Vinci operates in an "always on" mode, continuously observing the environment to deliver seamless interaction and assistance. Users can wake up the system and engage in natural conversations to ask questions or seek assistance, with responses delivered through audio for hands-free convenience. With its ability to process long video streams in real-time, Vinci can answer user queries about current observations and historical context while also providing task planning based on past interactions. To further enhance usability, Vinci integrates a video generation module that creates step-by-step visual demonstrations for tasks that require detailed guidance. We hope that Vinci can establish a robust framework for portable, real-time egocentric AI systems, empowering users with contextual and actionable insights. We release the complete implementation for the development of the device in conjunction with a demo web platform to test uploaded videos at https://github.com/OpenGVLab/vinci.

Contact Compliance Visuo-Proprioceptive Policy for Contact-Rich Manipulation with Cost-Efficient Haptic Hand-Arm Teleoperation System

Learning robot manipulation skills in real-world environments is extremely challenging. Robots learning manipulation skills in real-world environments is extremely challenging. Recent research on imitation learning and visuomotor policies has significantly enhanced the ability of robots to perform manipulation tasks. In this paper, we propose Admit Policy, a visuo-proprioceptive imitation learning framework with force compliance, designed to reduce contact force fluctuations during robot execution of contact-rich manipulation tasks. This framework also includes a hand-arm teleoperation system with vibrotactile feedback for efficient data collection. Our framework utilizes RGB images, robot joint positions, and contact forces as observations and leverages a consistency-constrained teacher-student probabilistic diffusion model to generate future trajectories for end-effector positions and contact forces. An admittance model is then employed to track these trajectories, enabling effective force-position control across various tasks.We validated our framework on five challenging contact-rich manipulation tasks. Among these tasks, while improving success rates, our approach most significantly reduced the mean contact force required to complete the tasks by up to 53.92% and decreased the standard deviation of contact force fluctuations by 76.51% compared to imitation learning algorithms without dynamic contact force prediction and tracking.

Improving Channel Resilience for Task-Oriented Semantic Communications: A Unified Information Bottleneck Approach

Task-oriented semantic communications (TSC) enhance radio resource efficiency by transmitting task-relevant semantic information. However, current research often overlooks the inherent semantic distinctions among encoded features. Due to unavoidable channel variations from time and frequency-selective fading, semantically sensitive feature units could be more susceptible to erroneous inference if corrupted by dynamic channels. Therefore, this letter introduces a unified channel-resilient TSC framework via information bottleneck. This framework complements existing TSC approaches by controlling information flow to capture fine-grained feature-level semantic robustness. Experiments on a case study for real-time subchannel allocation validate the framework's effectiveness.

Active Admittance Control with Iterative Learning for General-Purpose Contact-Rich Manipulation

Force interaction is inevitable when robots face multiple operation scenarios. How to make the robot competent in force control for generalized operations such as multi-tasks still remains a challenging problem. Aiming at the reproducibility of interaction tasks and the lack of a generalized force control framework for multi-task scenarios, this paper proposes a novel hybrid control framework based on active admittance control with iterative learning parameters-tunning mechanism. The method adopts admittance control as the underlying algorithm to ensure flexibility, and iterative learning as the high-level algorithm to regulate the parameters of the admittance model. The whole algorithm has flexibility and learning ability, which is capable of achieving the goal of excellent versatility. Four representative interactive robot manipulation tasks are chosen to investigate the consistency and generalisability of the proposed method. Experiments are designed to verify the effectiveness of the whole framework, and an average of 98.21% and 91.52% improvement of RMSE is obtained relative to the traditional admittance control as well as the model-free adaptive control, respectively.