Capturing complex hand movements and object interactions using machine learning-powered stretchable smart textile gloves

Accurate real-time tracking of dexterous hand movements and interactions has numerous applications in human-computer interaction, metaverse, robotics, and tele-health. Capturing realistic hand movements is challenging because of the large number of articulations and degrees of freedom. Here, we report accurate and dynamic tracking of articulated hand and finger movements using stretchable, washable smart gloves with embedded helical sensor yarns and inertial measurement units. The sensor yarns have a high dynamic range, responding to low 0.005 % to high 155 % strains, and show stability during extensive use and washing cycles. We use multi-stage machine learning to report average joint angle estimation root mean square errors of 1.21 and 1.45 degrees for intra- and inter-subjects cross-validation, respectively, matching accuracy of costly motion capture cameras without occlusion or field of view limitations. We report a data augmentation technique that enhances robustness to noise and variations of sensors. We demonstrate accurate tracking of dexterous hand movements during object interactions, opening new avenues of applications including accurate typing on a mock paper keyboard, recognition of complex dynamic and static gestures adapted from American Sign Language and object identification.

SemiPFL: Personalized Semi-Supervised Federated Learning Framework for Edge Intelligence

Recent advances in wearable devices and Internet-of-Things (IoT) have led to massive growth in sensor data generated in edge devices. Labeling such massive data for classification tasks has proven to be challenging. In addition, data generated by different users bear various personal attributes and edge heterogeneity, rendering it impractical to develop a global model that adapts well to all users. Concerns over data privacy and communication costs also prohibit centralized data accumulation and training. This paper proposes a novel personalized semi-supervised federated learning (SemiPFL) framework to support edge users having no label or limited labeled datasets and a sizable amount of unlabeled data that is insufficient to train a well-performing model. In this work, edge users collaborate to train a hyper-network in the server, generating personalized autoencoders for each user. After receiving updates from edge users, the server produces a set of base models for each user, which the users locally aggregate them using their own labeled dataset. We comprehensively evaluate our proposed framework on various public datasets and demonstrate that SemiPFL outperforms state-of-art federated learning frameworks under the same assumptions. We also show that the solution performs well for users without labeled datasets or having limited labeled datasets and increasing performance for increased labeled data and number of users, signifying the effectiveness of SemiPFL for handling edge heterogeneity and limited annotation. By leveraging personalized semi-supervised learning, SemiPFL dramatically reduces the need for annotating data and preserving privacy in a wide range of application scenarios, from wearable health to IoT.

Survey on Multi-Agent Q-Learning frameworks for resource management in wireless sensor network

This report aims to survey multi-agent Q-Learning algorithms, analyze different game theory frameworks used, address each framework's applications, and report challenges and future directions. The target application for this study is resource management in the wireless sensor network. In the first section, the author provided an introduction regarding the applications of wireless sensor networks. After that, the author presented a summary of the Q-Learning algorithm, a well-known classic solution for model-free reinforcement learning problems. In the third section, the author extended the Q-Learning algorithm for multi-agent scenarios and discussed its challenges. In the fourth section, the author surveyed sets of game-theoretic frameworks that researchers used to address this problem for resource allocation and task scheduling in the wireless sensor networks. Lastly, the author mentioned some interesting open challenges in this domain.