Personalised 3D Human Digital Twin with Soft-Body Feet for Walking Simulation

With the increasing use of assistive robots in rehabilitation and assisted mobility of human patients, there has been a need for a deeper understanding of human-robot interactions particularly through simulations, allowing an understanding of these interactions in a digital environment. There is an emphasis on accurately modelling personalised 3D human digital twins in these simulations, to glean more insights on human-robot interactions. In this paper, we propose to integrate personalised soft-body feet, generated using the motion capture data of real human subjects, into a skeletal model and train it with a walking control policy. Through evaluation using ground reaction force and joint angle results, the soft-body feet were able to generate ground reaction force results comparable to real measured data and closely follow joint angle results of the bare skeletal model and the reference motion. This presents an interesting avenue to produce a dynamically accurate human model in simulation driven by their own control policy while only seeing kinematic information during training.

Design and Evaluation of a Compact 3D End-effector Assistive Robot for Adaptive Arm Support

We developed a 3D end-effector type of upper limb assistive robot, named as Assistive Robotic Arm Extender (ARAE), that provides transparency movement and adaptive arm support control to achieve home-based therapy and training in the real environment. The proposed system composes five degrees of freedom, including three active motors and two passive joints at the end-effector module. The core structure of the system is based on a parallel mechanism. The kinematic and dynamic modeling are illustrated in detail. The proposed adaptive arm support control framework calculates the compensated force based on the estimated human arm posture in 3D space. It firstly estimates human arm joint angles using two proposed methods: fixed torso and sagittal plane models without using external sensors such as IMUs, magnetic sensors, or depth cameras. The experiments were carried out to evaluate the performance of the two proposed angle estimation methods. Then, the estimated human joint angles were input into the human upper limb dynamics model to derive the required support force generated by the robot. The muscular activities were measured to evaluate the effects of the proposed framework. The obvious reduction of muscular activities was exhibited when participants were tested with the ARAE under an adaptive arm gravity compensation control framework. The overall results suggest that the ARAE system, when combined with the proposed control framework, has the potential to offer adaptive arm support. This integration could enable effective training with Activities of Daily Living (ADLs) and interaction with real environments.

ExTraCT -- Explainable Trajectory Corrections from language inputs using Textual description of features

Natural language provides an intuitive and expressive way of conveying human intent to robots. Prior works employed end-to-end methods for learning trajectory deformations from language corrections. However, such methods do not generalize to new initial trajectories or object configurations. This work presents ExTraCT, a modular framework for trajectory corrections using natural language that combines Large Language Models (LLMs) for natural language understanding and trajectory deformation functions. Given a scene, ExTraCT generates the trajectory modification features (scene-specific and scene-independent) and their corresponding natural language textual descriptions for the objects in the scene online based on a template. We use LLMs for semantic matching of user utterances to the textual descriptions of features. Based on the feature matched, a trajectory modification function is applied to the initial trajectory, allowing generalization to unseen trajectories and object configurations. Through user studies conducted both in simulation and with a physical robot arm, we demonstrate that trajectories deformed using our method were more accurate and were preferred in about 80\% of cases, outperforming the baseline. We also showcase the versatility of our system in a manipulation task and an assistive feeding task.

Graceful User Following for Mobile Balance Assistive Robot in Daily Activities Assistance

Numerous diseases and aging can cause degeneration of people's balance ability resulting in limited mobility and even high risks of fall. Robotic technologies can provide more intensive rehabilitation exercises or be used as assistive devices to compensate for balance ability. However, With the new healthcare paradigm shifting from hospital care to home care, there is a gap in robotic systems that can provide care at home. This paper introduces Mobile Robotic Balance Assistant (MRBA), a compact and cost-effective balance assistive robot that can provide both rehabilitation training and activities of daily living (ADLs) assistance at home. A three degrees of freedom (3-DoF) robotic arm was designed to mimic the therapist arm function to provide balance assistance to the user. To minimize the interference to users' natural pelvis movements and gait patterns, the robot must have a Human-Robot Interface(HRI) that can detect user intention accurately and follow the user's movement smoothly and timely. Thus, a graceful user following control rule was proposed. The overall control architecture consists of two parts: an observer for human inputs estimation and an LQR-based controller with disturbance rejection. The proposed controller is validated in high-fidelity simulation with actual human trajectories, and the results successfully show the effectiveness of the method in different walking modes.

Bioinspired Straight Walking Task-Space Planner

Despite the improvements in humanoids robots over the last decades, they are still far behind compared to human locomotor abilities. Their performance limitations can be partially attributed to the hardware, but the primary constraint has been the understanding of bipedal dynamics. Based on the recently developed model of potential energy for bipedal structures, this work proposes a task-space planner for human-like straight locomotion. The proposed architecture is based on potential energy model and employs locomotor strategies from human data as a reference behaviour. The model generates Centre of Mass (CoM) trajectories, foot swing trajectories and the Base of Support (BoS). Their calculation relies on the knowledge of the desired speed, initial posture, height, weight, number of steps and the angle between the foot and the ground during heel-strike. The data show that the proposed architecture can generate behaviour in line with human walking strategies for both the CoM and the foot swing. Although the planned trajectory is not smooth compared to human trajectories, the proposed model significantly reduces the error in the estimation of the CoM vertical trajectory. Moreover, the proposed planner can generate a single stride in less than 140 ms and sequences of 10 strides in less than 600 ms, it allows an online task-space planning for locomotion. Lastly, the proposed architecture is also supported by analogies with current theories on human motor control of locomotion.

Deployment of the Saddle Space Transformation in Tracking the Base of Support

Balance is the fundamental skill behind human locomotion, and its impairment is the principal indicator of self-perceived disability. Despite significant improvements in balance assessment, there is still large incidence of fall related injuries among elderlies. The Base of Support (BoS) is a popular method for bipedal stability assessment, but its accuracy depends on the accuracy the BoS geometry measurement. This work presents a method to ease the BoS tracking by the identification of a reference frame that allows to define postural models of the BoS geometry. Although we also propose a geometry based on the geometry determined from centre of pressure range of motion within the foot obtained from literature, this methodology can be used with other models (i.e., the feasible base of support). The model has been tested with 12 healthy subjects, which have been asked to explore their stability in six different postures. The results show that the model can accurate deform the geometry of the BoS to adapt its shape to the different postures, which can remove the necessity of force/torque sensors in some application. Potentially the proposed method can be also applied to describe any posture dependent attribute (e.g., gravitational forces), and it can be also applied to bipedal robots. Therefore, it constitutes a novel mathematical tool that can be deployed to develop both better sensors and models for bipeds. For example, it can be used with the Extrapolated CoM model to evaluate dynamic stability from the body kinematics.

The Strange Attractor of Bipedal Locomotion and Consequences on Motor Control

Despite decades of study, the mechanisms that determine human locomotion are still unknown, available models and motor control theories can only partially capture the phenomenon. This is probably the principal cause of the reduced efficacy of lower limbs rehabilitation therapies. Recently, it has been proposed that human locomotion may be planned in the task-space by taking advantage of the gravitational pull acting on the Centre of Mass (CoM) that we have used to formulate a task-space planner for straight locomotion at a constant speed. The proposed model represents the CoM transversal trajectory as simple harmonic oscillator moving forward at a constant speed. On the other hand, the vertical trajectory of the CoM is controlled through the ankle strategies. Our solution is composed of closed-form equations which can plan human-like trajectories for both the CoM and the foot swing. The model output can be seen as the optimal trajectory determined based on the average behaviour of 12 healthy subjects walking at three self-selected speeds. Furthermore, the planner formulation is compatible with an extended formulation of the Passive Motion Paradigm which enables us to design a hierarchical architecture of semi-autonomous controllers. The final architecture can also describe the motor primitives as a particular case of dynamic primitives, shows strong parallels with the nervous system organization, and is compatible with the optimal feedback controller theory.