Control the Soft Robot Arm with its Physical Twin

To exploit the compliant capabilities of soft robot arms we require controller which can exploit their physical capabilities. Teleoperation, leveraging a human in the loop, is a key step towards achieving more complex control strategies. Whilst teleoperation is widely used for rigid robots, for soft robots we require teleoperation methods where the configuration of the whole body is considered. We propose a method of using an identical 'physical twin', or demonstrator of the robot. This tendon robot can be back-driven, with the tendon lengths providing configuration perception, and enabling a direct mapping of tendon lengths for the execture. We demonstrate how this teleoperation across the entire configuration of the robot enables complex interactions with exploit the envrionment, such as squeezing into gaps. We also show how this method can generalize to robots which are a larger scale that the physical twin, and how, tuneability of the stiffness properties of the physical twin simplify its use.

Online Imitation Learning for Manipulation via Decaying Relative Correction through Teleoperation

Teleoperated robotic manipulators enable the collection of demonstration data, which can be used to train control policies through imitation learning. However, such methods can require significant amounts of training data to develop robust policies or adapt them to new and unseen tasks. While expert feedback can significantly enhance policy performance, providing continuous feedback can be cognitively demanding and time-consuming for experts. To address this challenge, we propose to use a cable-driven teleoperation system which can provide spatial corrections with 6 degree of freedom to the trajectories generated by a policy model. Specifically, we propose a correction method termed Decaying Relative Correction (DRC) which is based upon the spatial offset vector provided by the expert and exists temporarily, and which reduces the intervention steps required by an expert. Our results demonstrate that DRC reduces the required expert intervention rate by 30\% compared to a standard absolute corrective method. Furthermore, we show that integrating DRC within an online imitation learning framework rapidly increases the success rate of manipulation tasks such as raspberry harvesting and cloth wiping.

Embodied Manipulation with Past and Future Morphologies through an Open Parametric Hand Design

A human-shaped robotic hand offers unparalleled versatility and fine motor skills, enabling it to perform a broad spectrum of tasks with precision, power and robustness. Across the paleontological record and animal kingdom we see a wide range of alternative hand and actuation designs. Understanding the morphological design space and the resulting emergent behaviors can not only aid our understanding of dexterous manipulation and its evolution, but also assist design optimization, achieving, and eventually surpassing human capabilities. Exploration of hand embodiment has to date been limited by inaccessibility of customizable hands in the real-world, and by the reality gap in simulation of complex interactions. We introduce an open parametric design which integrates techniques for simplified customization, fabrication, and control with design features to maximize behavioral diversity. Non-linear rolling joints, anatomical tendon routing, and a low degree-of-freedom, modulating, actuation system, enable rapid production of single-piece 3D printable hands without compromising dexterous behaviors. To demonstrate this, we evaluated the design's low-level behavior range and stability, showing variable stiffness over two orders of magnitude. Additionally, we fabricated three hand designs: human, mirrored human with two thumbs, and aye-aye hands. Manipulation tests evaluate the variation in each hand's proficiency at handling diverse objects, and demonstrate emergent behaviors unique to each design. Overall, we shed light on new possible designs for robotic hands, provide a design space to compare and contrast different hand morphologies and structures, and share a practical and open-source design for exploring embodied manipulation.

Vision-Language-Action Model and Diffusion Policy Switching Enables Dexterous Control of an Anthropomorphic Hand

To advance autonomous dexterous manipulation, we propose a hybrid control method that combines the relative advantages of a fine-tuned Vision-Language-Action (VLA) model and diffusion models. The VLA model provides language commanded high-level planning, which is highly generalizable, while the diffusion model handles low-level interactions which offers the precision and robustness required for specific objects and environments. By incorporating a switching signal into the training-data, we enable event based transitions between these two models for a pick-and-place task where the target object and placement location is commanded through language. This approach is deployed on our anthropomorphic ADAPT Hand 2, a 13DoF robotic hand, which incorporates compliance through series elastic actuation allowing for resilience for any interactions: showing the first use of a multi-fingered hand controlled with a VLA model. We demonstrate this model switching approach results in a over 80\% success rate compared to under 40\% when only using a VLA model, enabled by accurate near-object arm motion by the VLA model and a multi-modal grasping motion with error recovery abilities from the diffusion model.

S2C2A: A Flexible Task Space Planning and Control Strategy for Modular Soft Robot Arms

Modular soft robot arms (MSRAs) are composed of multiple independent modules connected in a sequence. Due to their modular structure and high degrees of freedom (DOFs), these modules can simultaneously bend at different angles in various directions, enabling complex deformation. This capability allows MSRAs to perform more intricate tasks than single module robots. However, the modular structure also induces challenges in accurate planning, modeling, and control. Nonlinearity, hysteresis, and gravity complicate the physical model, while the modular structure and increased DOFs further lead to accumulative errors along the sequence. To address these challenges, we propose a flexible task space planning and control strategy for MSRAs, named S2C2A (State to Configuration to Action). Our approach formulates an optimization problem, S2C (State to Configuration planning), which integrates various loss functions and a forward MSRA model to generate configuration trajectories based on target MSRA states. Given the model complexity, we leverage a biLSTM network as the forward model. Subsequently, a configuration controller C2A (Configuration to Action control) is implemented to follow the planned configuration trajectories, leveraging only inaccurate internal sensing feedback. Both a biLSTM network and a physical model are utilized for configuration control. We validated our strategy using a cable-driven MSRA, demonstrating its ability to perform diverse offline tasks such as position control, orientation control, and obstacle avoidance. Furthermore, our strategy endows MSRA with online interaction capability with targets and obstacles. Future work will focus on addressing MSRA challenges, such as developing more accurate physical models and reducing configuration estimation errors along the module sequence.

Robotic Optimization of Powdered Beverages Leveraging Computer Vision and Bayesian Optimization

The growing demand for innovative research in the food industry is driving the adoption of robots in large-scale experimentation, as it offers increased precision, replicability, and efficiency in product manufacturing and evaluation. To this end, we introduce a robotic system designed to optimize food product quality, focusing on powdered cappuccino preparation as a case study. By leveraging optimization algorithms and computer vision, the robot explores the parameter space to identify the ideal conditions for producing a cappuccino with the best foam quality. The system also incorporates computer vision-driven feedback in a closed-loop control to further improve the beverage. Our findings demonstrate the effectiveness of robotic automation in achieving high repeatability and extensive parameter exploration, paving the way for more advanced and reliable food product development.

From Rigid to Soft Robotic Approaches for Minimally Invasive Neurosurgery

Robotic assistance has significantly improved the outcomes of open microsurgery and rigid endoscopic surgery, however is yet to make an impact in flexible endoscopic neurosurgery. Some of the most common intracranial procedures for treatment of hydrocephalus and tumors stand to benefit from increased dexterity and reduced invasiveness offered by robotic systems that can navigate in the deep ventricular system of the brain. We review a spectrum of flexible robotic devices, from the traditional highly actuated approach, to more novel and bio-inspired mechanisms for safe navigation. For each technology, we identify the operating principle and are able to evaluate the potential for minimally invasive surgical applications. Overall, rigid-type continuum robots have seen the most development, however, approaches combining rigid and soft robotic principles into innovative devices, are ideally situated to address safety and complexity limitations after future design evolution. We also observe a number of related challenges in the field, from surgeon-robot interfaces to robot evaluation procedures. Fundamentally, the challenges revolve around a guarantee of safety in robotic devices with the prerequisites to assist and improve upon surgical tasks. With innovative designs, materials and evaluation techniques emerging, we see potential impacts in the next 5--10 years.

Robust Anthropomorphic Robotic Manipulation through Biomimetic Distributed Compliance

The impressive capabilities of humans to robustly perform manipulation relies on compliant interactions, enabled through the structure and materials spatially distributed in our hands. We propose by mimicking this distributed compliance in an anthropomorphic robotic hand, the open-loop manipulation robustness increases and observe the emergence of human-like behaviours. To achieve this, we introduce the ADAPT Hand equipped with tunable compliance throughout the skin, fingers, and the wrist. Through extensive automated pick-and-place tests, we show the grasping robustness closely mirrors an estimated geometric theoretical limit, while `stress-testing' the robot hand to perform 800+ grasps. Finally, 24 items with largely varying geometries are grasped in a constrained environment with a success rate of 93%. We demonstrate the hand-object self-organization behavior underlines this extreme robustness, where the hand automatically exhibits different grasp types depending on object geometries. Furthermore, the robot grasp type mimics a natural human grasp with a direct similarity of 68%.

Can Large Language Models design a Robot?

Large Language Models can lead researchers in the design of robots.

Control and Morphology Optimization of Passive Asymmetric Structures for Robotic Swimming

Aquatic creatures exhibit remarkable adaptations of their body to efficiently interact with the surrounding fluid. The tight coupling between their morphology, motion, and the environment are highly complex but serves as a valuable example when creating biomimetic structures in soft robotic swimmers. We focus on the use of asymmetry in structures to aid thrust generation and maneuverability. Designs of structures with asymmetric profiles are explored so that we can use morphology to `shape' the thrust generation. We propose combining simple simulation with automatic data-driven methods to explore their interactions with the fluid. The asymmetric structure with its co-optimized morphology and controller is able to produce 2.5 times the useful thrust compared to a baseline symmetric structure. Furthermore these asymmetric feather-like arms are validated on a robotic system capable of forward swimming motion while the same robot fitted with a plain feather is not able to move forward.