Hitting the Gym: Reinforcement Learning Control of Exercise-Strengthened Biohybrid Robots in Simulation

Animals can accomplish many incredible behavioral feats across a wide range of operational environments and scales that current robots struggle to match. One explanation for this performance gap is the extraordinary properties of the biological materials that comprise animals, such as muscle tissue. Using living muscle tissue as an actuator can endow robotic systems with highly desirable properties such as self-healing, compliance, and biocompatibility. Unlike traditional soft robotic actuators, living muscle biohybrid actuators exhibit unique adaptability, growing stronger with use. The dependency of a muscle's force output on its use history endows muscular organisms the ability to dynamically adapt to their environment, getting better at tasks over time. While muscle adaptability is a benefit to muscular organisms, it currently presents a challenge for biohybrid researchers: how does one design and control a robot whose actuators' force output changes over time? Here, we incorporate muscle adaptability into a many-muscle biohybrid robot design and modeling tool, leveraging reinforcement learning as both a co-design partner and system controller. As a controller, our learning agents coordinated the independent contraction of 42 muscles distributed on a lattice worm structure to successfully steer it towards eight distinct targets while incorporating muscle adaptability. As a co-design tool, our agents enable users to identify which muscles are important to accomplishing a given task. Our results show that adaptive agents outperform non-adaptive agents in terms of maximum rewards and training time. Together, these contributions can both enable the elucidation of muscle actuator adaptation and inform the design and modeling of adaptive, performant, many-muscle robots.

Design and Characterization of Viscoelastic McKibben Actuators with Tunable Force-Velocity Curves

The McKibben pneumatic artificial muscle is a commonly studied soft robotic actuator, and its quasistatic force-length properties have been well characterized and modeled. However, its damping and force-velocity properties are less well studied. Understanding these properties will allow for more robust dynamic modeling of soft robotic systems. The force-velocity response of these actuators is of particular interest because these actuators are often used as hardware models of skeletal muscles for bioinspired robots, and this force-velocity relationship is fundamental to muscle physiology. In this work, we investigated the force-velocity response of McKibben actuators and the ability to tune this response through the use of viscoelastic polymer sheaths. These viscoelastic McKibben actuators (VMAs) were characterized using iso-velocity experiments inspired by skeletal muscle physiology tests. A simplified 1D model of the actuators was developed to connect the shape of the force-velocity curve to the material parameters of the actuator and sheaths. Using these viscoelastic materials, we were able to modulate the shape and magnitude of the actuators' force-velocity curves, and using the developed model, these changes were connected back to the material properties of the sheaths.

SLUGBOT, an Aplysia-inspired Robotic Grasper for Studying Control

Living systems can use a single periphery to perform a variety of tasks and adapt to a dynamic environment. This multifunctionality is achieved through the use of neural circuitry that adaptively controls the reconfigurable musculature. Current robotic systems struggle to flexibly adapt to unstructured environments. Through mimicry of the neuromechanical coupling seen in living organisms, robotic systems could potentially achieve greater autonomy. The tractable neuromechanics of the sea slug $\textit{Aplysia californica's}$ feeding apparatus, or buccal mass, make it an ideal candidate for applying neuromechanical principles to the control of a soft robot. In this work, a robotic grasper was designed to mimic specific morphology of the $\textit{Aplysia}$ feeding apparatus. These include the use of soft actuators akin to biological muscle, a deformable grasping surface, and a similar muscular architecture. A previously developed Boolean neural controller was then adapted for the control of this soft robotic system. The robot was capable of qualitatively replicating swallowing behavior by cyclically ingesting a plastic tube. The robot's normalized translational and rotational kinematics of the odontophore followed profiles observed $\textit{in vivo}$ despite morphological differences. This brings $\textit{Aplysia}$-inspired control $\textit{in roboto}$ one step closer to multifunctional neural control schema $\textit{in vivo}$ and $\textit{in silico}$. Future additions may improve SLUGBOT's viability as a neuromechanical research platform.

Design of a Biomimetic Tactile Sensor for Material Classification

Tactile sensing typically involves active exploration of unknown surfaces and objects, making it especially effective at processing the characteristics of materials and textures. A key property extracted by human tactile perception is surface roughness, which relies on measuring vibratory signals using the multi-layered fingertip structure. Existing robotic systems lack tactile sensors that are able to provide high dynamic sensing ranges, perceive material properties, and maintain a low hardware cost. In this work, we introduce the reference design and fabrication procedure of a miniature and low-cost tactile sensor consisting of a biomimetic cutaneous structure, including the artificial fingerprint, dermis, epidermis, and an embedded magnet-sensor structure which serves as a mechanoreceptor for converting mechanical information to digital signals. The presented sensor is capable of detecting high-resolution magnetic field data through the Hall effect and creating high-dimensional time-frequency domain features for material texture classification. Additionally, we investigate the effects of different superficial sensor fingerprint patterns for classifying materials through both simulation and physical experimentation. After extracting time series and frequency domain features, we assess a k-nearest neighbors classifier for distinguishing between different materials. The results from our experiments show that our biomimetic tactile sensors with fingerprint ridges can classify materials with more than 8% higher accuracy and lower variability than ridge-less sensors. These results, along with the low cost and customizability of our sensor, demonstrate high potential for lowering the barrier to entry for a wide array of robotic applications, including model-less tactile sensing for texture classification, material inspection, and object recognition.

Control for Multifunctionality: Bioinspired Control Based on Feeding in Aplysia californica

Animals exhibit remarkable feats of behavioral flexibility and multifunctional control that remain challenging for robotic systems. The neural and morphological basis of multifunctionality in animals can provide a source of bio-inspiration for robotic controllers. However, many existing approaches to modeling biological neural networks rely on computationally expensive models and tend to focus solely on the nervous system, often neglecting the biomechanics of the periphery. As a consequence, while these models are excellent tools for neuroscience, they fail to predict functional behavior in real time, which is a critical capability for robotic control. To meet the need for real-time multifunctional control, we have developed a hybrid Boolean model framework capable of modeling neural bursting activity and simple biomechanics at speeds faster than real time. Using this approach, we present a multifunctional model of Aplysia californica feeding that qualitatively reproduces three key feeding behaviors (biting, swallowing, and rejection), demonstrates behavioral switching in response to external sensory cues, and incorporates both known neural connectivity and a simple bioinspired mechanical model of the feeding apparatus. We demonstrate that the model can be used for formulating testable hypotheses and discuss the implications of this approach for robotic control and neuroscience.