Toward smart composites: small-scale, untethered prediction and control for soft sensor/actuator systems

We present a suite of algorithms and tools for model-predictive control of sensor/actuator systems with embedded microcontroller units (MCU). These MCUs can be colocated with sensors and actuators, thereby enabling a new class of smart composites capable of autonomous behavior that does not require an external computer. In this approach, kinematics are learned using a neural network model from offline data and compiled into MCU code using nn4mc, an open-source tool. Online Newton-Raphson optimization solves for the control input. Shallow neural network models applied to 1D sensor signals allow for reduced model sizes and increased control loop frequencies. We validate this approach on a simulated mass-spring-damper system and two experimental setups with different sensing, actuation, and computational hardware: a tendon-based platform with embedded optical lace sensors and a HASEL-based platform with magnetic sensors. Experimental results indicate effective high-bandwidth tracking of reference paths (120 Hz and higher) with a small memory footprint (less than or equal to 6.4% of available flash). The measured path following error does not exceed 2 mm in the tendon-based platform, and the predicted path following error does not exceed 1 mm in the HASEL-based platform. This controller code's mean power consumption in an ARM Cortex-M4 computer is 45.4 mW. This control approach is also compatible with Tensorflow Lite models and equivalent compilers. Embedded intelligence in composite materials enables a new class of composites that infuse intelligence into structures and systems, making them capable of responding to environmental stimuli using their proprioception.

High-bandwidth nonlinear control for soft actuators with recursive network models

We present a high-bandwidth, lightweight, and nonlinear output tracking technique for soft actuators that combines parsimonious recursive layers for forward output predictions and online optimization using Newton-Raphson. This technique allows for reduced model sizes and increased control loop frequencies when compared with conventional RNN models. Experimental results of this controller prototype on a single soft actuator with soft positional sensors indicate effective tracking of referenced spatial trajectories and rejection of mechanical and electromagnetic disturbances. These are evidenced by root mean squared path tracking errors (RMSE) of 1.8mm using a fully connected (FC) substructure, 1.62mm using a gated recurrent unit (GRU) and 2.11mm using a long short term memory (LSTM) unit, all averaged over three tasks. Among these models, the highest flash memory requirement is 2.22kB enabling co-location of controller and actuator.

Embedded Neural Networks for Robot Autonomy

We present a library to automatically embed signal processing and neural network predictions into the material robots are made of. Deep and shallow neural network models are first trained offline using state-of-the-art machine learning tools and then transferred onto general purpose microcontrollers that are co-located with a robot's sensors and actuators. We validate this approach using multiple examples: a smart robotic tire for terrain classification, a robotic finger sensor for load classification and a smart composite capable of regressing impact source localization. In each example, sensing and computation are embedded inside the material, creating artifacts that serve as stand-in replacement for otherwise inert conventional parts. The open source software library takes as inputs trained model files from higher level learning software, such as Tensorflow/Keras, and outputs code that is readable in a microcontroller that supports C. We compare the performance of this approach for various embedded platforms. In particular, we show that low-cost off-the-shelf microcontrollers can match the accuracy of a desktop computer, while being fast enough for real-time applications at different neural network configurations. We provide means to estimate the maximum number of parameters that the hardware will support based on the microcontroller's specifications.