Toward smart composites: small-scale, untethered prediction and control for soft sensor/actuator systems

We present a suite of algorithms and tools for model-predictive control of sensor/actuator systems with embedded microcontroller units (MCU). These MCUs can be colocated with sensors and actuators, thereby enabling a new class of smart composites capable of autonomous behavior that does not require an external computer. In this approach, kinematics are learned using a neural network model from offline data and compiled into MCU code using nn4mc, an open-source tool. Online Newton-Raphson optimization solves for the control input. Shallow neural network models applied to 1D sensor signals allow for reduced model sizes and increased control loop frequencies. We validate this approach on a simulated mass-spring-damper system and two experimental setups with different sensing, actuation, and computational hardware: a tendon-based platform with embedded optical lace sensors and a HASEL-based platform with magnetic sensors. Experimental results indicate effective high-bandwidth tracking of reference paths (120 Hz and higher) with a small memory footprint (less than or equal to 6.4% of available flash). The measured path following error does not exceed 2 mm in the tendon-based platform, and the predicted path following error does not exceed 1 mm in the HASEL-based platform. This controller code's mean power consumption in an ARM Cortex-M4 computer is 45.4 mW. This control approach is also compatible with Tensorflow Lite models and equivalent compilers. Embedded intelligence in composite materials enables a new class of composites that infuse intelligence into structures and systems, making them capable of responding to environmental stimuli using their proprioception.

High-bandwidth nonlinear control for soft actuators with recursive network models

We present a high-bandwidth, lightweight, and nonlinear output tracking technique for soft actuators that combines parsimonious recursive layers for forward output predictions and online optimization using Newton-Raphson. This technique allows for reduced model sizes and increased control loop frequencies when compared with conventional RNN models. Experimental results of this controller prototype on a single soft actuator with soft positional sensors indicate effective tracking of referenced spatial trajectories and rejection of mechanical and electromagnetic disturbances. These are evidenced by root mean squared path tracking errors (RMSE) of 1.8mm using a fully connected (FC) substructure, 1.62mm using a gated recurrent unit (GRU) and 2.11mm using a long short term memory (LSTM) unit, all averaged over three tasks. Among these models, the highest flash memory requirement is 2.22kB enabling co-location of controller and actuator.

A Deformable Interface for Human Touch Recognition using Stretchable Carbon Nanotube Dielectric Elastomer Sensors and Deep Neural Networks

User interfaces provide an interactive window between physical and virtual environments. A new concept in the field of human-computer interaction is a soft user interface; a compliant surface that facilitates touch interaction through deformation. Despite the potential of these interfaces, they currently lack a signal processing framework that can efficiently extract information from their deformation. Here we present OrbTouch, a device that uses statistical learning algorithms, based on convolutional neural networks, to map deformations from human touch to categorical labels (i.e., gestures) and touch location using stretchable capacitor signals as inputs. We demonstrate this approach by using the device to control the popular game Tetris. OrbTouch provides a modular, robust framework to interpret deformation in soft media, laying a foundation for new modes of human computer interaction through shape changing solids.