S2C2A: A Flexible Task Space Planning and Control Strategy for Modular Soft Robot Arms

Modular soft robot arms (MSRAs) are composed of multiple independent modules connected in a sequence. Due to their modular structure and high degrees of freedom (DOFs), these modules can simultaneously bend at different angles in various directions, enabling complex deformation. This capability allows MSRAs to perform more intricate tasks than single module robots. However, the modular structure also induces challenges in accurate planning, modeling, and control. Nonlinearity, hysteresis, and gravity complicate the physical model, while the modular structure and increased DOFs further lead to accumulative errors along the sequence. To address these challenges, we propose a flexible task space planning and control strategy for MSRAs, named S2C2A (State to Configuration to Action). Our approach formulates an optimization problem, S2C (State to Configuration planning), which integrates various loss functions and a forward MSRA model to generate configuration trajectories based on target MSRA states. Given the model complexity, we leverage a biLSTM network as the forward model. Subsequently, a configuration controller C2A (Configuration to Action control) is implemented to follow the planned configuration trajectories, leveraging only inaccurate internal sensing feedback. Both a biLSTM network and a physical model are utilized for configuration control. We validated our strategy using a cable-driven MSRA, demonstrating its ability to perform diverse offline tasks such as position control, orientation control, and obstacle avoidance. Furthermore, our strategy endows MSRA with online interaction capability with targets and obstacles. Future work will focus on addressing MSRA challenges, such as developing more accurate physical models and reducing configuration estimation errors along the module sequence.

Wireless teleoperation of HSURF artificial fish in complex paths

In this paper we show the application of the new robotic multi-platform system HSURF to a specific use case of teleoperation, aimed at monitoring and inspection. The HSURF system, consists of 3 different kinds of platforms: floater, sinker and robotic fishes. The collaborative control of the 3 platforms allows a remotely based operator to control the fish in order to visit and inspect several targets underwater following a complex trajectory. A shared autonomy solution shows to be the most suitable, in order to minimize the effect of limited bandwidth and relevant delay intrinsic to acoustic communications. The control architecture is described and preliminary results of the acoustically teleoperated visits of multiple targets in a testing pool are provided.

Data-driven Explainable Controller for Soft Robots based on Recurrent Neural Networks

The nonlinearity and hysteresis of soft robot motions have posed challenges in accurate soft robot control. Neural networks, especially recurrent neural networks (RNNs), have been widely leveraged for this issue due to their nonlinear activation functions and recurrent structures. Although they have shown satisfying accuracy in most tasks, these black-box approaches are not explainable, and hence, they are unsuitable for areas with high safety requirements, like robot-assisted surgery. Based on the RNN controllers, we propose a data-driven explainable controller (DDEC) whose parameters can be updated online. We discuss the Jacobian controller and kinematics controller in theory and demonstrate that they are only special cases of DDEC. Moreover, we utilize RNN, the Jacobian controller, the kinematics controller, and DDECs for trajectory following tasks. Experimental results have shown that our approach outperforms the other controllers considering trajectory following errors while being explainable. We also conduct a study to explore and explain the functions of each DDEC component. This is the first interpretable soft robot controller that overcomes the shortcomings of both NN controllers and interpretable controllers. Future work may involve proposing different DDECs based on different RNN controllers and exploiting them for high-safety-required applications.

Animal Behavior Analysis Methods Using Deep Learning: A Survey

Animal behavior serves as a reliable indicator of the adaptation of organisms to their environment and their overall well-being. Through rigorous observation of animal actions and interactions, researchers and observers can glean valuable insights into diverse facets of their lives, encompassing health, social dynamics, ecological relationships, and neuroethological dimensions. Although state-of-the-art deep learning models have demonstrated remarkable accuracy in classifying various forms of animal data, their adoption in animal behavior studies remains limited. This survey article endeavors to comprehensively explore deep learning architectures and strategies applied to the identification of animal behavior, spanning auditory, visual, and audiovisual methodologies. Furthermore, the manuscript scrutinizes extant animal behavior datasets, offering a detailed examination of the principal challenges confronting this research domain. The article culminates in a comprehensive discussion of key research directions within deep learning that hold potential for advancing the field of animal behavior studies.

Soft Contact Simulation and Manipulation Learning of Deformable Objects with Vision-based Tactile Sensor

Deformable object manipulation is a classical and challenging research area in robotics. Compared with rigid object manipulation, this problem is more complex due to the deformation properties including elastic, plastic, and elastoplastic deformation. In this paper, we describe a new deformable object manipulation method including soft contact simulation, manipulation learning, and sim-to-real transfer. We propose a novel approach utilizing Vision-Based Tactile Sensors (VBTSs) as the end-effector in simulation to produce observations like relative position, squeezed area, and object contour, which are transferable to real robots. For a more realistic contact simulation, a new simulation environment including elastic, plastic, and elastoplastic deformations is created. We utilize RL strategies to train agents in the simulation, and expert demonstrations are applied for challenging tasks. Finally, we build a real experimental platform to complete the sim-to-real transfer and achieve a 90% success rate on difficult tasks such as cylinder and sphere. To test the robustness of our method, we use plasticine of different hardness and sizes to repeat the tasks including cylinder and sphere. The experimental results show superior performances of deformable object manipulation with the proposed method.

A Novel and Accurate BiLSTM Configuration Controller for Modular Soft Robots with Module Number Adaptability

Modular soft robots have shown higher potential in sophisticated tasks than single-module robots. However, the modular structure incurs the complexity of accurate control and necessitates a control strategy specifically for modular robots. In this paper, we introduce a data collection strategy and a novel and accurate bidirectional LSTM configuration controller for modular soft robots with module number adaptability. Such a controller can control module configurations in robots with different module numbers. Simulation cable-driven robots and real pneumatic robots have been included in experiments to validate the proposed approaches, and we have proven that our controller can be leveraged even with the increase or decrease of module number. This is the first paper that gets inspiration from the physical structure of modular robots and utilizes bidirectional LSTM for module number adaptability. Future work may include a planning method that bridges the task and configuration spaces and the integration of an online controller.

A Hybrid Adaptive Controller for Soft Robot Interchangeability

Soft robots have been leveraged in considerable areas like surgery, rehabilitation, and bionics due to their softness, flexibility, and safety. However, it is challenging to produce two same soft robots even with the same mold and manufacturing process owing to the complexity of soft materials. Meanwhile, widespread usage of a system requires the ability to fabricate replaceable components, which is interchangeability. Due to the necessity of this property, a hybrid adaptive controller is introduced to achieve interchangeability from the perspective of control approaches. This method utilizes an offline trained recurrent neural network controller to cope with the nonlinear and delayed response from soft robots. Furthermore, an online optimizing kinematics controller is applied to decrease the error caused by the above neural network controller. Soft pneumatic robots with different deformation properties but the same mold have been included for validation experiments. In the experiments, the systems with different actuation configurations and the different robots follow the desired trajectory with errors of 0.040 and 0.030 compared with the working space length, respectively. Such an adaptive controller also shows good performance on different control frequencies and desired velocities. This controller endows soft robots with the potential for wide application, and future work may include different offline and online controllers. A weight parameter adjusting strategy may also be proposed in the future.

Data Models Applied to Soft Robot Modeling and Control: A Review

Soft robots show compliance and have infinite degrees of freedom. Thanks to these properties, such robots are leveraged for surgery, rehabilitation, biomimetics, unstructured environment exploring, and industrial gripper. In this case, they attract scholars from a variety of areas. However, nonlinearity and hysteresis effects also bring a burden to robot modeling. Moreover, following their flexibility and adaptation, soft robot control is more challenging than rigid robot control. In order to model and control soft robots, a large number of data models are utilized in pairs or separately. This review classifies these applied data models into five kinds, which are the Jacobian model, analytical model, statistical model, neural network, and reinforcement learning, and compares the modeling and controller features, e.g., model dynamics, data requirement, and target task, within and among these categories. A discussion about the development of the existing modeling and control approaches is presented, and we forecast that the combination of offline-trained and online-learning controllers will be the widespread implementation in the future.

Design of magnetic coupling-based anti-biofouling mechanism for underwater optical sensors

Water monitoring is crucial for environmental monitoring, transportation, energy and telecommunication. One of the main problems in aquatic environmental monitoring is biofouling. The simplest method among the current antifouling strategies is the use of wiper technologies like brushes and wipers which apply mechanical pressure. In designing built-in strategies however, manufacturers usually build the sensor around the biofouling system. The current state-of-the-art is a fully integrated central wiper in the sensor that enables cleaning of all probes mounted on the sonde. Improvements in antifouling strategies lag rapid advancements in sensor technologies such as in miniaturization, specialization, and costs. Hence, improving built-in designs by decreasing size and complexity will decrease maintenance and overall costs. This design is targeted for the EU project Robocoenosis since bio-hybrid systems in this project incorporate living organisms. This technology targets selective proliferation of the organisms which only prevents biofilms on components where they are unwanted. Beyond this, the use of autonomous activation based on image processing may likely be advantageous for minimizing the need for human inspection and maintenance. In addition to Robocoenosis, we also aim at incorporating this design in another project entitled Heterogeneous Swarm of Underwater Autonomous Vehicles where a swarm of heterogeneous underwater robotic fish is being developed.