Tolerance of Reinforcement Learning Controllers against Deviations in Cyber Physical Systems

Cyber-physical systems (CPS) with reinforcement learning (RL)-based controllers are increasingly being deployed in complex physical environments such as autonomous vehicles, the Internet-of-Things(IoT), and smart cities. An important property of a CPS is tolerance; i.e., its ability to function safely under possible disturbances and uncertainties in the actual operation. In this paper, we introduce a new, expressive notion of tolerance that describes how well a controller is capable of satisfying a desired system requirement, specified using Signal Temporal Logic (STL), under possible deviations in the system. Based on this definition, we propose a novel analysis problem, called the tolerance falsification problem, which involves finding small deviations that result in a violation of the given requirement. We present a novel, two-layer simulation-based analysis framework and a novel search heuristic for finding small tolerance violations. To evaluate our approach, we construct a set of benchmark problems where system parameters can be configured to represent different types of uncertainties and disturbancesin the system. Our evaluation shows that our falsification approach and heuristic can effectively find small tolerance violations.

A Soft-Bodied Aerial Robot for Collision Resilience and Contact-Reactive Perching

Compared to their biological counterparts, aerial robots demonstrate limited capabilities when tasked to interact in unstructured environments. Very often, the limitation lies in their inability to tolerate collisions and to successfully land, or perch, on objects of unknown shape. Over the past years, efforts to address this have introduced designs that incorporate mechanical impact protection and grasping/perching structures at the cost of reduced agility and flight time due to added weight and bulkiness. In this work, we develop a fabric-based, soft-bodied aerial robot (SoBAR) composed of both contact-reactive perching and embodied impact protection structures while remaining lightweight and streamlined. The robot is capable to 1) pneumatically vary its body stiffness for collision resilience and 2) utilize a hybrid fabric-based, bistable (HFB) grasper to perform passive grasping. When compared to conventional rigid drone frames the SoBAR successfully demonstrates its ability to dissipate impact from head-on collisions and maintain flight stability without any structural damage. Furthermore, in dynamic perching scenarios the HFB grasper is capable to convert impact energy upon contact into firm grasp through rapid body shape conforming in less than 4ms. We exhaustively study and offer insights for this novel perching scheme through grasping characterization, grasp wrench analysis, and experimental grasping validations in objects with various shapes. Finally, we demonstrate the complete control pipeline for SoBAR to approach an object, dynamically perch on it, recover from it, and land.

Probabilistic Consensus on Feature Distribution for Multi-robot Systems with Markovian Exploration Dynamics

In this paper, we present a consensus-based decentralized multi-robot approach to reconstruct a discrete distribution of features, modeled as an occupancy grid map, that represent information contained in a bounded planar environment, such as visual cues used for navigation or semantic labels associated with object detection. The robots explore the environment according to a random walk modeled by a discrete-time discrete-state (DTDS) Markov chain and estimate the feature distribution from their own measurements and the estimates communicated by neighboring robots, using a distributed Chernoff fusion protocol. We prove that under this decentralized fusion protocol, each robot's feature distribution converges to the actual distribution in an almost sure sense. We verify this result in numerical simulations that show that the Hellinger distance between the estimated and actual feature distributions converges to zero over time for each robot. We also validate our strategy through Software-In-The-Loop (SITL) simulations of quadrotors that search a bounded square grid for a set of visual features distributed on a discretized circle.

Collision Recovery Control of a Foldable Quadrotor

Autonomous missions of small unmanned aerial vehicles (UAVs) are prone to collisions owing to environmental disturbances and localization errors. Consequently, a UAV that can endure collisions and perform recovery control in critical aerial missions is desirable to prevent loss of the vehicle and/or payload. We address this problem by proposing a novel foldable quadrotor system which can sustain collisions and recover safely. The quadrotor is designed with integrated mechanical compliance using a torsional spring such that the impact time is increased and the net impact force on the main body is decreased. The post-collision dynamics is analysed and a recovery controller is proposed which stabilizes the system to a hovering location without additional collisions. Flight test results on the proposed and a conventional quadrotor demonstrate that for the former, integrated spring-damper characteristics reduce the rebound velocity and lead to simple recovery control algorithms in the event of unintended collisions as compared to a rigid quadrotor of the same dimension.

Design and Control of SQUEEZE: A Spring-augmented QUadrotor for intEractions with the Environment to squeeZE-and-fly

This paper presents the design and control of a novel quadrotor with a variable geometry to physically interact with cluttered environments and fly through narrow gaps and passageways. This compliant quadrotor with passive morphing capabilities is designed using torsional springs at every arm hinge to allow for rotation driven by external forces. We derive the dynamic model of this variable geometry quadrotor (SQUEEZE), and develop an adaptive controller for trajectory tracking. The corresponding Lyapunov stability proof of attitude tracking is also presented. Further, an admittance controller is designed to account for changes in yaw due to physical interactions with the environment. Finally, the proposed design is validated in flight tests with two setups: a small gap and a passageway. The experimental results demonstrate the unique capability of the SQUEEZE in navigating through constrained narrow spaces.