Coprocessor Actor Critic: A Model-Based Reinforcement Learning Approach For Adaptive Brain Stimulation

Adaptive brain stimulation can treat neurological conditions such as Parkinson's disease and post-stroke motor deficits by influencing abnormal neural activity. Because of patient heterogeneity, each patient requires a unique stimulation policy to achieve optimal neural responses. Model-free reinforcement learning (MFRL) holds promise in learning effective policies for a variety of similar control tasks, but is limited in domains like brain stimulation by a need for numerous costly environment interactions. In this work we introduce Coprocessor Actor Critic, a novel, model-based reinforcement learning (MBRL) approach for learning neural coprocessor policies for brain stimulation. Our key insight is that coprocessor policy learning is a combination of learning how to act optimally in the world and learning how to induce optimal actions in the world through stimulation of an injured brain. We show that our approach overcomes the limitations of traditional MFRL methods in terms of sample efficiency and task success and outperforms baseline MBRL approaches in a neurologically realistic model of an injured brain.

The Effect of Robot Skill Level and Communication in Rapid, Proximate Human-Robot Collaboration

As high-speed, agile robots become more commonplace, these robots will have the potential to better aid and collaborate with humans. However, due to the increased agility and functionality of these robots, close collaboration with humans can create safety concerns that alter team dynamics and degrade task performance. In this work, we aim to enable the deployment of safe and trustworthy agile robots that operate in proximity with humans. We do so by 1) Proposing a novel human-robot doubles table tennis scenario to serve as a testbed for studying agile, proximate human-robot collaboration and 2) Conducting a user-study to understand how attributes of the robot (e.g., robot competency or capacity to communicate) impact team dynamics, perceived safety, and perceived trust, and how these latent factors affect human-robot collaboration (HRC) performance. We find that robot competency significantly increases perceived trust ($p<.001$), extending skill-to-trust assessments in prior studies to agile, proximate HRC. Furthermore, interestingly, we find that when the robot vocalizes its intention to perform a task, it results in a significant decrease in team performance ($p=.037$) and perceived safety of the system ($p=.009$).

When Your Robot Breaks: Active Learning During Plant Failure

Detecting and adapting to catastrophic failures in robotic systems requires a robot to learn its new dynamics quickly and safely to best accomplish its goals. To address this challenging problem, we propose probabilistically-safe, online learning techniques to infer the altered dynamics of a robot at the moment a failure (e.g., physical damage) occurs. We combine model predictive control and active learning within a chance-constrained optimization framework to safely and efficiently learn the new plant model of the robot. We leverage a neural network for function approximation in learning the latent dynamics of the robot under failure conditions. Our framework generalizes to various damage conditions while being computationally light-weight to advance real-time deployment. We empirically validate within a virtual environment that we can regain control of a severely damaged aircraft in seconds and require only 0.1 seconds to find safe, information-rich trajectories, outperforming state-of-the-art approaches.

An Efficient Production Process for Extracting Salivary Glands from Mosquitoes

Malaria is the one of the leading causes of morbidity and mortality in many developing countries. The development of a highly effective and readily deployable vaccine represents a major goal for world health. There has been recent progress in developing a clinically effective vaccine manufactured using Plasmodium falciparum sporozoites (PfSPZ) extracted from the salivary glands of Anopheles sp. Mosquitoes. The harvesting of PfSPZ requires dissection of the mosquito and manual removal of the salivary glands from each mosquito by trained technicians. While PfSPZ-based vaccines have shown highly promising results, the process of dissection of salivary glands is tedious and labor intensive. We propose a mechanical device that will greatly increase the rate of mosquito dissection and deskill the process to make malaria vaccines more affordable and more readily available. This device consists of several components: a sorting stage in which the mosquitoes are sorted into slots, a cutting stage in which the heads are removed, and a squeezing stage in which the salivary glands are extracted and collected. This method allows mosquitoes to be dissected twenty at a time instead of one by one as previously done and significantly reduces the dissection time per mosquito.