Deep Reinforcement Learning based Autonomous Decision-Making for Cooperative UAVs: A Search and Rescue Real World Application

This paper proposes a holistic framework for autonomous guidance, navigation, and task distribution among multi-drone systems operating in Global Navigation Satellite System (GNSS)-denied indoor settings. We advocate for a Deep Reinforcement Learning (DRL)-based guidance mechanism, utilising the Twin Delayed Deep Deterministic Policy Gradient algorithm. To improve the efficiency of the training process, we incorporate an Artificial Potential Field (APF)-based reward structure, enabling the agent to refine its movements, thereby promoting smoother paths and enhanced obstacle avoidance in indoor contexts. Furthermore, we tackle the issue of task distribution among cooperative UAVs through a DRL-trained Graph Convolutional Network (GCN). This GCN represents the interactions between drones and tasks, facilitating dynamic and real-time task allocation that reflects the current environmental conditions and the capabilities of the drones. Such an approach fosters effective coordination and collaboration among multiple drones during search and rescue operations or other exploratory endeavours. Lastly, to ensure precise odometry in environments lacking GNSS, we employ Light Detection And Ranging Simultaneous Localisation and Mapping complemented by a depth camera to mitigate the hallway problem. This integration offers robust localisation and mapping functionalities, thereby enhancing the systems dependability in indoor navigation. The proposed multi-drone framework not only elevates individual navigation capabilities but also optimises coordinated task allocation in complex, obstacle-laden environments. Experimental evaluations conducted in a setup tailored to meet the requirements of the NATO Sapience Autonomous Cooperative Drone Competition demonstrate the efficacy of the proposed system, yielding outstanding results and culminating in a first-place finish in the 2024 Sapience competition.

Using Convolutional Neural Networks for Relative Pose Estimation of a Non-Cooperative Spacecraft with Thermal Infrared Imagery

Recent interest in on-orbit servicing and Active Debris Removal (ADR) missions have driven the need for technologies to enable non-cooperative rendezvous manoeuvres. Such manoeuvres put heavy burden on the perception capabilities of a chaser spacecraft. This paper demonstrates Convolutional Neural Networks (CNNs) capable of providing an initial coarse pose estimation of a target from a passive thermal infrared camera feed. Thermal cameras offer a promising alternative to visible cameras, which struggle in low light conditions and are susceptible to overexposure. Often, thermal information on the target is not available a priori; this paper therefore proposes using visible images to train networks. The robustness of the models is demonstrated on two different targets, first on synthetic data, and then in a laboratory environment for a realistic scenario that might be faced during an ADR mission. Given that there is much concern over the use of CNN in critical applications due to their black box nature, we use innovative techniques to explain what is important to our network and fault conditions.