Deep Reinforcement Learning-Based Approach for a Single Vehicle Persistent Surveillance Problem with Fuel Constraints

This article presents a deep reinforcement learning-based approach to tackle a persistent surveillance mission requiring a single unmanned aerial vehicle initially stationed at a depot with fuel or time-of-flight constraints to repeatedly visit a set of targets with equal priority. Owing to the vehicle's fuel or time-of-flight constraints, the vehicle must be regularly refueled, or its battery must be recharged at the depot. The objective of the problem is to determine an optimal sequence of visits to the targets that minimizes the maximum time elapsed between successive visits to any target while ensuring that the vehicle never runs out of fuel or charge. We present a deep reinforcement learning algorithm to solve this problem and present the results of numerical experiments that corroborate the effectiveness of this approach in comparison with common-sense greedy heuristics.

Cooperative Planning for Fuel-constrained Aerial Vehicles and Ground-based Refueling Vehicles for Large-Scale Coverage

Low cost Unmanned Aerial Vehicles (UAVs) need multiple refuels to accomplish large area coverage. The number of refueling stations and their placement plays a vital role in determining coverage efficiency. In this paper, we propose the use of a ground-based refueling vehicle (RV) to increase the operational range of a UAV in both spatial and temporal domains. Determining optimal routes for the UAV and RV, and selecting optimized locations for refueling to aid in minimizing coverage time is a challenging problem due to different vehicle speeds, coupling between refueling location placement, and the coverage area at each location. We develop a two-stage strategy for coupled route planning for UAV and RV to perform a coverage mission. The first stage computes a minimal set of refueling sites that permit a feasible UAV route. In the second stage, multiple Mixed-Integer Linear Programming (MILP) formulations are developed to plan optimal routes for the UAV and the refueling vehicle taking into account the feasible set of refueling sites generated in stage one. The performance of different formulations is compared empirically. In addition, computationally efficient heuristics are developed to solve the routing problem. Extensive simulations are conducted to corroborate the effectiveness of proposed approaches.