Actuator Trajectory Planning for UAVs with Overhead Manipulator using Reinforcement Learning

In this paper, we investigate the operation of an aerial manipulator system, namely an Unmanned Aerial Vehicle (UAV) equipped with a controllable arm with two degrees of freedom to carry out actuation tasks on the fly. Our solution is based on employing a Q-learning method to control the trajectory of the tip of the arm, also called end-effector. More specifically, we develop a motion planning model based on Time To Collision (TTC), which enables a quadrotor UAV to navigate around obstacles while ensuring the manipulator's reachability. Additionally, we utilize a model-based Q-learning model to independently track and control the desired trajectory of the manipulator's end-effector, given an arbitrary baseline trajectory for the UAV platform. Such a combination enables a variety of actuation tasks such as high-altitude welding, structural monitoring and repair, battery replacement, gutter cleaning, skyscrapper cleaning, and power line maintenance in hard-to-reach and risky environments while retaining compatibility with flight control firmware. Our RL-based control mechanism results in a robust control strategy that can handle uncertainties in the motion of the UAV, offering promising performance. Specifically, our method achieves 92% accuracy in terms of average displacement error (i.e. the mean distance between the target and obtained trajectory points) using Q-learning with 15,000 episodes

Feasibility Assessment of a Cost-Effective Two-Wheel Kian-I Mobile Robot for Autonomous Navigation

A two-wheeled mobile robot, namely Kian-I, is designed and prototyped in this research. The Kian-I is comparable with Khepera-IV in terms of dimensional specifications, mounted sensors, and performance capabilities and can be used for educational purposes and cost-effective experimental tests. A motion control architecture is designed for Kian-I in this study to facilitate accurate navigation for the robot in an immersive environment. The implemented control structure consists of two main components of the path recommender system and trajectory tracking controller. Given partial knowledge about the operation field, the path recommender system adopts B-spline curves and Particle Swarm Optimization (PSO) algorithm to determine a collision-free path curve with translational velocity constraint. The provided optimal reference path feeds into the trajectory tracking controller enabling Kian-I to navigate autonomously in the operating field. The trajectory tracking module eliminate the error between the desired path and the followed trajectory through controlling the wheels' velocity. To assess the feasibility of the proposed control architecture, the performance of Kian-I robot in autonomous navigation from any arbitrary initial pose to a target of interest is evaluated through numerous simulation and experimental studies. The experimental results demonstrate the functional capacities and performance of the prototyped robot to be used as a benchmark for investigation and verification of various mobile robot algorithms in the laboratory environment.