There is an increased interest in the use of Unmanned Aerial Vehicles (UAVs) for agriculture, military, disaster management and aerial photography around the world. UAVs are scalable, flexible and are useful in various environments where direct human intervention is difficult. In general, the use of UAVs with cameras mounted to them has increased in number due to their wide range of applications in real life scenarios. With the advent of deep learning models in computer vision many models have shown great success in visual tasks. But most of evaluation models are done on high end CPUs and GPUs. One of major challenges in using UAVs for Visual Assistance tasks in real time is managing the memory usage and power consumption of the these tasks which are computationally intensive and are difficult to be performed on low end processor board of the UAV. This projects describes a novel method to optimize the general image processing tasks like object tracking and object detection for UAV hardware in real time scenarios without affecting the flight time and not tampering the latency and accuracy of these models.