Deep Reinforcement Learning-based Quadcopter Controller: A Practical Approach and Experiments

Quadcopters have been studied for decades thanks to their maneuverability and capability of operating in a variety of circumstances. However, quadcopters suffer from dynamical nonlinearity, actuator saturation, as well as sensor noise that make it challenging and time consuming to obtain accurate dynamic models and achieve satisfactory control performance. Fortunately, deep reinforcement learning came and has shown significant potential in system modelling and control of autonomous multirotor aerial vehicles, with recent advancements in deployment, performance enhancement, and generalization. In this paper, an end-to-end deep reinforcement learning-based controller for quadcopters is proposed that is secure for real-world implementation, data-efficient, and free of human gain adjustments. First, a novel actor-critic-based architecture is designed to map the robot states directly to the motor outputs. Then, a quadcopter dynamics-based simulator was devised to facilitate the training of the controller policy. Finally, the trained policy is deployed on a real Crazyflie nano quadrotor platform, without any additional fine-tuning process. Experimental results show that the quadcopter exhibits satisfactory performance as it tracks a given complicated trajectory, which demonstrates the effectiveness and feasibility of the proposed method and signifies its capability in filling the simulation-to-reality gap.

Real-time Human Detection in Fire Scenarios using Infrared and Thermal Imaging Fusion

Fire is considered one of the most serious threats to human lives which results in a high probability of fatalities. Those severe consequences stem from the heavy smoke emitted from a fire that mostly restricts the visibility of escaping victims and rescuing squad. In such hazardous circumstances, the use of a vision-based human detection system is able to improve the ability to save more lives. To this end, a thermal and infrared imaging fusion strategy based on multiple cameras for human detection in low-visibility scenarios caused by smoke is proposed in this paper. By processing with multiple cameras, vital information can be gathered to generate more useful features for human detection. Firstly, the cameras are calibrated using a Light Heating Chessboard. Afterward, the features extracted from the input images are merged prior to being passed through a lightweight deep neural network to perform the human detection task. The experiments conducted on an NVIDIA Jetson Nano computer demonstrated that the proposed method can process with reasonable speed and can achieve favorable performance with a mAP@0.5 of 95%.

A Vision-based Autonomous Perching Approach for Nano Aerial Vehicles

Over the past decades, quadcopters have been investigated, due to their mobility and flexibility to operate in a wide range of environments. They have been used in various areas, including surveillance and monitoring. During a mission, drones do not have to remain active once they have reached a target location. To conserve energy and maintain a static position, it is possible to perch and stop the motors in such situations. The problem of achieving a reliable and highly accurate perching method remains a challenge and promising. In this paper, a vision-based autonomous perching approach for nano quadcopters onto a predefined perching target on horizontal surfaces is proposed. First, a perching target with a small marker inside a larger one is designed to improve detection capability at a variety of ranges. Second, a monocular camera is used to calculate the relative poses of the flying vehicle from the markers detected. Then, a Kalman filter is applied to determine the pose more reliably, especially when measurement data is missing. Next, we introduce an algorithm for merging the pose data from multiple markers. Finally, the poses are sent to the perching planner to conduct the real flight test to align the drone with the target's center and steer it there. Based on the experimental results, the approach proved to be effective and feasible. The drone can successfully perch on the center of markers within two centimeters of precision.

Vision-based Target Pose Estimation with Multiple Markers for the Perching of UAVs

Autonomous Nano Aerial Vehicles have been increasingly popular in surveillance and monitoring operations due to their efficiency and maneuverability. Once a target location has been reached, drones do not have to remain active during the mission. It is possible for the vehicle to perch and stop its motors in such situations to conserve energy, as well as maintain a static position in unfavorable flying conditions. In the perching target estimation phase, the steady and accuracy of a visual camera with markers is a significant challenge. It is rapidly detectable from afar when using a large marker, but when the drone approaches, it quickly disappears as out of camera view. In this paper, a vision-based target poses estimation method using multiple markers is proposed to deal with the above-mentioned problems. First, a perching target with a small marker inside a larger one is designed to improve detection capability at wide and close ranges. Second, the relative poses of the flying vehicle are calculated from detected markers using a monocular camera. Next, a Kalman filter is applied to provide a more stable and reliable pose estimation, especially when the measurement data is missing due to unexpected reasons. Finally, we introduced an algorithm for merging the poses data from multi markers. The poses are then sent to the position controller to align the drone and the marker's center and steer it to perch on the target. The experimental results demonstrated the effectiveness and feasibility of the adopted approach. The drone can perch successfully onto the center of the markers with the attached 25mm-diameter rounded magnet.

POCS-based Clustering Algorithm

A novel clustering technique based on the projection onto convex set (POCS) method, called POCS-based clustering algorithm, is proposed in this paper. The proposed POCS-based clustering algorithm exploits a parallel projection method of POCS to find appropriate cluster prototypes in the feature space. The algorithm considers each data point as a convex set and projects the cluster prototypes parallelly to the member data points. The projections are convexly combined to minimize the objective function for data clustering purpose. The performance of the proposed POCS-based clustering algorithm is verified through experiments on various synthetic datasets. The experimental results show that the proposed POCS-based clustering algorithm is competitive and efficient in terms of clustering error and execution speed when compared with other conventional clustering methods including Fuzzy C-Means (FCM) and K-means clustering algorithms.

Robustness Enhancement of Object Detection in Advanced Driver Assistance Systems (ADAS)

A unified system integrating a compact object detector and a surrounding environmental condition classifier for enhancing the robustness of object detection scheme in advanced driver assistance systems (ADAS) is proposed in this paper. ADAS are invented to improve traffic safety and effectiveness in autonomous driving systems where object detection plays an extremely important role. However, modern object detectors integrated in ADAS are still unstable due to high latency and the variation of the environmental contexts in the deployment phase. Our system is proposed to address the aforementioned problems. The proposed system includes two main components: (1) a compact one-stage object detector which is expected to be able to perform at a comparable accuracy compared to state-of-the-art object detectors, and (2) an environmental condition detector that helps to send a warning signal to the cloud in case the self-driving car needs human actions due to the significance of the situation. The empirical results prove the reliability and the scalability of the proposed system to realistic scenarios.