End-to-End Deep Learning of Lane Detection and Path Prediction for Real-Time Autonomous Driving

We propose an end-to-end three-task convolutional neural network (3TCNN) having two regression branches of bounding boxes and Hu moments and one classification branch of object masks for lane detection and road recognition. The Hu-moment regressor performs lane localization and road guidance using local and global Hu moments of segmented lane objects, respectively. Based on 3TCNN, we then propose lateral offset and path prediction (PP) algorithms to form an integrated model (3TCNN-PP) that can predict driving path with dynamic estimation of lane centerline and path curvature for real-time autonomous driving. We also develop a CNN-PP simulator that can be used to train a CNN by real or artificial traffic images, test it by artificial images, quantify its dynamic errors, and visualize its qualitative performance. Simulation results show that 3TCNN-PP is comparable to related CNNs and better than a previous CNN-PP, respectively. The code, annotated data, and simulation videos of this work can be found on our website for further research on NN-PP algorithms of autonomous driving.