Nonlinear Traffic Prediction as a Matrix Completion Problem with Ensemble Learning

We focus on short-term traffic forecasting for traffic operations management. Specifically, we focus on forecasting traffic network sensor states in high-resolution (second-by-second). Most work on traffic forecasting has focused on predicting aggregated traffic variables, typically over intervals that are no shorter than 5 minutes. The data resolution required for traffic operations is challenging since high-resolution data exhibit heavier oscillations and precise patterns are harder to capture. We propose a (big) data-driven methodology for this purpose. Our contributions can be summarized as offering three major insights: first, we show how the forecasting problem can be modeled as a matrix completion problem. Second, we employ a block-coordinate descent algorithm and demonstrate that the algorithm converges in sub-linear time to a block coordinate-wise optimizer. This allows us to capitalize on the "bigness" of high-resolution data in a computationally feasible way. Third, we develop an adaptive boosting (or ensemble learning) approach to reduce the training error to within any arbitrary error threshold. The latter utilizes past days so that the boosting can be interpreted as capturing periodic patterns in the data. The performance of the proposed method is analyzed theoretically and tested empirically using a real-world high-resolution traffic dataset from Abu Dhabi, UAE. Our experimental results show that the proposed method outperforms other state-of-the-art algorithms.