Exploiting Spatial Correlation in Convolutional Neural Networks for Activation Value Prediction

Convolutional neural networks (CNNs) compute their output using weighted-sums of adjacent input elements. This method enables CNNs to achieve state-of-the-art results in a wide range of applications such as computer vision and speech recognition. However, it also comes with the cost of high computational intensity. In this paper we propose to exploit the spatial correlation inherent in CNNs, and use it for value prediction. We show that spatial correlation may be exploited to predict activation values, thus reducing the needed computations in the network. We demonstrate this method with a heuristic that predicts which activations are zero-valued according to nearby activation values, in a scheme we call cross-neuron prediction. Our prediction heuristic reduces the number of multiply-accumulate operations by an average of 40.8%, 36.2%, and 20.8%, with degradation in top-5 accuracy of 2.9%, 5.1%, and 7.6%, for AlexNet, VGG-16, and ResNet-18, respectively.