This article proposes a deep neural network, namely CrackPropNet, to measure crack propagation on asphalt concrete (AC) specimens. It offers an accurate, flexible, efficient, and low-cost solution for crack propagation measurement using images collected during cracking tests. CrackPropNet significantly differs from traditional deep learning networks, as it involves learning to locate displacement field discontinuities by matching features at various locations in the reference and deformed images. An image library representing the diversified cracking behavior of AC was developed for supervised training. CrackPropNet achieved an optimal dataset scale F-1 of 0.755 and optimal image scale F-1 of 0.781 on the testing dataset at a running speed of 26 frame-per-second. Experiments demonstrated that low to medium-level Gaussian noises had a limited impact on the measurement accuracy of CrackPropNet. Moreover, the model showed promising generalization on fundamentally different images. As a crack measurement technique, the CrackPropNet can detect complex crack patterns accurately and efficiently in AC cracking tests. It can be applied to characterize the cracking phenomenon, evaluate AC cracking potential, validate test protocols, and verify theoretical models.