This paper develops a channel estimation technique for millimeter wave (mmWave) communication systems. Our method exploits the sparse structure in mmWave channels for low training overhead and accounts for the phase errors in the channel measurements due to phase noise at the oscillator. Specifically, in IEEE 802.11ad/ay-based mmWave systems, the phase errors within a beam refinement protocol packet are almost the same, while the errors across different packets are substantially different. Consequently, standard sparsity-aware algorithms, which ignore phase errors, fail when channel measurements are acquired over multiple beam refinement protocol packets. We present a novel algorithm called partially coherent matching pursuit for sparse channel estimation under practical phase noise perturbations. Our method iteratively detects the support of sparse signal and employs alternating minimization to jointly estimate the signal and the phase errors. We numerically show that our algorithm can reconstruct the channel accurately at a lower complexity than the benchmarks.