Recently, channel state information (CSI) at the physical-layer has been utilized to detect spoofing attacks in wireless communications. However, due to hardware impairments and communication noise, the CSI cannot be estimated accurately, which significantly degrades the attack detection performance. Besides, the reliability of CSI based detection schemes is challenged by time-varying scenarios. To address these issues, we propose an adaptive Kalman based detection scheme. By utilizing the knowledge of the predicted channel we eliminate the channel estimation error, especially the random phase error which occurs due to the lack of synchronization between transmitter and receiver. Furthermore, we define a Kalman residual based test statistic for attack detection. Simulation results show that our proposed scheme makes the detection more robust at low signal-to-noise ratio (SNR) and in dynamic scenarios.