Machine learning research in GNSS radio frequency interference (RFI) detection often lacks a clear empirical justification for the choice of deep learning architectures over simpler machine learning approaches. In this work, we argue for a change in research direction-from developing ever more complex deep learning models to carefully assessing their real-world effectiveness in comparison to interpretable and lightweight machine learning baselines. Our findings reveal that state-of-the-art deep learning models frequently fail to outperform simple, well-engineered machine learning methods in the context of GNSS RFI detection. Leveraging a unique large-scale dataset collected by the Swiss Air Force and Swiss Air-Rescue (Rega), and preprocessed by Swiss Air Navigation Services Ltd. (Skyguide), we demonstrate that a simple baseline model achieves 91\% accuracy in detecting GNSS RFI, outperforming more complex deep learning counterparts. These results highlight the effectiveness of pragmatic solutions and offer valuable insights to guide future research in this critical application domain.