Rapid intensification (RI) is when a sudden and considerable increase in tropical cyclone (TC) intensity occurs. Accurate early prediction of RI from TC images is important for preventing the possible damages caused by TCs. The main difficulty of RI prediction is to extract important features that are effective for RI prediction, which is challenging even for experienced meteorologists. Inspired by the success of deep learning models for automatic feature extraction and strong predictive performance, we initiate this study that experiments with multiple domain-knowledge guided deep learning models. The goal is to evaluate the potential use of these models for RI prediction. Furthermore, we examine the internal states of the models to obtain visualizable insights for RI prediction. Our model is efficient in training while achieving state-of-the-art performance on the benchmark dataset on HSS metric. The results showcase the success of adapting deep learning to solve complex meteorology problems.