Ultrasonic guided waves are commonly used to localize structural damage in infrastructures such as buildings, airplanes, bridges. Damage localization can be viewed as an inverse problem. Physical model based techniques are popular for guided wave based damage localization. The performance of these techniques depend on the degree of faithfulness with which the physical model describes wave propagation. External factors such as environmental variations and random noise are a source of uncertainty in wave propagation. The physical modeling of uncertainty in an inverse problem is still a challenging problem. In this work, we propose a deep learning based model for robust damage localization in presence of uncertainty. Wave data with uncertainty is simulated to reflect variations due to external factors and Gaussian noise is added to reflect random noise in the environment. After evaluating the localization error on test data with uncertainty, we observe that the deep learning model trained with uncertainty can learn robust representations. The approach shows potential for dealing with uncertainty in physical science problems using deep learning models.