Due to digitalization in everyday life, the need for automatically recognizing handwritten digits is increasing. Handwritten digit recognition is essential for numerous applications in various industries. Bengali ranks the fifth largest language in the world with 265 million speakers (Native and non-native combined) and 4 percent of the world population speaks Bengali. Due to the complexity of Bengali writing in terms of variety in shape, size, and writing style, researchers did not get better accuracy using Supervised machine learning algorithms to date. Moreover, fewer studies have been done on Bangla handwritten digit recognition (BHwDR). In this paper, we proposed a novel CNN-based pre-trained handwritten digit recognition model which includes Resnet-50, Inception-v3, and EfficientNetB0 on NumtaDB dataset of 17 thousand instances with 10 classes.. The Result outperformed the performance of other models to date with 97% accuracy in the 10-digit classes. Furthermore, we have evaluated the result or our model with other research studies while suggesting future study