Malaria is a life-threatening mosquito-borne blood disease, hence early detection is very crucial for health. The conventional method for the detection is a microscopic examination of Giemsa-stained blood smears, which needs a highly trained skilled technician. Automated classifications of different stages of malaria still a challenging task, especially having poor sensitivity in detecting the early trophozoite and late trophozoite or schizont stage with limited labelled datasize. The study aims to develop a fast, robust and fully automated system for the classification of different stages of malaria with limited data size by using the pre-trained convolutional neural networks (CNNs) as a classifier and multi-wavelength to increase the sample size. We also compare our customized CNN with other well-known CNNs and shows that our network have a comparable performance with less computational time. We believe that our proposed method can be applied to other limited labelled biological datasets.