Cerebrovascular accident (CVA) or stroke is the rapid loss of brain function due to disturbance in the blood supply to the brain. Statistically, stroke is the second leading cause of death. This has motivated us to suggest a two-tier system for predicting stroke; the first tier makes use of Artificial Neural Network (ANN) to predict the chances of a person suffering from stroke. The ANN is trained the using the values of various risk factors of stroke of several patients who had stroke. Once a person is classified as having a high risk of stroke, s/he undergoes another the tier-2 classification test where his/her neuro MRI (Magnetic resonance imaging) is analysed to predict the chances of stroke. The tier-2 uses Non-negative Matrix Factorization and Haralick Textural features for feature extraction and SVM classifier for classification. We have obtained an accuracy of 96.67% in tier-1 and an accuracy of 70% in tier-2.