Given the current social distancing regulations across the world, social media has become the primary mode of communication for most people. This has resulted in the isolation of many people suffering from mental illnesses who are unable to receive assistance in person. They have increasingly turned to social media to express themselves and to look for guidance in dealing with their illnesses. Keeping this in mind, we propose a solution to detect and classify mental illness posts on social media thereby enabling users to seek appropriate help. In this work, we detect and classify five prominent kinds of mental illnesses: depression, anxiety, bipolar disorder, ADHD and PTSD by analyzing unstructured user data on social media platforms. In addition, we are sharing a new high-quality dataset to drive research on this topic. We believe that our work is the first multi-class model that uses a Transformer-based architecture such as RoBERTa to analyze people's emotions and psychology. We also demonstrate how we stress-test our model using behavioral testing. With this research, we hope to be able to contribute to the public health system by automating some of the detection and classification process.