In this paper, we analyze some of our real-world deployment of face recognition (FR) systems for various applications and discuss the gaps between expectations of the user and what the system can deliver. We evaluate some of our proposed algorithms with ad-hoc modifications for applications such as FR on wearable devices (like Google Glass), monitoring of elderly people in senior citizens centers, FR of children in child care centers and face matching between a scanned IC/passport face image and a few live webcam images for automatic hotel/resort checkouts. We describe each of these applications, the challenges involved and proposed solutions. Since FR is intuitive in nature and we human beings use it for interactions with the outside world, people have high expectations of its performance in real-world scenarios. However, we analyze and discuss here that it is not the case, machine recognition of faces for each of these applications poses unique challenges and demands specific research components so as to adapt in the actual sites.