The last few decades have seen significant breakthroughs in the fields of deep learning and quantum computing. Research at the junction of the two fields has garnered an increasing amount of interest, which has led to the development of quantum deep learning and quantum-inspired deep learning techniques in recent times. In this work, we present an overview of advances in the intersection of quantum computing and deep learning by discussing the technical contributions, strengths and similarities of various research works in this domain. To this end, we review and summarise the different schemes proposed to model quantum neural networks (QNNs) and other variants like quantum convolutional networks (QCNNs). We also briefly describe the recent progress in quantum inspired classic deep learning algorithms and their applications to natural language processing.