Several important models of machine learning algorithms have been successfully generalized to the quantum world, with potential applications to data analytics in quantum physics that can be implemented on the near future quantum computers. However, noise and decoherence are two major obstacles to the practical implementation of quantum machine learning. In this work, we introduce a general framework for the robustness analysis of quantum machine learning algorithms against noise and decoherence. We argue that fidelity is the only pick of measuring the robustness. A robust bound is derived and an algorithm is developed to check whether or not a quantum machine learning algorithm is robust with respect to the training data. In particular, this algorithm can help to defense attacks and improve the accuracy as it can identify useful new training data during checking. The effectiveness of our robust bound and algorithm is confirmed by the case study of quantum phase recognition. Furthermore, this experiment demonstrates a trade-off between the accuracy of quantum machine learning algorithms and their robustness.