In this paper, we propose standard statistical tools as a solution to commonly highlighted problems in the explainability literature. Indeed, leveraging statistical estimators allows for a proper definition of explanations, enabling theoretical guarantees and the formulation of evaluation metrics to quantitatively assess the quality of explanations. This approach circumvents, among other things, the subjective human assessment currently prevalent in the literature. Moreover, we argue that uncertainty quantification is essential for providing robust and trustworthy explanations, and it can be achieved in this framework through classical statistical procedures such as the bootstrap. However, it is crucial to note that while Statistics offers valuable contributions, it is not a panacea for resolving all the challenges. Future research avenues could focus on open problems, such as defining a purpose for the explanations or establishing a statistical framework for counterfactual or adversarial scenarios.