Predictive Process Analytics is becoming an essential aid for organizations, providing online operational support of their processes. However, process stakeholders need to be provided with an explanation of the reasons why a given process execution is predicted to behave in a certain way. Otherwise, they will be unlikely to trust the predictive monitoring technology and, hence, adopt it. This paper proposes a predictive analytics framework that is also equipped with explanation capabilities based on the game theory of Shapley Values. The framework has been implemented in the IBM Process Mining suite and commercialized for business users. The framework has been tested on real-life event data to assess the quality of the predictions and the corresponding evaluations. In particular, a user evaluation has been performed in order to understand if the explanations provided by the system were intelligible to process stakeholders.