Functional Correlations in the Pursuit of Performance Assessment of Classifiers

In statistical classification, machine learning, social and other sciences, a number of measures of association have been developed and used for assessing and comparing individual classifiers, raters, and their groups. Among the measures, we find the weighted kappa, extensively used by psychometricians, and the monotone and supremum correlation coefficients, prominently used by social scientists and statisticians. In this paper, we introduce, justify, and explore several new members of the class of functional correlation coefficients that naturally arise when comparing classifiers. We illustrate the performance of the coefficients by reanalyzing a number of confusion matrices that have appeared in the literature.