Interpretable Explainability in Facial Emotion Recognition and Gamification for Data Collection

Training facial emotion recognition models requires large sets of data and costly annotation processes. To alleviate this problem, we developed a gamified method of acquiring annotated facial emotion data without an explicit labeling effort by humans. The game, which we named Facegame, challenges the players to imitate a displayed image of a face that portrays a particular basic emotion. Every round played by the player creates new data that consists of a set of facial features and landmarks, already annotated with the emotion label of the target facial expression. Such an approach effectively creates a robust, sustainable, and continuous machine learning training process. We evaluated Facegame with an experiment that revealed several contributions to the field of affective computing. First, the gamified data collection approach allowed us to access a rich variation of facial expressions of each basic emotion due to the natural variations in the players' facial expressions and their expressive abilities. We report improved accuracy when the collected data were used to enrich well-known in-the-wild facial emotion datasets and consecutively used for training facial emotion recognition models. Second, the natural language prescription method used by the Facegame constitutes a novel approach for interpretable explainability that can be applied to any facial emotion recognition model. Finally, we observed significant improvements in the facial emotion perception and expression skills of the players through repeated game play.

Governance and Communication of Algorithmic Decision Making: A Case Study on Public Sector

Algorithmic Decision Making (ADM) has permeated all aspects of society. Government organizations are also affected by this trend. However, the use of ADM has been getting negative attention from the public, media, and interest groups. There is little to no actionable guidelines for government organizations to create positive impact through ADM. In this case study, we examined eight municipal organizations in the Netherlands regarding their actual and intended use of ADM. We interviewed key personnel and decision makers. Our results show that municipalities mostly use ADM in an ad hoc manner, and they have not systematically defined or institutionalized a data science process yet. They operate risk averse, and they clearly express the need for cooperation, guidance, and even supervision at the national level. Third parties, mostly commercial, are often involved in the ADM development lifecycle, without systematic governance. Communication on the use of ADM is generally responsive to negative attention from the media and public. There are strong indications for the need of an ADM governance framework. In this paper, we present our findings in detail, along with actionable insights on governance, communication, and performance evaluation of ADM systems.

Monitoring spatial sustainable development: Semi-automated analysis of satellite and aerial images for energy transition and sustainability indicators

Solar panels are installed by a large and growing number of households due to the convenience of having cheap and renewable energy to power house appliances. In contrast to other energy sources solar installations are distributed very decentralized and spread over hundred-thousands of locations. On a global level more than 25% of solar photovoltaic (PV) installations were decentralized. The effect of the quick energy transition from a carbon based economy to a green economy is though still very difficult to quantify. As a matter of fact the quick adoption of solar panels by households is difficult to track, with local registries that miss a large number of the newly built solar panels. This makes the task of assessing the impact of renewable energies an impossible task. Although models of the output of a region exist, they are often black box estimations. This project's aim is twofold: First automate the process to extract the location of solar panels from aerial or satellite images and second, produce a map of solar panels along with statistics on the number of solar panels. Further, this project takes place in a wider framework which investigates how official statistics can benefit from new digital data sources. At project completion, a method for detecting solar panels from aerial images via machine learning will be developed and the methodology initially developed for BE, DE and NL will be standardized for application to other EU countries. In practice, machine learning techniques are used to identify solar panels in satellite and aerial images for the province of Limburg (NL), Flanders (BE) and North Rhine-Westphalia (DE).