Deep Reinforcement Learning for Heat Pump Control

Heating in private households is a major contributor to the emissions generated today. Heat pumps are a promising alternative for heat generation and are a key technology in achieving our goals of the German energy transformation and to become less dependent on fossil fuels. Today, the majority of heat pumps in the field are controlled by a simple heating curve, which is a naive mapping of the current outdoor temperature to a control action. A more advanced control approach is model predictive control (MPC) which was applied in multiple research works to heat pump control. However, MPC is heavily dependent on the building model, which has several disadvantages. Motivated by this and by recent breakthroughs in the field, this work applies deep reinforcement learning (DRL) to heat pump control in a simulated environment. Through a comparison to MPC, it could be shown that it is possible to apply DRL in a model-free manner to achieve MPC-like performance. This work extends other works which have already applied DRL to building heating operation by performing an in-depth analysis of the learned control strategies and by giving a detailed comparison of the two state-of-the-art control methods.

GAN pretraining for deep convolutional autoencoders applied to Software-based Fingerprint Presentation Attack Detection

The need for reliable systems to determine fingerprint presentation attacks grows with the rising use of the fingerprint for authentication. This work presents a new approach to single-class classification for software-based fingerprint presentation attach detection. The described method utilizes a Wasserstein GAN to apply transfer learning to a deep convolutional autoencoder. By doing so, the autoencoder could be pretrained and finetuned on the LivDet2021 Dermalog sensor dataset with only 1122 bona fide training samples. Without making use of any presentation attack samples, the model could archive an average classification error rate of 16.79%. The Wasserstein GAN implemented to pretrain the autoencoders weights can further be used to generate realistic-looking artificial fingerprint patches. Extensive testing of different autoencoder architectures and hyperparameters led to coarse architectural guidelines as well as multiple implementations which can be utilized for future work.