Proximal Policy Optimization for Tracking Control Exploiting Future Reference Information

In recent years, reinforcement learning (RL) has gained increasing attention in control engineering. Especially, policy gradient methods are widely used. In this work, we improve the tracking performance of proximal policy optimization (PPO) for arbitrary reference signals by incorporating information about future reference values. Two variants of extending the argument of the actor and the critic taking future reference values into account are presented. In the first variant, global future reference values are added to the argument. For the second variant, a novel kind of residual space with future reference values applicable to model-free reinforcement learning is introduced. Our approach is evaluated against a PI controller on a simple drive train model. We expect our method to generalize to arbitrary references better than previous approaches, pointing towards the applicability of RL to control real systems.

A feasibility study of deep neural networks for the recognition of banknotes regarding central bank requirements

This paper contains a feasibility study of deep neural networks for the classification of Euro banknotes with respect to requirements of central banks on the ATM and high speed sorting industry. Instead of concentrating on the accuracy for a large number of classes as in the famous ImageNet Challenge we focus thus on conditions with few classes and the requirement of rejection of images belonging clearly to neither of the trained classes (i.e. classification in a so-called 0-class). These special requirements are part of frameworks defined by central banks as the European Central Bank and are met by current ATMs and high speed sorting machines. We also consider training and classification time on state of the art GPU hardware. The study concentrates on the banknote recognition whereas banknote class dependent authenticity and fitness checks are a topic of its own which is not considered in this work.