Adaptable Automation with Modular Deep Reinforcement Learning and Policy Transfer

Recent advances in deep Reinforcement Learning (RL) have created unprecedented opportunities for intelligent automation, where a machine can autonomously learn an optimal policy for performing a given task. However, current deep RL algorithms predominantly specialize in a narrow range of tasks, are sample inefficient, and lack sufficient stability, which in turn hinder their industrial adoption. This article tackles this limitation by developing and testing a Hyper-Actor Soft Actor-Critic (HASAC) RL framework based on the notions of task modularization and transfer learning. The goal of the proposed HASAC is to enhance the adaptability of an agent to new tasks by transferring the learned policies of former tasks to the new task via a "hyper-actor". The HASAC framework is tested on a new virtual robotic manipulation benchmark, Meta-World. Numerical experiments show superior performance by HASAC over state-of-the-art deep RL algorithms in terms of reward value, success rate, and task completion time.

Performance Analysis of Meta-heuristic Algorithms for a Quadratic Assignment Problem

A quadratic assignment problem (QAP) is a combinatorial optimization problem that belongs to the class of NP-hard ones. So, it is difficult to solve in the polynomial time even for small instances. Research on the QAP has thus focused on obtaining a method to overcome this problem. Heuristics and meta-heuristics algorithm are prevalent solution methods for this problem. This paper is one of comparative studies to apply different metaheuristic algorithms for solving the QAP. One of the most popular approaches for categorizing meta-heuristic algorithms is based on a search strategy, including (1) local search improvement meta-heuristics and (2) global search-based meta-heuristics. The matter that distinguishes this paper from the other is the comparative performance of local and global search (both EA and SI), in which meta-heuristics that consist of genetic algorithm (GA), particle swarm optimization (PSO), hybrid GA-PSO, grey wolf optimization (GWO), harmony search algorithm (HAS) and simulated annealing (SA). Also, one improvement heuristic algorithm (ie, 2-Opt) is used to compare with others. The PSO, GWO and 2-Opt algorithms are improved to achieve the better comparison toward the other algorithms for evaluation. In order to analysis the comparative advantage of these algorithms, eight different factors are presented. By taking into account all these factors, the test is implemented in six test problems of the QAP Library (QAPLIB) from different sizes. Another contribution of this paper is to measure a strong convergence condition for each algorithm in a new way.

Swapped Face Detection using Deep Learning and Subjective Assessment

The tremendous success of deep learning for imaging applications has resulted in numerous beneficial advances. Unfortunately, this success has also been a catalyst for malicious uses such as photo-realistic face swapping of parties without consent. Transferring one person's face from a source image to a target image of another person, while keeping the image photo-realistic overall has become increasingly easy and automatic, even for individuals without much knowledge of image processing. In this study, we use deep transfer learning for face swapping detection, showing true positive rates >96% with very few false alarms. Distinguished from existing methods that only provide detection accuracy, we also provide uncertainty for each prediction, which is critical for trust in the deployment of such detection systems. Moreover, we provide a comparison to human subjects. To capture human recognition performance, we build a website to collect pairwise comparisons of images from human subjects. Based on these comparisons, images are ranked from most real to most fake. We compare this ranking to the outputs from our automatic model, showing good, but imperfect, correspondence with linear correlations >0.75. Overall, the results show the effectiveness of our method. As part of this study, we create a novel, publicly available dataset that is, to the best of our knowledge, the largest public swapped face dataset created using still images. Our goal of this study is to inspire more research in the field of image forensics through the creation of a public dataset and initial analysis.