PAC-Net: A Model Pruning Approach to Inductive Transfer Learning

Inductive transfer learning aims to learn from a small amount of training data for the target task by utilizing a pre-trained model from the source task. Most strategies that involve large-scale deep learning models adopt initialization with the pre-trained model and fine-tuning for the target task. However, when using over-parameterized models, we can often prune the model without sacrificing the accuracy of the source task. This motivates us to adopt model pruning for transfer learning with deep learning models. In this paper, we propose PAC-Net, a simple yet effective approach for transfer learning based on pruning. PAC-Net consists of three steps: Prune, Allocate, and Calibrate (PAC). The main idea behind these steps is to identify essential weights for the source task, fine-tune on the source task by updating the essential weights, and then calibrate on the target task by updating the remaining redundant weights. Under the various and extensive set of inductive transfer learning experiments, we show that our method achieves state-of-the-art performance by a large margin.

Generative Evolutionary Strategy For Black-Box Optimizations

Many scientific and technological problems are related to optimization. Among them, black-box optimization in high-dimensional space is particularly challenging. Recent neural network-based black-box optimization studies have shown noteworthy achievements. However, their capability in high-dimensional search space is still limited. This study investigates a novel black-box optimization method based on evolution strategy and generative neural network model. We designed the algorithm so that the evolutionary strategy and the generative neural network model work cooperatively with each other. This hybrid model enables reliable training of surrogate networks; it optimizes multi-objective, high-dimensional, and stochastic black-box functions. In this experiment, our method outperforms baseline optimization methods, including an NSGA-II and Bayesian optimization.

Restructuring TCAD System: Teaching Traditional TCAD New Tricks

Traditional TCAD simulation has succeeded in predicting and optimizing the device performance; however, it still faces a massive challenge - a high computational cost. There have been many attempts to replace TCAD with deep learning, but it has not yet been completely replaced. This paper presents a novel algorithm restructuring the traditional TCAD system. The proposed algorithm predicts three-dimensional (3-D) TCAD simulation in real-time while capturing a variance, enables deep learning and TCAD to complement each other, and fully resolves convergence errors.

A Novel Approach for Semiconductor Etching Process with Inductive Biases

The etching process is one of the most important processes in semiconductor manufacturing. We have introduced the state-of-the-art deep learning model to predict the etching profiles. However, the significant problems violating physics have been found through various techniques such as explainable artificial intelligence and representation of prediction uncertainty. To address this problem, this paper presents a novel approach to apply the inductive biases for etching process. We demonstrate that our approach fits the measurement faster than physical simulator while following the physical behavior. Our approach would bring a new opportunity for better etching process with higher accuracy and lower cost.