Consistent Batch Normalization for Weighted Loss in Imbalanced-Data Environment

In this study, we consider classification problems based on neural networks in a data-imbalanced environment. Learning from an imbalanced dataset is one of the most important and practical problems in the field of machine learning. A weighted loss function (WLF) based on a cost-sensitive approach is a well-known and effective method for imbalanced datasets. We consider a combination of WLF and batch normalization (BN) in this study. BN is considered as a powerful standard technique in the recent developments in deep learning. A simple combination of both methods leads to a size-inconsistency problem due to a mismatch between the interpretations of the effective size of the dataset in both methods. We propose a simple modification to BN, called weighted batch normalization (WBN), to correct the size-mismatch. The idea of WBN is simple and natural. Using numerical experiments, we demonstrate that our method is effective in a data-imbalanced environment.