Learning to Optimize Domain Specific Normalization for Domain Generalization

We propose a simple but effective multi-source domain generalization technique based on deep neural networks by incorporating optimized normalization layers specific to individual domains. Our key idea is to decompose discriminative representations in each domain into domain-agnostic and domain-specific components by learning a mixture of multiple normalization types. Because each domain has different characteristics, we optimize the mixture weights specialized to each domain and maximize the generalizability of the learned representations per domain. To this end, we incorporate instance normalization into the network with batch normalization since instance normalization is effective to discard the discriminative domain-specific representations. Since the joint optimization of the parameters in convolutional and normalization layers is not straightforward especially in the lower layers, the mixture weight of the normalization types is shared across all layers for the robustness of trained models. We analyze the effectiveness of the optimized normalization layers and demonstrate the state-of-the-art accuracy of our algorithm in the standard benchmark datasets in various settings.