Semi-supervised source localization with deep generative modeling

We develop a semi-supervised learning (SSL) approach for acoustic source localization based on deep generative modeling. Source localization in reverberant environments remains an open challenge, which machine learning (ML) has shown promise in addressing. While there are often large volumes of acoustic data in reverberant environments, the labels available for supervised learning are usually few. This limitation can impair practical implementation of ML. In our approach, we perform SSL with variational autoencoders (VAEs). This VAE-SSL approach uses a classifier network to estimate source location, and a VAE as a generative physical model. The VAE is trained to generate the phase of relative transfer functions (RTFs), in parallel with classifier training, on both labelled and unlabeled RTF samples. The VAE-SSL approach is compared with SRP-PHAT and convolutional neural networks. The VAE-SSL generated RTF phase patterns are assessed.