FAST: Fast Audio Spectrogram Transformer

In audio classification, developing efficient and robust models is critical for real-time applications. Inspired by the design principles of MobileViT, we present FAST (Fast Audio Spectrogram Transformer), a new architecture that combines convolutional neural networks (CNNs) and transformers to capitalize on the strengths of both. FAST integrates the local feature extraction efficiencies of CNNs with the global context modeling capabilities of transformers, resulting in a model that is powerful yet lightweight, well-suited to a real-time or mobile use case. Additionally, we incorporate Lipschitz continuous attention mechanisms to improve training stability and accelerate convergence. We evaluate FAST on the ADIMA dataset, a multilingual corpus towards real-time profanity and abuse detection, as well as on the more traditional AudioSet. Our results show that FAST achieves state-of-the-art performance on both the ADIMA and AudioSet classification tasks and in some cases surpasses existing benchmarks while using up to 150x fewer parameters.

Fixed-MAML for Few Shot Classification in Multilingual Speech Emotion Recognition

In this paper, we analyze the feasibility of applying few-shot learning to speech emotion recognition task (SER). The current speech emotion recognition models work exceptionally well but fail when then input is multilingual. Moreover, when training such models, the models' performance is suitable only when the training corpus is vast. This availability of a big training corpus is a significant problem when choosing a language that is not much popular or obscure. We attempt to solve this challenge of multilingualism and lack of available data by turning this problem into a few-shot learning problem. We suggest relaxing the assumption that all N classes in an N-way K-shot problem be new and define an N+F way problem where N and F are the number of emotion classes and predefined fixed classes, respectively. We propose this modification to the Model-Agnostic MetaLearning (MAML) algorithm to solve the problem and call this new model F-MAML. This modification performs better than the original MAML and outperforms on EmoFilm dataset.

Removing Class Imbalance using Polarity-GAN: An Uncertainty Sampling Approach

Class imbalance is a challenging issue in practical classification problems for deep learning models as well as for traditional models. Traditionally successful countermeasures such as synthetic over-sampling have had limited success with complex, structured data handled by deep learning models. In this work, we propose to use a Generative Adversarial Network (GAN) equipped with a generator network G, a discriminator network D and a classifier network C to remove the class-imbalance in visual data sets. The generator network is initialized with auto-encoder to make it stable. The discriminator D ensures that G adheres to class distribution of imbalanced class. In conventional methods, where Generator G competes with discriminator D in a min-max game, we propose to further add an additional classifier network to the original network. Now, the generator network tries to compete in a min-max game with Discriminator as well as the new classifier that we have introduced. An additional condition is enforced on generator network G to produce points in the convex hull of desired imbalanced class. Further the contention of adversarial game with classifier C, pushes conditional distribution learned by G towards the periphery of the respective class, compensating the problem of class imbalance. Experimental evidence shows that this initialization results in stable training of the network. We achieve state of the art performance on extreme visual classification task on the FashionMNIST, MNIST, SVHN, ExDark, MVTec Anomaly Detection dataset, Chest X-Ray dataset and others.