Unsupervised MKL in Multi-layer Kernel Machines

Kernel based Deep Learning using multi-layer kernel machines(MKMs) was proposed by Y.Cho and L.K. Saul in \cite{saul}. In MKMs they used only one kernel(arc-cosine kernel) at a layer for the kernel PCA-based feature extraction. We propose to use multiple kernels in each layer by taking a convex combination of many kernels following an unsupervised learning strategy. Empirical study is conducted on \textit{mnist-back-rand}, \textit{mnist-back-image} and \textit{mnist-rot-back-image} datasets generated by adding random noise in the image background of MNIST dataset. Experimental results indicate that using MKL in MKMs earns a better representation of the raw data and improves the classifier performance.

Framework for Designing Filters of Spectral Graph Convolutional Neural Networks in the Context of Regularization Theory

Graph convolutional neural networks (GCNNs) have been widely used in graph learning. It has been observed that the smoothness functional on graphs can be defined in terms of the graph Laplacian. This fact points out in the direction of using Laplacian in deriving regularization operators on graphs and its consequent use with spectral GCNN filter designs. In this work, we explore the regularization properties of graph Laplacian and proposed a generalized framework for regularized filter designs in spectral GCNNs. We found that the filters used in many state-of-the-art GCNNs can be derived as a special case of the framework we developed. We designed new filters that are associated with well-defined regularization behavior and tested their performance on semi-supervised node classification tasks. Their performance was found to be superior to that of the other state-of-the-art techniques.

Disentanglement based Active Learning

We propose Disentanglement based Active Learning (DAL), a new active learning technique based on query synthesis which leverages the concept of disentanglement. Instead of requesting labels from the human oracle, our method automatically labels majority of the datapoints, thus drastically reducing the human labelling budget in active learning. The proposed method uses Information Maximizing Generative Adversarial Nets (InfoGAN) to achieve the task where the active learner provides a feedback on the generation of InfoGAN based on which decision is taken about the datapoints to be queried. Results on two benchmark datasets demonstrate that DAL is able to achieve nearly fully supervised accuracy with fairly less labelling budget compared to existing active learning approaches.