Fast Structured Orthogonal Dictionary Learning using Householder Reflections

In this paper, we propose and investigate algorithms for the structured orthogonal dictionary learning problem. First, we investigate the case when the dictionary is a Householder matrix. We give sample complexity results and show theoretically guaranteed approximate recovery (in the $l_{\infty}$ sense) with optimal computational complexity. We then attempt to generalize these techniques when the dictionary is a product of a few Householder matrices. We numerically validate these techniques in the sample-limited setting to show performance similar to or better than existing techniques while having much improved computational complexity.

Column Bound for Orthogonal Matrix Factorization

This article explores the intersection of the Coupon Collector's Problem and the Orthogonal Matrix Factorization (OMF) problem. Specifically, we derive bounds on the minimum number of columns $p$ (in $\mathbf{X}$) required for the OMF problem to be tractable, using insights from the Coupon Collector's Problem. Specifically, we establish a theorem outlining the relationship between the sparsity of the matrix $\mathbf{X}$ and the number of columns $p$ required to recover the matrices $\mathbf{V}$ and $\mathbf{X}$ in the OMF problem. We show that the minimum number of columns $p$ required is given by $p = \Omega \left(\max \left\{ \frac{n}{1 - (1 - \theta)^n}, \frac{1}{\theta} \log n \right\} \right)$, where $\theta$ is the i.i.d Bernoulli parameter from which the sparsity model of the matrix $\mathbf{X}$ is derived.

Efficient Matrix Factorization Via Householder Reflections

Motivated by orthogonal dictionary learning problems, we propose a novel method for matrix factorization, where the data matrix $\mathbf{Y}$ is a product of a Householder matrix $\mathbf{H}$ and a binary matrix $\mathbf{X}$. First, we show that the exact recovery of the factors $\mathbf{H}$ and $\mathbf{X}$ from $\mathbf{Y}$ is guaranteed with $\Omega(1)$ columns in $\mathbf{Y}$ . Next, we show approximate recovery (in the $l\infty$ sense) can be done in polynomial time($O(np)$) with $\Omega(\log n)$ columns in $\mathbf{Y}$ . We hope the techniques in this work help in developing alternate algorithms for orthogonal dictionary learning.