Memory and Capacity of Graph Embedding Methods

This paper analyzes the graph embedding method introduced in \cite{Qiu_Recipe}, which is a bind-and-sum approach using spherical codes and the tensor product to represent the edge set of a graph. We compare our spherical/tensor method to similar methods, and in particular we examine the competing scheme of Rademacher codes paired with the Hadamard product. We show that the while the Hadamard product doesn't increase the dimension like the tensor product, it suffers a proportional penalty in the number of edges it can accurately store in superposition. In fact, the memory capacity ratio of the Rademacher/Hadmard scheme is the same as the spherical/tensor scheme, and so there is no true memory savings when using the Hadamard and its related binding operations. We present some experimental results confirming our theoretical ones and show that while on the surface our method might require more parameters than competing methods, in reality it has the same efficiency.