In this paper, we present Neural Glyph, a system that generates novel glyph without any training data. The generator and the classifier are trained to communicate via visual symbols as a medium, which enforces the generator to come up with a set of distinctive symbols. Our method results in glyphs that resemble the human-made glyphs, which may imply that the visual appearances of existing glyphs can be attributed to constraints of communication via writing. Important tricks that enable this framework is described and the code is made available.