Learning to Generate Poetic Chinese Landscape Painting with Calligraphy

In this paper, we present a novel system (denoted as Polaca) to generate poetic Chinese landscape painting with calligraphy. Unlike previous single image-to-image painting generation, Polaca takes the classic poetry as input and outputs the artistic landscape painting image with the corresponding calligraphy. It is equipped with three different modules to complete the whole piece of landscape painting artwork: the first one is a text-to-image module to generate landscape painting image, the second one is an image-to-image module to generate stylistic calligraphy image, and the third one is an image fusion module to fuse the two images into a whole piece of aesthetic artwork.

Multi-granularity Item-based Contrastive Recommendation

Contrastive learning (CL) has shown its power in recommendation. However, most CL-based recommendation models build their CL tasks merely focusing on the user's aspects, ignoring the rich diverse information in items. In this work, we propose a novel Multi-granularity item-based contrastive learning (MicRec) framework for the matching stage (i.e., candidate generation) in recommendation, which systematically introduces multi-aspect item-related information to representation learning with CL. Specifically, we build three item-based CL tasks as a set of plug-and-play auxiliary objectives to capture item correlations in feature, semantic and session levels. The feature-level item CL aims to learn the fine-grained feature-level item correlations via items and their augmentations. The semantic-level item CL focuses on the coarse-grained semantic correlations between semantically related items. The session-level item CL highlights the global behavioral correlations of items from users' sequential behaviors in all sessions. In experiments, we conduct both offline and online evaluations on real-world datasets, where MicRec achieves significant improvements over competitive baselines. Moreover, we further verify the effectiveness of three CL tasks as well as the universality of MicRec on different matching models. The proposed MicRec is effective, efficient, universal, and easy to deploy, which has been deployed on a real-world recommendation system, affecting millions of users. The source code will be released in the future.

SE-GAN: Skeleton Enhanced GAN-based Model for Brush Handwriting Font Generation

Previous works on font generation mainly focus on the standard print fonts where character's shape is stable and strokes are clearly separated. There is rare research on brush handwriting font generation, which involves holistic structure changes and complex strokes transfer. To address this issue, we propose a novel GAN-based image translation model by integrating the skeleton information. We first extract the skeleton from training images, then design an image encoder and a skeleton encoder to extract corresponding features. A self-attentive refined attention module is devised to guide the model to learn distinctive features between different domains. A skeleton discriminator is involved to first synthesize the skeleton image from the generated image with a pre-trained generator, then to judge its realness to the target one. We also contribute a large-scale brush handwriting font image dataset with six styles and 15,000 high-resolution images. Both quantitative and qualitative experimental results demonstrate the competitiveness of our proposed model.

Mappa Mundi: An Interactive Artistic Mind Map Generator with Artificial Imagination

We present a novel real-time, collaborative, and interactive AI painting system, Mappa Mundi, for artistic Mind Map creation. The system consists of a voice-based input interface, an automatic topic expansion module, and an image projection module. The key innovation is to inject Artificial Imagination into painting creation by considering lexical and phonological similarities of language, learning and inheriting artist's original painting style, and applying the principles of Dadaism and impossibility of improvisation. Our system indicates that AI and artist can collaborate seamlessly to create imaginative artistic painting and Mappa Mundi has been applied in art exhibition in UCCA, Beijing

From Knowledge Map to Mind Map: Artificial Imagination

Imagination is one of the most important factors which makes an artistic painting unique and impressive. With the rapid development of Artificial Intelligence, more and more researchers try to create painting with AI technology automatically. However, lacking of imagination is still a main problem for AI painting. In this paper, we propose a novel approach to inject rich imagination into a special painting art Mind Map creation. We firstly consider lexical and phonological similarities of seed word, then learn and inherit original painting style of the author, and finally apply Dadaism and impossibility of improvisation principles into painting process. We also design several metrics for imagination evaluation. Experimental results show that our proposed method can increase imagination of painting and also improve its overall quality.