Intelligent painter: New masking strategy and self-referencing with resampling

Abstract

Painting with our own hands is not everyone’s talent. Some of us may dream big to create our own artwork but do not have the ability to do so. With the help of deep learning techniques, we nowadays can generate text-based painting. However, just typing text to create our own artwork is still different from doing it yourself (DIY). We proposed an application called intelligent painter, which can let users decide the placement of the objects and use the diffusion models to fill all the gaps after the users finish their placement. In this paper, we propose two major contributions to make better generation of images by (i) a new masking strategy and (ii) speeding up the process by 50% compared with resampling Denoising Diffusion Probabilistic Models (DDPM), with a self-pre-processing input step.