The author of ControlNet's new project PaintsUndo - Foundation Model for Human Drawing Behavior

PaintsUndo, which was open-sourced yesterday, is a project aimed at providing a foundational model for human painting behavior, with the hope that future AI models will better meet the practical needs of human artists.

Showcases

  • Input:

  • Output:

Extracting Coarse Sketches

  • Input:

  • Coarse:

  • Very Coarse:

  • Extremely Coarse:

Interpolate from External Sketches

  • Origin:

  • Anime2Sketch's Output:

  • Interpolation Output:

It showcases a series of models that take images as input and then output the process of how the image was drawn.

The main functions of the model include:

  • Sketching
  • Outlining
  • Coloring
  • Shading
  • Deformation
  • Horizontal flip
  • Color curve adjustment
  • Layer visibility change
  • Change the overall concept during the painting process

Usage steps

https://huggingface.co/spaces/JacobLinCool/PaintsUndo

The first step: Upload images and generate prompts

The second step: Generate keyframes

The third step: Generate all videos