Skip to content

feat: add lanpaint support - #1978

Open
chaserhkj wants to merge 6 commits into
leejet:masterfrom
chaserhkj:lanpaint
Open

chaserhkj wants to merge 6 commits into
leejet:masterfrom
chaserhkj:lanpaint

Conversation

@chaserhkj

@chaserhkj chaserhkj commented Sep 15, 2026

Copy link
Copy Markdown

Summary

This PR adds lanpaint sampling implementation for sd.cpp, together with command line interface and documentation.

Lanpaint adds supports for training-free inpainting with any model by leveraging Langevin dynamics numerical methods. This algorithm particularly is ported from the Comfy implementation of lanpaint linked above.

Related Issue / Discussion

#1954

Additional Information

Example with comparisons:

Given input image (generated using Anima model with a separate prompt):

input

If we want to inpaint the dress of the character into a black dress using the following mask:

mask

Without lanpaint, using the Anima model:

sd-cli \
	--diffusion-model ./models/diff/anima-aesthetic-v1.1.safetensors \
	--vae ./models/vae/qwen_image_vae.safetensors \
	--llm ./models/encode/qwen_3_06b_base.safetensors \
	--diffusion-fa --sampling-method euler --cfg-scale 6.0 --steps 20 \
	-i input.png --mask mask.png --strength 1.0 \
	-p "1 girl, long hair, black hair, standing, simple background, white background, black dress, sleeveless, short dress, off-shoulder dress, white thighhighs, cowboy shot, looking at camera, arms at sides, happy, smile, closed mouth, anime"

The model, without inpainting training, would struggle with the mask boundaries:
bad_inpaint

With lanpaint, a much better inpaint result would be obtained:

sd-cli \
	--diffusion-model ./models/diff/anima-aesthetic-v1.1.safetensors \
	--vae ./models/vae/qwen_image_vae.safetensors \
	--llm ./models/encode/qwen_3_06b_base.safetensors \
	--diffusion-fa --sampling-method euler --cfg-scale 6.0 --steps 20 \
	-i input.png --mask mask.png --strength 1.0 --lanpaint \
	-p "1 girl, long hair, black hair, standing, simple background, white background, black dress, sleeveless, short dress, off-shoulder dress, white thighhighs, cowboy shot, looking at camera, arms at sides, happy, smile, closed mouth, anime"
output

This method can also be paired with an image edit model to enable seamless pixel-identical masked edit

Checklist

Assisted-by: prime-agent deepseek-v4-flash-0731
Assisted-by: prime-agent glm-5.3-flash
Signed-off-by: Chaser Huang <huangkangjing@gmail.com>
Assisted-by: prime-agent glm-5.3-flash
Signed-off-by: Chaser Huang <huangkangjing@gmail.com>
Assisted-by: prime-agent glm-5.3-flash
Signed-off-by: Chaser Huang <huangkangjing@gmail.com>
…s helper

Assisted-by: prime-agent glm-5.3-flash
Signed-off-by: Chaser Huang <huangkangjing@gmail.com>
Assisted-by: prime-agent glm-5.3-flash
Signed-off-by: Chaser Huang <huangkangjing@gmail.com>
Assisted-by: prime-agent glm-5.3-flash
Signed-off-by: Chaser Huang <huangkangjing@gmail.com>
@stduhpf

stduhpf commented Sep 15, 2026

Copy link
Copy Markdown
Contributor

This method can also be paired with an image edit model to enable seamless pixel-identical masked edit

Most image edit models can already handle masked edit pretty well, especially if they're prompted accordingly. But this seems like a huge improvement for inpainting with regular diffusion model that don't have any native editing or inpainting capabilities.

@chaserhkj

Copy link
Copy Markdown
Author

Most image edit models can already handle masked edit pretty well, especially if they're prompted accordingly.

Yes, but the edit model still does not have true vision over the shape of the mask. In other words, the edit is still performed grossly over the entire image and then applied mask, and the prompt needs to describe how the edit is localized properly (e.g. if there are multiple dresses in the image and we only wants to change one, we need to state "change the color of the dress on the right", etc.) For some complex localized edits where such descriptions are tricky (e.g. editing part of a checkered pattern), Lanpaint can be a lot more helpful as well.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants