[Bug]: Inpainting mask is sometimes ignored

[ArthurHeitmann]

Is there an existing issue for this?

• I have searched the existing issues and checked the recent builds/commits

What happened?

When painting a mask in inpaint mode, the mask is completely ignored, as if everything was painted, and behaves as if in normal img2img mode. Happens in Chrome but not in (chromium) Edge.
TLDR; Inpainting doesn't work at all

Potential fix and exactly what goes wrong at the end at "Additional information".

Steps to reproduce the problem

1. Send any image in Inpainting
2. Paint a small area
3. Generate
4. Entire image is regenerated :/

What should have happened?

Only the painted area should change.

Commit where the problem happens

v1.2.1 89f9faa

What platforms do you use to access the UI ?

Windows

What browsers do you use to access the UI ?

Google Chrome

Command Line Arguments

--opt-channelslast --opt-sdp-attention

List of extensions

sd-dynamic-prompts
stable-diffusion-webui-images-browser

Console logs

Python 3.10.6 (tags/v3.10.6:9c7b4bd, Aug  1 2022, 21:53:49) [MSC v.1932 64 bit (AMD64)]
Version: v1.2.1
Commit hash: 89f9faa63388756314e8a1d96cf86bf5e0663045
Installing requirements
Installing sd-dynamic-prompts requirements.txt


Launching Web UI with arguments: 
No module 'xformers'. Proceeding without it.
Image Browser: ImageReward is not installed, cannot be used.
Loading weights [a60cfaa90d] from D:\Cloud\Documents\Programming\python\stable-diffusion-webui\models\Stable-diffusion\dreamshaper_5BakedVae.safetensors
Creating model from config: D:\Cloud\Documents\Programming\python\stable-diffusion-webui\configs\v1-inference.yaml
LatentDiffusion: Running in eps-prediction mode
DiffusionWrapper has 859.52 M params.
Running on local URL:  http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.
Startup time: 9.3s (import torch: 1.6s, import gradio: 2.0s, import ldm: 0.5s, other imports: 1.1s, load scripts: 1.0s, create ui: 2.6s, gradio launch: 0.3s).
Applying cross attention optimization (Doggettx).
Textual inversion embeddings loaded(4): bad-hands-5, bad-image-v2-39000, bad_prompt_version2, EasyNegative
Model loaded in 5.0s (load weights from disk: 0.9s, create model: 0.8s, apply weights to model: 0.7s, apply half(): 0.9s, move model to device: 0.6s, load textual inversion embeddings: 1.0s).
100%|█████████████████████████████████████████████████████████████████████████████████████████████████| 13/13 [00:01<00:00,  9.68it/s]
Total progress: 100%|█████████████████████████████████████████████████████████████████████████████████| 13/13 [00:00<00:00, 14.89it/s]
Total progress: 100%|█████████████████████████████████████████████████████████████████████████████████| 13/13 [00:00<00:00, 17.40it/s]

Additional information

In short

Black pixels on HTML canvas aren't actually black and change each time when painting. Since they aren't 100% black the backend treats them as white, and as a result the entire mask as white.

Fix

At img2img.py line 98 change

mask = ImageChops.lighter(alpha_mask, mask.convert('L')).convert('L')

to

mask = mask.convert('L').point(lambda x: 255 if x > 128 else 0, mode='1')
mask = ImageChops.lighter(alpha_mask, mask).convert('L')

This will 100% make sure that only the painted area is masked and no noise is included. Only potential issue is, if there's a plan to have partial transparency painting in the UI.

In more detail

For some reason in my chrome there is some noise in the HTML canvas element. As a result supposedly black pixels aren't actually black. Each time you draw on the canvas, the noise changes slightly.

In the img2img.py in line 98

mask = ImageChops.lighter(alpha_mask, mask.convert('L')).convert('L')

the alpha_mask has the mode '1' (black or white pixels, 0 or 255) and mask will have the mode 'L' (grayscale, 0 - 255). However the ImageChops.lighter operation will automatically convert mask to also be '1'. This will turn any pixels that aren't 100% black to white. In my case no single mask pixel is 100% black. So the entire mask turns white.

To prevent this from happening the mask has to be manually converted to mode '1' beforehand.

I think this is a chrome and not a gradio issue. So waiting for any kind of fix will take a long time.

I will soon open a PR for the issue with the above fix.

[ArthurHeitmann]

Fixed in #10644

[GunnyNinja]

It's not fixed. If you don't reset the paintbrush between edits, it will also alter unmasked portions of the image.

[Photosounder]

Is it the same as #8288?

[ArthurHeitmann]

It could be. You can test the fix, which is on the dev branch

[Photosounder]

OK I confirm that it is not fixed, the problem remains with the latest dev commit.

[ArthurHeitmann]

So I looked further into the issue #8288
The image there has a small number of transparent pixel at the top:

It turns out that the inverted alpha channel of the image is also used as the inpainting mask.

In that case, removing the transparency would be the quick fix here.

Technically this is intentional, as described here. The question is if this is what is supposed to happen, when both an alpha mask exists and a mask is drawn in the UI.

[the-me-0]

I can confirm that in imgtoimg -> "inpaint sketch" mode, the "inpaint area" setting seem to be ignored sometimes when switching the base image. Idk precisely how to reproduce it, but it's kind of annoying.