[New preprocessor] The "tile_colorfix+sharp" is added

[lllyasviel]

1.1.196 added "tile_colorfix+sharp"
This method allows you to control the latent sharpness of the outputs of ControlNet tile.
This preprocessor can prevent the Tile model from the tendency to create the somewhat blurred "harmonious atmosphere" in your upscaled images.

comparison 1

Input image (480x640)

Meta (Using T2I or I2I with 100% denoising strength, no extra upscaler/super-resolution, all results use same seed 12345):

1girl, best quality
Negative prompt: lowres, bad anatomy, bad hands, cropped, worst quality
Steps: 20, Sampler: Euler a, CFG scale: 7, Seed: 12345, Size: 960x1280, Model hash: 8712e20a5d, Model: Anything-V3.0, Clip skip: 2, Version: v1.2.1, ControlNet 0: "preprocessor: tile_XXXX, model: control_v11f1e_sd15_tile [a371b31b], weight: 1, starting/ending: (0, 1), resize mode: Crop and Resize, pixel perfect: True, control mode: Balanced, preprocessor params: (64, 8, 1)"

tile_resample:

tile_colorfix:

tile_colorfix+sharp (sharpness=0.5):

tile_colorfix+sharp (default, sharpness=1.0):

img2img with 0.25 denoising strength without using any ControlNet:

comparison 2 (the difference for realistic image is less obvious)

Input image (480x640)

Meta (Using T2I or I2I with 100% denoising strength, no extra upscaler/super-resolution, all results use same seed 12345):
old man, best quality
Negative prompt: lowres, bad anatomy, bad hands, cropped, worst quality
Steps: 20, Sampler: Euler a, CFG scale: 7, Seed: 12345, Size: 960x1280, Model hash: c0d1994c73, Model: realisticVisionV20_v20, Version: v1.2.1, ControlNet 0: "preprocessor: tile_XXXX, model: control_v11f1e_sd15_tile [a371b31b], weight: 1, starting/ending: (0, 1), resize mode: Crop and Resize, pixel perfect: True, control mode: Balanced, preprocessor params: (64, 8, 1)"

tile_resample:

tile_colorfix:

tile_colorfix+sharp (sharpness=0.5):

tile_colorfix+sharp (default, sharpness=1.0):

img2img with 0.25 denoising strength without using any ControlNet:

Note that all above results in this post are not cherry picked (they all use seed 12345). It is possible to get much better results using these preprocessors in other workflows (e.g., like using r-esrgan or other sr methods).

version

You need at least 1.1.196 to use it.

[lllyasviel]

Note that after 1.1.196, tile_colorfix+sharp becomes ControlNet's new best method in upscaling images.

[lllyasviel]

"tile_colorfix+sharp" requires less parameter tuning to apply to other models.
This is CounterfeitV30

1girl, black_bow, flower, bow, long_hair, plant, solo, blue_eyes, outdoors, skirt, long_sleeves, white_shirt, two_side_up, breasts, looking_at_viewer, grass, shirt
Negative prompt: lowres, bad anatomy, bad hands, text, error, missing fingers, extra digit, fewer digits, cropped, worst quality, low quality, normal quality, jpeg artifacts, signature, watermark, username, blurry
Steps: 20, Sampler: Euler a, CFG scale: 7, Seed: 525096802, Size: 960x1280, Model hash: cbfba64e66, Model: CounterfeitV30_v30, Clip skip: 2, ENSD: 31337, Version: v1.2.1, ControlNet 0: "preprocessor: tile_colorfix+sharp, model: control_v11f1e_sd15_tile [a371b31b], weight: 1, starting/ending: (0, 1), resize mode: Crop and Resize, pixel perfect: True, control mode: Balanced, preprocessor params: (64, 8, 1)"
(prompt is from Danbooru interrogator this time)

[lllyasviel]

From "Steps: 20, Sampler: Euler a, CFG scale: 7" we can see that it works well for common parameters without requiring much tuning

[Hosiokaa]

Could be a Me problem but I'm having weird color issues shown here.

Right = Original
Left = ControlNet Tile colorfix + sharp (same happens on colorfix too)
Original image was 1280x1664

1girl, grey eyes, solo, dark atmosphere, outdoors, cloud, short hair, long sleeves, sky, black hair, cloudy sky, brown eyes, androgynous, standing, tree, cowboy shot, headband, closed mouth, grass
Negative prompt: (worst quality, low quality:1.2), lowres, bad anatomy, bad hands, nsfw, realistic
Steps: 30, Sampler: Euler a, CFG scale: 7, Seed: 88027789, Size: 1576x2048, Model hash: ca455ce14f, Model: Baka_Baka-DiffusionV4A(FP16)outattxn, Clip skip: 2, Version: v1.2.1, ControlNet 0: "preprocessor: tile_colorfix+sharp, model: control_v11f1e_sd15_tile [a371b31b], weight: 1, starting/ending: (0, 1), resize mode: Crop and Resize, pixel perfect: True, control mode: Balanced, preprocessor params: (64, 8, 1)"

[Rkkss]

Getting ValueError: ControlNet failed to use VAE. Please try to add --no-half-vae, --no-halfand remove--precision full in launch cmd.
when run tile+colorfix, tile+colorfix sharp with multidiffusion.
I don't have those command in launch. Tested with anime kl-f8 and mse 84000.

The regular tile resample prep runs without problem.

[lllyasviel]

update all extensions and completely restart terminal

[Rkkss]

update all extensions and completely restart terminal

I have to set batch size to 1 for it to run otherwise I'll get OOM.
with regular resample preprocessor, I can do 4.

[lllyasviel]

yes it needs more computation

[BNP1111]

preprocessor tile_colorfix+sharp cant used with inpainting model

get this error:
RuntimeError: The size of tensor a (9) must match the size of tensor b (4) at non-singleton dimension 1

[DamienCz]

good afternoon,i just try to use this funcion,but i met a question about the variation ,could u tell me what's useful? compare with sharpness,what's value i should put usually?

[VladAndronik]

Would be grateful if you could point out how to use it in diffusers pipelines or how to implement it there?