Tutorial

[PotatoBananaApple]

Hey,

I started creating tutorial for this extension, as it seemed like some people were interested in having one. It is not perfect and is written from my own experience from using the extension for while now. If you have time to check it out, let me know if you see something wrong with it.

As of now it consist of 3 parts.

1. How to use + t2i workflow
2. img2img workflow
3. region prompt control workflow

I plan to combine them into 1 big tutorial once i feel like i am finished with it.

https://civitai.com/models/34726

Thanks :)

[Pickles726]

Potato, this guide was very helpful. I noted in your tutorial that you're running on a 12gb card, which is what I have as well. Have you run into any limitations on your card when doing anything higher than 1280x1280 in img2img? I usually run OOM if I get much higher than 1280x1280 using your workflow.

[PotatoBananaApple]

Potato, this guide was very helpful. I noted in your tutorial that you're running on a 12gb card, which is what I have as well. Have you run into any limitations on your card when doing anything higher than 1280x1280 in img2img? I usually run OOM if I get much higher than 1280x1280 using your workflow.

Hey,

Thank you for liking the guide :). I have not run in to concrete wall yet. What i mean is with the right settings the limitations go higher. The highest i have ran is 2560x2560 as a test . The time becomes second issue at those resolutions :P

Make sure you have xformers enabled, It lowers VRAM usage and speeds up generating. Try lowering VAE tile size or adjusting tile width/height and batch size when you run out of VRAM

Drop this img into PNG info and you can see what settings i had for the 2560x2560. Sadly i had too high noise/too low tilesize for the image and got some random stuff in there. Fun stuff after generating it for 40 minutes (～￣▽￣)～

[Pickles726]

Potato, this guide was very helpful. I noted in your tutorial that you're running on a 12gb card, which is what I have as well. Have you run into any limitations on your card when doing anything higher than 1280x1280 in img2img? I usually run OOM if I get much higher than 1280x1280 using your workflow.

Hey,

Thank you for liking the guide :). I have not run in to concrete wall yet. What i mean is with the right settings the limitations go higher. The highest i have ran is 2560x2560 as a test . The time becomes second issue at those resolutions :P

Make sure you have xformers enabled, It lowers VRAM usage and speeds up generating. Try lowering VAE tile size or adjusting tile width/height and batch size when you run out of VRAM

Drop this img into PNG info and you can see what settings i had for the 2560x2560. Sadly i had too high noise/too low tilesize for the image and got some random stuff in there. Fun stuff after generating it for 40 minutes (～￣▽￣)～

Awesome, thank you for the reply. I enabled xformers and didn't see much of a difference at first. In fact, it was roughly six times worse on speed and memory consumption stayed the same. I went in and took a look at my startup options and removed everything but xformers and the share command. Good grief that difference is huge.

Complete user error on my part. I appreciate what you do, and I look forward to any further guides you may produce.

[PotatoBananaApple]

Awesome, thank you for the reply. I enabled xformers and didn't see much of a difference at first. In fact, it was roughly six times worse on speed and memory consumption stayed the same. I went in and took a look at my startup options and removed everything but xformers and the share command. Good grief that difference is huge.

Complete user error on my part. I appreciate what you do, and I look forward to any further guides you may produce.

No problem, glad you got it figured out :). Planning on doing guide on the Region Prompt Control soonish.

Only things i have on my start up are --no-half-vae --xformers .

[pkuliyi2015]

Some general tips on performance:

• Don't use --lowvram as it will insanely decrease your speed. When you use it, your generation goes to >30 min.
• Similarly, don't use --medvram as well, if possible.
• Larger latent tile batch size can significantly improve your speed. When it is 4 or 8 you can reach max speed (nearly doubled than batch size=1).
• Don't use DPM++ SDE Karras. It contains useless extra sampling steps. If you are using it, try DPM++ 2M Karras instead.
• Tile overlap should be less than 48. You can try 48, 32, or 16. Mixture of Diffusers is recommended for smaller settings.
• 25 denoise steps is generally enough.

Tips for the latest ControlNet 1.1 tile model (CN Tile for short):

• CN Tile + Mixture of diffusers + Noise inversion steps = 50 + denoise = 0.75 yields crazily good results.
  • When use CN Tile, you are likely to get weird details (e.g., a white hair girl's hair will be dirty) even if you use a low denoising strength (=0.3)
  • When use Noise Inversion, you are likely to get a retouched painting, where the details may be lost.
  • If you combine the two but the steps is not enough (i.e., <30 steps), you will get blurry images.
  • If you use high NI steps (e.g., 50 steps), while the time will be much longer, the result will be amazingly good. Possibly the best result I have ever seen.
• Comparisons: https://imgsli.com/MTc1OTc3 (2560 * 3880), generate in ~ 8 minutes.
• If you have sufficient time, you can go up to NI-200 steps (you need to modify ui-config.json to break the 100 upperlimit) and denoising strength=1. It will take ~ 30 minutes to do NI, but the result is insanely detailed (Note that denoise=1 will bring some color drifting): https://imgsli.com/MTc1OTc4
• One more result: https://imgsli.com/MTc2MDU0

[PotatoBananaApple]

Some general tips on performance:

* Don't use --lowvram as it will insanely decrease your speed. When you use it, your generation goes to >30 min.

* Similarly, don't use --medvram as well, if possible.

* Larger latent tile batch size can significantly improve your speed. When it is 4 or 8 you can reach max speed (nearly doubled than batch size=1).

* Don't use DPM++ SDE Karras. It contains useless extra sampling steps. If you are using it, try DPM++ 2M Karras instead.

* Tile overlap should be less than 48. You can try 48, 32, or 16. Mixture of Diffusers is recommended for smaller settings.

* 25 denoise steps is generally enough.

Tips for the latest ControlNet 1.1 tile model (CN Tile for short):

* **CN Tile + Mixture of diffusers + Noise inversion steps = 50 + denoise = 0.75** yields crazily good results.
  
  * When use CN Tile, you are likely to get weird details (e.g., a white hair girl's hair will be dirty) even if you use a low denoising strength (=0.3)
  * When use Noise Inversion, you are likely to get a retouched painting, where the details may be lost.
  * If you combine the two but the steps is not enough (i.e., <30 steps), you will get blurry images.
  * If you use high NI steps (e.g., 50 steps), while the time will be much longer, the result will be amazingly good. Possibly the best result I have ever seen.

* Comparisons: https://imgsli.com/MTc1OTc3 (2560 * 3880), generate in ~ 8 minutes.

* If you have sufficient time, you can go up to NI-200 steps (you need to modify ui-config.json to break the 100 upperlimit) and denoising strength=1. It will take ~ 30 minutes to do NI, but the result is insanely detailed (Note that denoise=1 will bring some color drifting): https://imgsli.com/MTc1OTc4

Those are nice tips, going to drop them to my tutorial :P I personally always use high denoise. What i have noticed is, if you want to use high denoise 0.5-0.60, it is better to have either height/width higher and denoise close to the lowest one. ex: height 64, width 92-112 and tile overlap 52-60, raises generate time but gives pretty nice results. With CN on, it seems like tile overlap can be dropped pretty low with high denoise.

For AUTOMATIC1111 webui you have to change setting called With img2img, do exactly the amount of steps the slider specifies (normally you'd do less with less denoising). from Settings -> Stable Diffusion to be able to do more sampling steps in img2img, it will automatically reduce it if it's not enabled. DPM++ 2M Karras does only 11 steps with 25 denoise if this is not enabled.

But i gotta say generated images starting to look pretty insane. This is best i have managed to make so far in terms of realistic images.

https://image.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/0de93fdb-34ac-40eb-9580-13e13479f55c/width=1280/00511--1013420585-aZovyaPhotoreal_v1Ultra-DPM++%202M%20Karras.jpeg

[PotatoBananaApple]

The tutorial was updated with Region prompt control :)

[crazygonuts]

What exactly does noise inversion and its various parameters do? From your description and the name it seems that it negates some amount of noise? How is that different from simply turning down the denoising value?

[crazygonuts]

@PotatoBananaApple @pkuliyi2015 I tried doing some generations with noise inversion vs not and I am just not sure what I am looking for / should be expecting.

[freezerain]

Consider edit your tutorial as you recomend 80 overlap with 100 tile, thats insanly high and insanly long with no benefit at all, I use 16-24-32 pixels overlap for each 100 tile at most in both this extension and ultimate sd upscaler

[PotatoBananaApple]

It is a bit outdated, with controlnet tile you barely need any overlap anymore. Have not had much time lately, will eventually do better one with controlnet tile.