v4.2.8

v4.2.8 brings Prompt Templates to Invoke, new schedulers and a number of minor fixes and enhancements.

Prompt Templates

Prompt templates are often used for commonly-used style keywords, letting you focus on subject and composition in your prompts - but you can use them in other creative ways.

Thanks to @maryhipp for implementing Prompt Templates!

Creating a Prompt Template

Create a prompt template from an existing image generated with Invoke. We'll add the positive and negative prompts from the image's metadata as the template, and the image will be used as a cover image for the template.

Screen.Recording.2024-08-22.at.7.43.19.pm.mov

You can also create a prompt template from scratch, uploading a cover image.

Screen.Recording.2024-08-22.at.7.55.43.pm.mov

How it Works

Add a positive and/or negative prompt to your template. Use the {prompt} placeholder in the template to indicate where your prompt should be inserted into the template:

• Template: highly detailed photo of {prompt}, award-winning, nikon dslr
• Prompt: a super cute fennec fox cub
• Result: highly detailed photo of a super cute fennec fox cub, award-winning, nikon dslr

If you omit the placeholder, the template will be appended to the end of your prompt:

• Template: turtles
• Prompt: i like
• Result: i like turtles

Default Prompt Templates

We're shipping a number of templates with the app, many of which were contributed by community members (thanks y'all!). We'll update these as we continue developing Invoke with improvements and new templates.

Screen.Recording.2024-08-22.at.8.06.45.pm.mov

Import and Export

You can import templates from other SD apps. We support CSV and JSON files with these columns/keys:

• name
• prompt or positive_prompt
• negative_prompt

Export your prompt templates to share with others. When you export prompt templates, only your own templates are exported.

Screen.Recording.2024-08-22.at.8.10.46.pm.mov

Preview and Flatten

Use the Preview button to see the prompt that will be used for generation. Flatten the prompt template to bake it into your prompts.

Screen.Recording.2024-08-22.at.8.14.38.pm.mov

Compatible with Dynamic Prompts

You can use dynamic prompt in prompt templates, and they will work with dynamic prompts in your positive prompt box.

Screen.Recording.2024-08-22.at.8.26.29.pm.mov

Other Changes

• Enhancement: Added DPM++ 3M, DPM++ 3M Karras, DEIS Karras, KDPM 2 Karras, KDPM 2 Ancestral Karras and UniPC Karras schedulers @StAlKeR7779
• Enhancement: Updated translations - Italian is 100%! Thanks @Harvester62!
• Enhancement: Grounded SAM node (text prompt image segmentation) @RyanJDick
• Enhancement: Update DepthAnything to V2 (small variant only) @blessedcoolant
• Fix: Image downloads with correct filename
• Fix: Delays with events (progress images will be smoother)
• Fix: Jank with board selection when hiding or deleting boards
• Fix: Error deleting images on systems without a "trash bin"
• Fix: Upscale metadata included in SDXL Multidiffusion upscales @maryhipp
• Fix: invoke.sh works with symlinks @max-maag
• Internal: Continued work on the modular backend refactor @StAlKeR7779

Installation and Updating

To install or update to v4.2.8, download the installer and follow the installation instructions.

To update, select the same installation location. Your user data (images, models, etc) will be retained.

Missing models after updating from v3 to v4

See this FAQ.

Error during installation ModuleNotFoundError: No module named 'controlnet_aux'

See this FAQ

What's Changed

• Modular backend - Seamless by @StAlKeR7779 in #6651
• Modular backend - T2I Adapter by @StAlKeR7779 in #6662
• fix(ui): deleting a board or hiding archived should reset selected & auto-add boards by @psychedelicious in #6694
• Modular backend - inpaint by @StAlKeR7779 in #6643
• update delete board modal to be more descriptive by @chainchompa in #6690
• Add more karras schedulers by @StAlKeR7779 in #6695
• Fix gradient mask values range by @StAlKeR7779 in #6688
• Modular backend - LoRA/LyCORIS by @StAlKeR7779 in #6667
• add base prop for destination to direct users to different tabs on initial load by @chainchompa in #6706
• Add Grounded SAM support (text prompt image segmentation) by @RyanJDick in #6701
• fix(ui): include upscale metadata for SDXL multidiffusion by @maryhipp in #6704
• feat(app): delete images without send2trash by @psychedelicious in #6713
• build: exclude matplotlib 3.9.1 by @psychedelicious in #6718
• feat(ui): restore optional upper limit on upscale resolution size by @maryhipp in #6721
• ui: translations update from weblate by @weblate in #6684
• Depth Anything V2 by @blessedcoolant in #6674
• LyCORIS - LoKR and Norm layers by @StAlKeR7779 in #6724
• fix(ui): show warning for maxUpscaleDimension if model tab is disabled by @maryhipp in #6733
• chore(ui): bump deps by @psychedelicious in #6734
• fix(api): image downloads with correct filename by @psychedelicious in #6735
• fix(app): delays in events by @psychedelicious in #6716
• fix(ui): combobox scroll by @psychedelicious in #6738
• feat(ui, api): prompt templates by @maryhipp in #6729
• ui: translations update from weblate by @weblate in #6727
• feat(api,ui): import prompt template CSV by @maryhipp in #6743
• feat: add base prop for selectedWorkflow to allow loading a workflow on launch by @chainchompa in #6742
• ui: translations update from weblate by @weblate in #6746
• feat(ui, api): prompt template export by @maryhipp in #6745
• Fix invoke.sh not detecting symlinks by @max-maag in #6731
• chore: bump version to v4.2.8rc1 by @psychedelicious in #6748
• fix: bundle style preset defaults, bump version to v4.2.8rc2 by @psychedelicious in #6750
• fix(ui): use translations for style preset strings by @psychedelicious in #6752
• fix(ui): prompt template preset preview out of order by @psychedelicious in #6754
• some style preset follow-ups by @maryhipp in #6761
• fix(ui): clear prompt template when prompts are recalled by @maryhipp in #6763
• fix(ui): more prompt template follow-ups by @maryhipp in #6762
• fix(ui): translation fixes by @psychedelicious in #6758
• ui: translations update from weblate by @weblate in #6747
• chore: bump version to v4.2.8 by @psychedelicious in #6767

New Contributors

• @max-maag made their first contribution in #6731

Full Changelog: v4.2.7post1...v4.2.8

v4.2.8rc2

v4.2.8rc2 brings Prompt Templates to Invoke, plus a number of minor fixes and enhancements.

This second RC fixes an issue where the default prompt templates were not packaged correctly, causing an error on startup.

Prompt Templates

We've added the ability to create, import and export prompt templates. These are saved prompts that you may add to your existing prompt.

How it Works

Add a positive and/or negative prompt to your template. Use the {prompt} placeholder in the template to indicate where your prompt should be inserted into the template:

• Template: highly detailed photo of {prompt}, award-winning, nikon dslr
• Prompt: a super cute fennec fox cub
• Result: highly detailed photo of a super cute fennec fox cub, award-winning, nikon dslr

If you omit the placeholder, the template will be appended to the end of your prompt:

• Template: turtles
• Prompt: i like
• Result: i like turtles

Creating a Prompt Template

You can create a prompt templates from within Invoke in two ways:

• Directly, by providing the name, positive prompt and negative prompt. You can upload an image to be the preview image for the template.
• Via metadata from an image generated with Invoke. We'll use the positive and negative prompts from the image's metadata, and that image will be the preview image for that template.

Default Prompt Templates

We're shipping a number of templates with the app. We'll update these as we continue developing Invoke with improvements and new templates.

Import and Export

You can import templates from other SD apps. We support CSV and JSON files with these columns/keys:

• name
• prompt or positive_prompt
• negative_prompt

Export your prompt templates to share with others. When you export prompt templates, only your own templates are exported.

Preview and Flatten

Use the Preview button to see the prompt that will be used for generation. Flatten the prompt template to bake it into your prompts.

Thanks to @maryhipp for implementing Prompt Templates!

Other Changes

• Enhancement: Added DPM++ 3M, DPM++ 3M Karras, DEIS Karras, KDPM 2 Karras, KDPM 2 Ancestral Karras and UniPC Karras schedulers @StAlKeR7779
• Enhancement: Updated translations - Italian is 100%! Thanks @Harvester62!
• Enhancement: Grounded SAM node (text prompt image segmentation) @RyanJDick
• Enhancement: Update DepthAnything to V2 (small variant only) @blessedcoolant
• Fix: Image downloads with correct filename
• Fix: Delays with events (progress images will be smoother)
• Fix: Jank with board selection when hiding or deleting boards
• Fix: Error deleting images on systems without a "trash bin"
• Fix: Upscale metadata included in SDXL Multidiffusion upscales @maryhipp
• Fix: invoke.sh works with symlinks @max-maag
• Internal: Continued work on the modular backend refactor @StAlKeR7779

Installation and Updating

To install or update to v4.2.8rc2, download the installer and follow the installation instructions.

To update, select the same installation location. Your user data (images, models, etc) will be retained.

Missing models after updating from v3 to v4

See this FAQ.

Error during installation ModuleNotFoundError: No module named 'controlnet_aux'

See this FAQ

What's Changed

• Modular backend - Seamless by @StAlKeR7779 in #6651
• Modular backend - T2I Adapter by @StAlKeR7779 in #6662
• fix(ui): deleting a board or hiding archived should reset selected & auto-add boards by @psychedelicious in #6694
• Modular backend - inpaint by @StAlKeR7779 in #6643
• update delete board modal to be more descriptive by @chainchompa in #6690
• Add more karras schedulers by @StAlKeR7779 in #6695
• Fix gradient mask values range by @StAlKeR7779 in #6688
• Modular backend - LoRA/LyCORIS by @StAlKeR7779 in #6667
• add base prop for destination to direct users to different tabs on initial load by @chainchompa in #6706
• Add Grounded SAM support (text prompt image segmentation) by @RyanJDick in #6701
• fix(ui): include upscale metadata for SDXL multidiffusion by @maryhipp in #6704
• feat(app): delete images without send2trash by @psychedelicious in #6713
• build: exclude matplotlib 3.9.1 by @psychedelicious in #6718
• feat(ui): restore optional upper limit on upscale resolution size by @maryhipp in #6721
• ui: translations update from weblate by @weblate in #6684
• Depth Anything V2 by @blessedcoolant in #6674
• LyCORIS - LoKR and Norm layers by @StAlKeR7779 in #6724
• fix(ui): show warning for maxUpscaleDimension if model tab is disabled by @maryhipp in #6733
• chore(ui): bump deps by @psychedelicious in #6734
• fix(api): image downloads with correct filename by @psychedelicious in #6735
• fix(app): delays in events by @psychedelicious in #6716
• fix(ui): combobox scroll by @psychedelicious in #6738
• feat(ui, api): prompt templates by @maryhipp in #6729
• ui: translations update from weblate by @weblate in #6727
• feat(api,ui): import prompt template CSV by @maryhipp in #6743
• feat: add base prop for selectedWorkflow to allow loading a workflow on launch by @chainchompa in #6742
• ui: translations update from weblate by @weblate in #6746
• feat(ui, api): prompt template export by @maryhipp in #6745
• Fix invoke.sh not detecting symlinks by @max-maag in #6731
• chore: bump version to v4.2.8rc1 by @psychedelicious in #6748
• fix: bundle style preset defaults, bump version to v4.2.8rc2 by @psychedelicious in #6750

New Contributors

• @max-maag made their first contribution in #6731

Full Changelog: v4.2.7...v4.2.8rc2

v4.2.8rc1

v4.2.8rc1 brings Prompt Templates to Invoke, plus a number of minor fixes and enhancements.

Prompt Templates

We've added the ability to create, import and export prompt templates. These are saved prompts that you may add to your existing prompt.

How it Works

Add a positive and/or negative prompt to your template. Use the {prompt} placeholder in the template to indicate where your prompt should be inserted into the template:

• Template: highly detailed photo of {prompt}, award-winning, nikon dslr
• Prompt: a super cute fennec fox cub
• Result: highly detailed photo of a super cute fennec fox cub, award-winning, nikon dslr

If you omit the placeholder, the template will be appended to the end of your prompt:

• Template: turtles
• Prompt: i like
• Result: i like turtles

Creating a Prompt Template

You can create a prompt templates from within Invoke in two ways:

• Directly, by providing the name, positive prompt and negative prompt. You can upload an image to be the preview image for the template.
• Via metadata from an image generated with Invoke. We'll use the positive and negative prompts from the image's metadata, and that image will be the preview image for that template.

Default Prompt Templates

We're shipping a number of templates with the app. We'll update these as we continue developing Invoke with improvements and new templates.

Import and Export

You can import templates from other SD apps. We support CSV and JSON files with these columns/keys:

• name
• prompt or positive_prompt
• negative_prompt

Export your prompt templates to share with others. When you export prompt templates, only your own templates are exported.

Preview and Flatten

Use the Preview button to see the prompt that will be used for generation. Flatten the prompt template to bake it into your prompts.

Thanks to @maryhipp for implementing Prompt Templates!

Other Changes

• Enhancement: Added DPM++ 3M, DPM++ 3M Karras, DEIS Karras, KDPM 2 Karras, KDPM 2 Ancestral Karras and UniPC Karras schedulers @StAlKeR7779
• Enhancement: Updated translations - Italian is 100%! Thanks @Harvester62!
• Enhancement: Grounded SAM node (text prompt image segmentation) @RyanJDick
• Enhancement: Update DepthAnything to V2 (small variant only) @blessedcoolant
• Fix: Image downloads with correct filename
• Fix: Delays with events (progress images will be smoother)
• Fix: Jank with board selection when hiding or deleting boards
• Fix: Error deleting images on systems without a "trash bin"
• Fix: Upscale metadata included in SDXL Multidiffusion upscales @maryhipp
• Fix: invoke.sh works with symlinks @max-maag
• Internal: Continued work on the modular backend refactor @StAlKeR7779

Installation and Updating

To install or update to v4.2.8rc1, download the installer and follow the installation instructions.

To update, select the same installation location. Your user data (images, models, etc) will be retained.

Missing models after updating from v3 to v4

See this FAQ.

Error during installation ModuleNotFoundError: No module named 'controlnet_aux'

See this FAQ

What's Changed

• Modular backend - Seamless by @StAlKeR7779 in #6651
• Modular backend - T2I Adapter by @StAlKeR7779 in #6662
• fix(ui): deleting a board or hiding archived should reset selected & auto-add boards by @psychedelicious in #6694
• Modular backend - inpaint by @StAlKeR7779 in #6643
• update delete board modal to be more descriptive by @chainchompa in #6690
• Add more karras schedulers by @StAlKeR7779 in #6695
• Fix gradient mask values range by @StAlKeR7779 in #6688
• Modular backend - LoRA/LyCORIS by @StAlKeR7779 in #6667
• add base prop for destination to direct users to different tabs on initial load by @chainchompa in #6706
• Add Grounded SAM support (text prompt image segmentation) by @RyanJDick in #6701
• fix(ui): include upscale metadata for SDXL multidiffusion by @maryhipp in #6704
• feat(app): delete images without send2trash by @psychedelicious in #6713
• build: exclude matplotlib 3.9.1 by @psychedelicious in #6718
• feat(ui): restore optional upper limit on upscale resolution size by @maryhipp in #6721
• ui: translations update from weblate by @weblate in #6684
• Depth Anything V2 by @blessedcoolant in #6674
• LyCORIS - LoKR and Norm layers by @StAlKeR7779 in #6724
• fix(ui): show warning for maxUpscaleDimension if model tab is disabled by @maryhipp in #6733
• chore(ui): bump deps by @psychedelicious in #6734
• fix(api): image downloads with correct filename by @psychedelicious in #6735
• fix(app): delays in events by @psychedelicious in #6716
• fix(ui): combobox scroll by @psychedelicious in #6738
• feat(ui, api): prompt templates by @maryhipp in #6729
• ui: translations update from weblate by @weblate in #6727
• feat(api,ui): import prompt template CSV by @maryhipp in #6743
• feat: add base prop for selectedWorkflow to allow loading a workflow on launch by @chainchompa in #6742
• ui: translations update from weblate by @weblate in #6746
• feat(ui, api): prompt template export by @maryhipp in #6745
• Fix invoke.sh not detecting symlinks by @max-maag in #6731
• chore: bump version to v4.2.8rc1 by @psychedelicious in #6748

New Contributors

• @max-maag made their first contribution in #6731

Full Changelog: v4.2.7...v4.2.8rc1

v4.2.7post1

🚨 v4.2.7post1 resolves an issue with Windows installs. 🚨

v4.2.7 includes gallery improvements and some major features focused on upscaling.

Upscaling

We've added a dedicated upscaling tab, support for custom upscaling models, and some new nodes.

Thanks to @RyanJDick (backend implementation), @chainchompa (frontend) and @maryhipp (frontend) for working on this!

Dedicated Upscaling Tab

The new upscaling tab provides a simple and powerful UI to Invoke's MultiDiffusion implementation. This builds on the workflow released in v4.2.6, allowing for memory-efficient upscaling to huge output image sizes.

Upscaling.Tab.mov

We're pretty happy with the results!

4x scale, 4x_NMKD-Siax_200k upscale model, Deliberate_v5 SD1.5 model, KDPM 2 scheduler @ 30 steps, all other settings default

Requirements

You need 3 models installed to use this feature:

• An upscale model for the first pass upscale
• A main SD model (SD1.5 or SDXL) for the image-to-image
• A tile ControlNet model of the same model architecture as your main SD model

If you are missing any of these, you'll see a warning directing you to the model manager to install them. You can search the starter models for upscale, main, and tile to get you started.

Tips

• The main SD model architecture has the biggest impact on VRAM usage. For example, SD1.5 @ 2k needs just under 4GB, while SDXL @ 2k needs just under 9GB. VRAM usage increases a small amount as output size increases - SD1.5 @ 8k needs ~4.5GB while SDXL @ 8k needs ~10.5GB.
• The upscale and main SD model choices matter. Choose models best suited to your input image or desired output characteristics.
• Some schedulers work better than others. KDPM 2 is a good choice.
• LoRAs - like a detail-adding LoRA - can make a big impact.
• Higher Creativity values give the SD model more leeway in creating new details. This parameter controls denoising start and end percentages.
• Higher Structure values tell the SD model to stick closer to the input image's structure. This parameter controls the tile ControlNet.

Custom Upscaling Models

You can now install and use custom upscaling models in Invoke. The excellent spandrel library handles loading and running the models.

Custom.Upscaling.Models.mov

spandrel can do a lot more than upscaling - it supports a wide range of "image to image" models. This includes single-image super resolution like ESRGAN (upscalers) but also things like GFPGAN (face restoration) and DeJPEG (cleans up JPEG compression artifacts).

A complete list of supported architectures can be found here.

Note: We have not enabled the restrictively-licensed architectures, which are denoted with a + symbol in the list.

Installing Models

We've added a few popular upscaling models to the Starter Models tab in the Model Manager - search for "upscale" to find them.

You can install models found online via the Model Manager, just like any other model. OpenModelDB is a popular place to get these models. For most of them, you can copy the model's download link and paste in into the Model Manager to install.

Nodes

Two nodes have been added to support processing images with spandrel - be that upscaling or any of the other tasks these models support.

• Image-to-Image - Runs the selected model without any extra processing.
• Image-to-Image (Autoscale) - Runs the selected model repeatedly until the desired scale is reached. This node is intended for upscaling models specifically, providing some useful extra functionality:
  • If the model overshoots the target scale, the final image will be downscaled to the target scale with Lanczos resampling.
  • As a convenience, the output image width and height can be fit to a multiple of 8, as is required for SD. This will only resize down, and may change the aspect ratio slightly.
  • If the model doesn't actually upscale the image, the scale parameter will be ignored.

Gallery Improvements

Thanks to @maryhipp and @chainchompa for continued iteration on the gallery!

• Cleaner boards UI.
• Improved boards and image search UI.
• Fixed issues where board counts don't update when images are moved between boards.
• Added a "Jump" button to allow you to skip pages of the gallery

Gallery_Jump_Example.mp4

Other Changes

• Enhancement: When installing starter models, the description is carried over. Thanks @lstein!
• Enhancement: Updated translations.
• Fix: Model unpatching when running on CPU, causing bad/no outputs.
• Fix: Occasional visible seams on images with smooth textures, like skies. MultiDiffusion tiling now uses gradient blending to mitigate this issue.
• Fix: Model names overflow the model selection drop-downs.
• Internal: Backend SD pipeline refactor (WIP). This will allow contributors to add functionality to Invoke more easily. This will be behind a feature flag until the refactor is complete and tested. Thanks to @StAlKeR7779 for leading the effort, with major contributions from @dunkeroni and @RyanJDick.

Installation and Updating

To install or update to v4.2.7post1, download the installer and follow the installation instructions.

To update, select the same installation location. Your user data (images, models, etc) will be retained.

Missing models after updating from v3 to v4

See this FAQ.

Error during installation ModuleNotFoundError: No module named 'controlnet_aux'

See this FAQ

What's Changed

• fix(backend): revert non-blocking device transfer by @psychedelicious in #6624
• chore: bump version to 4.2.6post1 by @psychedelicious in #6625
• Add support for Spandrel Image-to-Image models (e.g. ESRGAN, Real-ESRGAN, Swin-IR, DAT, etc.) by @RyanJDick in #6556
• Add tiling to SpandrelImageToImageInvocation by @RyanJDick in #6594
• Add Spandrel upscale starter models by @RyanJDick in #6605
• Fix model unpatching process when running on CPU by @lstein in #6631
• Add gradient blending to tile seams in MultiDiffusion by @RyanJDick in #6635
• Base of modular backend by @StAlKeR7779 in #6606
• add sdxl tile to starter models by @maryhipp in #6634
• Fix function call that we forgot to update in #6606 by @RyanJDick in #6636
• Bump fastapi-events dependency by @ebr in #6644
• fix: update uncategorized board totals when deleting and moving images by @chainchompa in #6646
• feat(ui): upscaling tab by @maryhipp in #6633
• Math Updates by @hipsterusername in #6648
• Modular backend - add rescale cfg by @StAlKeR7779 in #6640
• Modular backend - add FreeU by @StAlKeR7779 in #6641
• Modular backend - add ControlNet by @StAlKeR7779 in #6642
• feat(ui): add upsells for pro edition to settings menu by @maryhipp in #6650
• Update Simple Upscale Button to work with spandrel models by @chainchompa in #6649
• Simple upscale bugfixes by @chainchompa in #6655
• fix(ui): settings menu layout by @psychedelicious in #6654
• [MM2] Use typed ModelRecordChanges for model_install() rather than untyped dict by @lstein in #6645
• fix(ui): restore pnpm-lock.yaml by @psychedelicious in #6659
• feat(ui): upscaling & ad-hoc post-processing misc by @psychedelicious in #6658
• fix(ui): model select overflowing when model names are too long by @psychedelicious in #6660
• feat(ui): more gallery UX updates by @maryhipp in #6652
• fix(ui): few cases where board totals don't updated when moving by @psychedelicious in #6665
• ui: translations update from weblate by @weblate in #6653
• chore: bump version v...

v4.2.7

v4.2.7 includes gallery improvements and some major features focused on upscaling.

Upscaling

We've added a dedicated upscaling tab, support for custom upscaling models, and some new nodes.

Thanks to @RyanJDick (backend implementation), @chainchompa (frontend) and @maryhipp (frontend) for working on this!

Dedicated Upscaling Tab

The new upscaling tab provides a simple and powerful UI to Invoke's MultiDiffusion implementation. This builds on the workflow released in v4.2.6, allowing for memory-efficient upscaling to huge output image sizes.

Upscaling.Tab.mov

We're pretty happy with the results!

4x scale, 4x_NMKD-Siax_200k upscale model, Deliberate_v5 SD1.5 model, KDPM 2 scheduler @ 30 steps, all other settings default

Requirements

You need 3 models installed to use this feature:

• An upscale model for the first pass upscale
• A main SD model (SD1.5 or SDXL) for the image-to-image
• A tile ControlNet model of the same model architecture as your main SD model

If you are missing any of these, you'll see a warning directing you to the model manager to install them. You can search the starter models for upscale, main, and tile to get you started.

Tips

• The main SD model architecture has the biggest impact on VRAM usage. For example, SD1.5 @ 2k needs just under 4GB, while SDXL @ 2k needs just under 9GB. VRAM usage increases a small amount as output size increases - SD1.5 @ 8k needs ~4.5GB while SDXL @ 8k needs ~10.5GB.
• The upscale and main SD model choices matter. Choose models best suited to your input image or desired output characteristics.
• Some schedulers work better than others. KDPM 2 is a good choice.
• LoRAs - like a detail-adding LoRA - can make a big impact.
• Higher Creativity values give the SD model more leeway in creating new details. This parameter controls denoising start and end percentages.
• Higher Structure values tell the SD model to stick closer to the input image's structure. This parameter controls the tile ControlNet.

Custom Upscaling Models

You can now install and use custom upscaling models in Invoke. The excellent spandrel library handles loading and running the models.

Custom.Upscaling.Models.mov

spandrel can do a lot more than upscaling - it supports a wide range of "image to image" models. This includes single-image super resolution like ESRGAN (upscalers) but also things like GFPGAN (face restoration) and DeJPEG (cleans up JPEG compression artifacts).

A complete list of supported architectures can be found here.

Note: We have not enabled the restrictively-licensed architectures, which are denoted with a + symbol in the list.

Installing Models

We've added a few popular upscaling models to the Starter Models tab in the Model Manager - search for "upscale" to find them.

You can install models found online via the Model Manager, just like any other model. OpenModelDB is a popular place to get these models. For most of them, you can copy the model's download link and paste in into the Model Manager to install.

Nodes

Two nodes have been added to support processing images with spandrel - be that upscaling or any of the other tasks these models support.

• Image-to-Image - Runs the selected model without any extra processing.
• Image-to-Image (Autoscale) - Runs the selected model repeatedly until the desired scale is reached. This node is intended for upscaling models specifically, providing some useful extra functionality:
  • If the model overshoots the target scale, the final image will be downscaled to the target scale with Lanczos resampling.
  • As a convenience, the output image width and height can be fit to a multiple of 8, as is required for SD. This will only resize down, and may change the aspect ratio slightly.
  • If the model doesn't actually upscale the image, the scale parameter will be ignored.

Gallery Improvements

Thanks to @maryhipp and @chainchompa for continued iteration on the gallery!

• Cleaner boards UI.
• Improved boards and image search UI.
• Fixed issues where board counts don't update when images are moved between boards.
• Added a "Jump" button to allow you to skip pages of the gallery

Gallery_Jump_Example.mp4

Other Changes

• Enhancement: When installing starter models, the description is carried over. Thanks @lstein!
• Enhancement: Updated translations.
• Fix: Model unpatching when running on CPU, causing bad/no outputs.
• Fix: Occasional visible seams on images with smooth textures, like skies. MultiDiffusion tiling now uses gradient blending to mitigate this issue.
• Fix: Model names overflow the model selection drop-downs.
• Internal: Backend SD pipeline refactor (WIP). This will allow contributors to add functionality to Invoke more easily. This will be behind a feature flag until the refactor is complete and tested. Thanks to @StAlKeR7779 for leading the effort, with major contributions from @dunkeroni and @RyanJDick.

Installation and Updating

To install or update to v4.2.7, download the installer and follow the installation instructions.

To update, select the same installation location. Your user data (images, models, etc) will be retained.

Missing models after updating from v3 to v4

See this FAQ.

Error during installation ModuleNotFoundError: No module named 'controlnet_aux'

See this FAQ

What's Changed

• Add support for Spandrel Image-to-Image models (e.g. ESRGAN, Real-ESRGAN, Swin-IR, DAT, etc.) by @RyanJDick in #6556
• Add tiling to SpandrelImageToImageInvocation by @RyanJDick in #6594
• Add Spandrel upscale starter models by @RyanJDick in #6605
• Fix model unpatching process when running on CPU by @lstein in #6631
• Add gradient blending to tile seams in MultiDiffusion by @RyanJDick in #6635
• Base of modular backend by @StAlKeR7779 in #6606
• add sdxl tile to starter models by @maryhipp in #6634
• Fix function call that we forgot to update in #6606 by @RyanJDick in #6636
• Bump fastapi-events dependency by @ebr in #6644
• fix: update uncategorized board totals when deleting and moving images by @chainchompa in #6646
• feat(ui): upscaling tab by @maryhipp in #6633
• Math Updates by @hipsterusername in #6648
• Modular backend - add rescale cfg by @StAlKeR7779 in #6640
• Modular backend - add FreeU by @StAlKeR7779 in #6641
• Modular backend - add ControlNet by @StAlKeR7779 in #6642
• feat(ui): add upsells for pro edition to settings menu by @maryhipp in #6650
• Update Simple Upscale Button to work with spandrel models by @chainchompa in #6649
• Simple upscale bugfixes by @chainchompa in #6655
• fix(ui): settings menu layout by @psychedelicious in #6654
• [MM2] Use typed ModelRecordChanges for model_install() rather than untyped dict by @lstein in #6645
• fix(ui): restore pnpm-lock.yaml by @psychedelicious in #6659
• feat(ui): upscaling & ad-hoc post-processing misc by @psychedelicious in #6658
• fix(ui): model select overflowing when model names are too long by @psychedelicious in #6660
• feat(ui): more gallery UX updates by @maryhipp in #6652
• fix(ui): few cases where board totals don't updated when moving by @psychedelicious in #6665
• ui: translations update from weblate by @weblate in #6653
• chore: bump version v4.2.7rc1 by @psychedelicious in #6670
• feat(ui): add jump to option for gallery pagination by @maryhipp in #6668
• feat(ui): info popovers on by default by @psychedelicious in #66...

v4.2.7rc1

v4.2.7rc1 includes gallery improvements and some major features focused on upscaling.

Upscaling

We've added a dedicated upscaling tab, support for custom upscaling models, and some new nodes.

Thanks to @RyanJDick (backend implementation), @chainchompa (frontend) and @maryhipp (frontend) for working on this!

Dedicated Upscaling Tab

The new upscaling tab provides a simple and powerful UI to Invoke's MultiDiffusion implementation. This builds on the workflow released in v4.2.6, allowing for memory-efficient upscaling to huge output image sizes.

Upscaling.Tab.mov

We're pretty happy with the results!

4x scale, 4x_NMKD-Siax_200k upscale model, Deliberate_v5 SD1.5 model, KDPM 2 scheduler @ 30 steps, all other settings default

Requirements

You need 3 models installed to use this feature:

• An upscale model for the first pass upscale
• A main SD model (SD1.5 or SDXL) for the image-to-image
• A tile ControlNet model of the same model architecture as your main SD model

If you are missing any of these, you'll see a warning directing you to the model manager to install them. You can search the starter models for upscale, main, and tile to get you started.

Tips

• The main SD model architecture has the biggest impact on VRAM usage. For example, SD1.5 @ 2k needs just under 4GB, while SDXL @ 2k needs just under 9GB. VRAM usage increases a small amount as output size increases - SD1.5 @ 8k needs ~4.5GB while SDXL @ 8k needs ~10.5GB.
• The upscale and main SD model choices matter. Choose models best suited to your input image or desired output characteristics.
• Some schedulers work better than others. KDPM 2 is a good choice.
• LoRAs - like a detail-adding LoRA - can make a big impact.
• Higher Creativity values give the SD model more leeway in creating new details. This parameter controls denoising start and end percentages.
• Higher Structure values tell the SD model to stick closer to the input image's structure. This parameter controls the tile ControlNet.

Custom Upscaling Models

You can now install and use custom upscaling models in Invoke. The excellent spandrel library handles loading and running the models.

Custom.Upscaling.Models.mov

spandrel can do a lot more than upscaling - it supports a wide range of "image to image" models. This includes single-image super resolution like ESRGAN (upscalers) but also things like GFPGAN (face restoration) and DeJPEG (cleans up JPEG compression artifacts).

A complete list of supported architectures can be found here.

Note: We have not enabled the restrictively-licensed architectures, which are denoted with a + symbol in the list.

Installing Models

We've added a few popular upscaling models to the Starter Models tab in the Model Manager - search for "upscale" to find them.

You can install models found online via the Model Manager, just like any other model. OpenModelDB is a popular place to get these models. For most of them, you can copy the model's download link and paste in into the Model Manager to install.

Nodes

Two nodes have been added to support processing images with spandrel - be that upscaling or any of the other tasks these models support.

• Image-to-Image - Runs the selected model without any extra processing.
• Image-to-Image (Autoscale) - Runs the selected model repeatedly until the desired scale is reached. This node is intended for upscaling models specifically, providing some useful extra functionality:
  • If the model overshoots the target scale, the final image will be downscaled to the target scale with Lanczos resampling.
  • As a convenience, the output image width and height can be fit to a multiple of 8, as is required for SD. This will only resize down, and may change the aspect ratio slightly.
  • If the model doesn't actually upscale the image, the scale parameter will be ignored.

Gallery Improvements

Thanks to @maryhipp and @chainchompa for continued iteration on the gallery!

• Cleaner boards UI.
• Improved boards and image search UI.
• Fixed issues where board counts don't update when images are moved between boards.

Other Changes

• Enhancement: When installing starter models, the description is carried over. Thanks @lstein!
• Enhancement: Updated translations.
• Fix: Model unpatching when running on CPU, causing bad/no outputs.
• Fix: Occasional visible seams on images with smooth textures, like skies. MultiDiffusion tiling now uses gradient blending to mitigate this issue.
• Fix: Model names overflow the model selection drop-downs.
• Internal: Backend SD pipeline refactor (WIP). This will allow contributors to add functionality to Invoke more easily. This will be behind a feature flag until the refactor is complete and tested. Thanks to @StAlKeR7779 for leading the effort, with major contributions from @dunkeroni and @RyanJDick.

Installation and Updating

To install or update to v4.2.7rc1, download the installer and follow the installation instructions.

To update, select the same installation location. Your user data (images, models, etc) will be retained.

Missing models after updating from v3 to v4

See this FAQ.

Error during installation ModuleNotFoundError: No module named 'controlnet_aux'

See this FAQ

What's Changed

• Add support for Spandrel Image-to-Image models (e.g. ESRGAN, Real-ESRGAN, Swin-IR, DAT, etc.) by @RyanJDick in #6556
• Add tiling to SpandrelImageToImageInvocation by @RyanJDick in #6594
• Add Spandrel upscale starter models by @RyanJDick in #6605
• Fix model unpatching process when running on CPU by @lstein in #6631
• Add gradient blending to tile seams in MultiDiffusion by @RyanJDick in #6635
• Base of modular backend by @StAlKeR7779 in #6606
• add sdxl tile to starter models by @maryhipp in #6634
• Fix function call that we forgot to update in #6606 by @RyanJDick in #6636
• Bump fastapi-events dependency by @ebr in #6644
• fix: update uncategorized board totals when deleting and moving images by @chainchompa in #6646
• feat(ui): upscaling tab by @maryhipp in #6633
• Math Updates by @hipsterusername in #6648
• Modular backend - add rescale cfg by @StAlKeR7779 in #6640
• Modular backend - add FreeU by @StAlKeR7779 in #6641
• Modular backend - add ControlNet by @StAlKeR7779 in #6642
• feat(ui): add upsells for pro edition to settings menu by @maryhipp in #6650
• Update Simple Upscale Button to work with spandrel models by @chainchompa in #6649
• Simple upscale bugfixes by @chainchompa in #6655
• fix(ui): settings menu layout by @psychedelicious in #6654
• [MM2] Use typed ModelRecordChanges for model_install() rather than untyped dict by @lstein in #6645
• fix(ui): restore pnpm-lock.yaml by @psychedelicious in #6659
• feat(ui): upscaling & ad-hoc post-processing misc by @psychedelicious in #6658
• fix(ui): model select overflowing when model names are too long by @psychedelicious in #6660
• feat(ui): more gallery UX updates by @maryhipp in #6652
• fix(ui): few cases where board totals don't updated when moving by @psychedelicious in #6665
• ui: translations update from weblate by @weblate in #6653
• chore: bump version v4.2.7rc1 by @psychedelicious in #6670

Full Changelog: v4.2.6post1...v4.2.7rc1

v4.2.6post1

v4.2.6post1 fixes issues some users may experience with memory management and sporadic black image outputs.

Please see the v4.2.6 release for full release notes.

💾 Installation and Updating

To install or update to v4.2.6post1, download the installer and follow the installation instructions.

To update, select the same installation location. Your user data (images, models, etc) will be retained.

Missing models after updating from v3 to v4

See this FAQ.

Error during installation ModuleNotFoundError: No module named 'controlnet_aux'

See this FAQ

What's Changed

• fix(backend): revert non-blocking device transfer by @psychedelicious in #6624
• chore: bump version to 4.2.6post1 by @psychedelicious in #6625

Full Changelog: v4.2.6...v4.2.6post1

v4.2.6

v4.2.6 includes a handful of fixes and improvements, plus three major changes:

• Gallery updates
• Tiled upscaling via MultiDiffusion
• Checkpoint models work without conversion to diffusers

Gallery Updates

We've made some changes to the gallery, adding features, improving the performance of the app and reducing memory usage. The changes also fix a number of bugs relating to stale data - for example, a board not updating as expected after moving an image to it.

Thanks to @chainchompa and @maryhipp for working on this major effort.

Pagination & Selection

Infinite scroll is dead, long live infinite scroll!

The gallery is now paginated. Selection logic has been updated to work with pagination. An indicator shows how many images are selected and allows you to clear the selection entirely. Arrow keys still navigate.

Gallery.Pagination.and.Selection.mov

The number of images per page is dynamically calculated as the panel is resized, ensuring the panel is always filled with images.

Boards UI Refresh

The bulky tiled boards grid has been replaced by a scrollable list. The boards list panel is now a resizable, collapsible panel.

Boards.List.and.Resizable.Panel.mov

Boards and Image Search

Search for boards by name and images by metadata. The search term is matched against the image's metadata as a string. We landed on full-text search as a flexible yet simple implementation after considering a few methods for search.

Boards.and.Images.Search.mov

Archived Boards

Archive a board to hide it from the main boards list. This is purely an organizational enhancement. You can still interact with archived boards as you would any other board.

Archived.Boards.mov

Image Sorting

You can now change the sort for images to show oldest first. A switch allows starred images to be placed in the list according to their age, instead of always showing them first.

Image.Sorting.mov

Tiled Upscaling via MultiDiffusion

MultiDiffusion is a fairly straightforward technique for tiled denoising. The gist is similar to other tiled upscaling methods - split the input image up in to tiles, process each independently, and stitch them back together. The main innovation for MultiDiffusion is to do this in latent space, blending the tensors together continuously. This results in excellent consistency across the output image, with no seams.

This feature is exposed as a Tiled MultiDiffusion Denoise Latents node, currently classified as a beta version. It works much the same as the OG Denoise Latents node. You can find an example workflow in the workflow library's default workflows.

We are still thinking about to expose this in the linear UI. Most likely, we expose this with very minimal settings. If you want to tweak it, use the workflow.

Thanks to @RyanJDick for designing and implementing MultiDiffusion.

How to use it

This technique is fundamentally the same as normal img2img. Appropriate use of conditioning and control will greatly improve the output. The one hard requirement is to use the Tile ControlNet model.

Besides that, here are some tips from our initial testing:

• Use a detail-adding or style LoRAs.
• Use a base model best suited for the desired output style.
• Prompts make a difference.
• The initial upscaling method makes a difference.
• Scheduler makes a difference. Some produce softer outputs.

VRAM Usage

This technique can upscale images to very large sizes without substantially increasing VRAM usage beyond what you'd see for a "normal" sized generation. The VRAM bottlenecks then become the first VAE encode (Image to Latents) and final VAE decode (Latents to Image) steps.

You may run into OOM errors during these steps. The solution is to enable tiling using the toggle on the Image to Latents and Latents to Image nodes. This allows the VAE operations to be done piecewise, similar to the tiled denoising process, without using gobs of VRAM.

There's one caveat - VAE tiling often introduces inconsistency across tiles. Textures and colors may differ from tile to tile. This is a function of diffusers' handling of VAE tiling, not the new tiled denoising process. We are investigating ways to improve this.

Takeaway: If your GPU can handle non-tiled VAE encode and decode for a given output size, use that for best results.

Checkpoint models work without conversion to diffusers

The required conversion of checkpoint format models to diffusers format has long been a pain point. The diffusers library now supports loading single-file (checkpoint) models directly, and we have removed the mandatory checkpoint-to-diffusers conversion step.

The main user-facing change is that there is no longer a conversion cache directory.

Major thanks to @lstein for getting this working.

📈 Patch Nodes for v4.2.6

Enhancements

• When downloading image metadata, graphs or workflows, the JSON file includes the image name and type of data. Thanks @jstnlowe!
• Add clear_queue_on_startup config setting to clear problematic queues. This is useful for a rare edge case where your queue is full of items that somehow crash the app. Set this to true, and the queue will clear before it has time to attempt to execute the problematic item. Thanks @steffy-lo!
• Performance and memory efficiency improvements for LoRA patching and model offloading.
• Addition of a simplified model installation methods to the Invocation API: download_and_cache_model, load_local_model and load_remote_model. These methods allow models to be used without needing them to be added to the model manager. For example, we are now using these methods to load ESRGAN models.
• Support for probing and loading SDXL VAE checkpoint.
• Updated gallery UI.
• Checkpoint models work without conversion to diffusers.
• When using a VAE in tiled mode, you may now select the tile size.

Fixes

• Fix handling handling of 0-step denoising process.
• If a control image's processed version is missing when the app loads, it is now re-processed.
• Fixed an issue where a model's size could be misreported as 0, possibly causing memory issues.
• Fixed an issue where images - especially large images - may fail to delete.

Performance improvements

• Improved LoRA patching.
• Improved RAM <-> VRAM model transfer performance.

Internal changes

• The DenoiseLatentsInvocation has had its internal methods split up to support tiled upscaling via MultiDiffusion. This included some amount of file shuffling and renaming. The invokeai package's exported classes should still be the same. Please let us know if this has broken an import for you.
• Internal cleanup, intending to eliminate circular import issues. There's a lot left to do for this issue, but we are making progress.

💾 Installation and Updating

To install or update to v4.2.6, download the installer and follow the installation instructions.

To update, select the same installation location. Your user data (images, models, etc) will be retained.

Missing models after updating from v3 to v4

See this FAQ.

Error during installation ModuleNotFoundError: No module named 'controlnet_aux'

See this FAQ

What's Changed

• Prefixed JSON filenames with the image UUID by @jstnlowe in #6486
• feat(ui): control layers internals cleanup by @psychedelicious in #6487
• LoRA patching optimization by @lstein in #6439
• fix(ui): re-process control image if processed image is missing on page load by @psychedelicious in #6494
• Split up latent.py (code reorganization, no functional changes) by @RyanJDick in #6491
• Add simplified model manager install API to InvocationContext by @lstein in #6132
• fix: Some imports from previous PR's by @blessedcoolant in #6501
• Improve RAM<->VRAM memory copy performance in LoRA patching and elsewhere by @lstein in #6490
• Fix DEFAULT_PRECISION handling by @RyanJDick in #6492
• added route to install huggingface models from model marketplace by @chainchompa in #6515
• Model hash validator by @brandonrising in #6520
• Tidy SilenceWarnings context manager by @RyanJDick in #6493
• [#6333] Add clear_queue_on_startup config to clear problematic queues by @steffy-lo in #6502
• [MM] Add support for probing and loading SDXL VAE checkpoint files by @lstein in #6524
• Add TiledMultiDiffusionDenoiseLatents invocation (for upscaling workflows) by @RyanJDick in https://github.com/invoke-ai/InvokeAI/...

v4.2.6rc1

v4.2.6 includes a handful of fixes and improvements, plus three major changes:

• Gallery updates
• Tiled upscaling via MultiDiffusion
• Checkpoint models work without conversion to diffusers

Known Issues

Our last release, v4.2.5, was quickly pulled after a black image issue on MPS (macOS) was discovered. We also had reports of CUDA (Nvidia) GPUs getting unexpected OOM (Out of Memory) errors.

The MPS issue is resolved in this release, but we haven't been able to replicate unexpected OOMs on Linux or Windows. We did fix one issue that may have been a factor.

If you get OOMs on this alpha release with settings that worked fine on v4.2.4 - or have any other issues - please let us know via GH issues or discord.

Gallery Updates

We've made some changes to the gallery, adding features, improving the performance of the app and reducing memory usage. The changes also fix a number of bugs relating to stale data - for example, a board not updating as expected after moving an image to it.

Thanks to @chainchompa and @maryhipp for working on this major effort.

Pagination & Selection

Infinite scroll is dead, long live infinite scroll!

The gallery is now paginated. Selection logic has been updated to work with pagination. An indicator shows how many images are selected and allows you to clear the selection entirely. Arrow keys still navigate.

Gallery.Pagination.and.Selection.mov

The number of images per page is dynamically calculated as the panel is resized, ensuring the panel is always filled with images.

Boards UI Refresh

The bulky tiled boards grid has been replaced by a scrollable list. The boards list panel is now a resizable, collapsible panel.

Boards.List.and.Resizable.Panel.mov

Boards and Image Search

Search for boards by name and images by metadata. The search term is matched against the image's metadata as a string. We landed on full-text search as a flexible yet simple implementation after considering a few methods for search.

Boards.and.Images.Search.mov

Archived Boards

Archive a board to hide it from the main boards list. This is purely an organizational enhancement. You can still interact with archived boards as you would any other board.

Archived.Boards.mov

Image Sorting

You can now change the sort for images to show oldest first. A switch allows starred images to be placed in the list according to their age, instead of always showing them first.

Image.Sorting.mov

Tiled Upscaling via MultiDiffusion

MultiDiffusion is a fairly straightforward technique for tiled denoising. The gist is similar to other tiled upscaling methods - split the input image up in to tiles, process each independently, and stitch them back together. The main innovation for MultiDiffusion is to do this in latent space, blending the tensors together continually. This results in excellent consistency across the output image, with no seams.

This feature is exposed as a Tiled MultiDiffusion Denoise Latents node, currently classified as a beta version. It works much the same as the OG Denoise Latents node. Here's a workflow to get you started: sd15_multi_diffusion_esrgan_x2_upscale.json

We are still thinking about to expose this in the linear UI. Most likely, we expose this with very minimal settings. If you want to tweak it, use the workflow.

Thanks to @RyanJDick for designing and implementing MultiDiffusion.

How to use it

This technique is fundamentally the same as normal img2img. Appropriate use of conditioning and control will greatly improve the output. The one hard requirement is to use the Tile ControlNet model.

Besides that, here are some tips from our initial testing:

• Use a detail-adding or style LoRAs.
• Use a base model best suited for the desired output style.
• Prompts make a difference.
• The initial upscaling method makes a difference.
• Scheduler makes a difference. Some produce softer outputs.

VRAM Usage

This technique can upscale images to very large sizes without substantially increasing VRAM usage beyond what you'd see for a "normal" sized generation. The VRAM bottlenecks then become the first VAE encode (Image to Latents) and final VAE decode (Latents to Image) steps.

You may run into OOM errors during these steps. The solution is to enable tiling using the toggle on the Image to Latents and Latents to Image nodes. This allows the VAE operations to be done piecewise, similar to the tiled denoising process, without using gobs of VRAM.

There's one caveat - VAE tiling often introduces inconsistency across tiles. Textures and colors may differ from tile to tile. This is a function of the diffusers handling of VAE tiling, not the tiled denoising process introduced in v4.2.5. We are investigating ways to improve this.

Takeaway: If your GPU can handle non-tiled VAE encode and decode for a given output size, use that for best results.

Checkpoint models work without conversion to diffusers

The required conversion of checkpoint format models to diffusers format has long been a pain point. Diffusers now supports loading single-file (checkpoint) models directly, and we have removed the mandatory checkpoint-to-diffusers conversion step.

The main user-facing change is that there is no longer a conversion cache directory!

Major thanks to @lstein for getting this working.

📈 Patch Nodes for v4.2.6

Enhancements

• When downloading image metadata, graphs or workflows, the JSON file includes the image name and type of data. Thanks @jstnlowe!
• Add clear_queue_on_startup config setting to clear problematic queues. This is useful for a rare edge case where your queue is full of items that somehow crash the app. Set this to true, and the queue will clear before it has time to attempt to execute the problematic item. Thanks @steffy-lo!
• Performance and memory efficiency improvements for LoRA patching and model offloading.
• Addition of a simplified model installation methods to the Invocation API: download_and_cache_model, load_local_model and load_remote_model. These methods allow models to be used without needing them to be added to the model manager. For example, we are now using these methods to load ESRGAN models.
• Support for probing and loading SDXL VAE checkpoint.
• Updated gallery UI.
• Checkpoint models work without conversion to diffusers.
• When using a VAE in tiled mode, you may now select the tile size.

Fixes

• Fix handling handling of 0-step denoising process.
• If a control image's processed version is missing when the app loads, it is now re-processed.
• Fixed an issue where a model's size could be misreported as 0, possibly causing memory issues.
• Fixed an issue where images - especially large images - may fail to delete.

Performance improvements

• Improved LoRA patching.
• Improved RAM <-> VRAM model transfer performance.

Internal changes

• The DenoiseLatentsInvocation has had its internal methods split up to support tiled upscaling via MultiDiffusion. This included some amount of file shuffling and renaming. The invokeai package's exported classes should still be the same. Please let us know if this has broken an import for you.
• Internal cleanup, intending to eliminate circular import issues. There's a lot left to do for this issue, but we are making progress.

💾 Installation and Updating

To install or update to v4.2.6rc1, download the installer and follow the installation instructions.

To update, select the same installation location. Your user data (images, models, etc) will be retained.

Missing models after updating from v3 to v4

See this FAQ.

Error during installation ModuleNotFoundError: No module named 'controlnet_aux'

See this FAQ

What's Changed

• Prefixed JSON filenames with the image UUID by @jstnlowe in #6486
• feat(ui): control layers internals cleanup by @psychedelicious in #6487
• LoRA patching optimization by @lstein in #6439
• fix(ui): re-process control image if processed image is missing on page load by @psychedelicious in #6494
• Split up latent.py (code reorganization, no functional changes) by @RyanJDick in #6491
• Add simplified model manager install API to InvocationContext by @lstein in #6132
• fix: Some imports from previous PR's by @blessedcoolant in #6501
• Improve RAM<->VRAM memory copy performance in LoRA patching and elsewhere by @lstein in https://github.com/invoke-ai/InvokeAI/pull...

v4.2.6a1

v4.2.6a1 includes a handful of fixes and improvements, plus three major changes:

• Gallery updates
• Tiled upscaling via MultiDiffusion
• Checkpoint models work without conversion to diffusers

Known Issues

Our last release, v4.2.5, was quickly pulled after a black image issue on MPS (macOS) was discovered. We also had reports of CUDA (Nvidia) GPUs getting unexpected OOM (Out of Memory) errors.

The MPS issue is resolved in this release, but we haven't been able to replicate unexpected OOMs on Linux or Windows. We did fix one issue that may have been a factor.

If you get OOMs on this alpha release with settings that worked fine on v4.2.4 - or have any other issues - please let us know via GH issues or discord.

Gallery Updates

We've made some changes to the gallery, adding features, improving the performance of the app and reducing memory usage. The changes also fix a number of bugs relating to stale data - for example, a board not updating as expected after moving an image to it.

Thanks to @chainchompa and @maryhipp for working on this major effort.

Pagination & Selection

Infinite scroll is dead, long live infinite scroll!

The gallery is now paginated. Selection logic has been updated to work with pagination. An indicator shows how many images are selected and allows you to clear the selection entirely. Arrow keys still navigate.

Gallery.Pagination.and.Selection.mov

The number of images per page is dynamically calculated as the panel is resized, ensuring the panel is always filled with images.

Boards UI Refresh

The bulky tiled boards grid has been replaced by a scrollable list. The boards list panel is now a resizable, collapsible panel.

Boards.List.and.Resizable.Panel.mov

Boards and Image Search

Search for boards by name and images by metadata. The search term is matched against the image's metadata as a string. We landed on full-text search as a flexible yet simple implementation after considering a few methods for search.

Boards.and.Images.Search.mov

Archived Boards

Archive a board to hide it from the main boards list. This is purely an organizational enhancement. You can still interact with archived boards as you would any other board.

Archived.Boards.mov

Image Sorting

You can now change the sort for images to show oldest first. A switch allows starred images to be placed in the list according to their age, instead of always showing them first.

Image.Sorting.mov

Tiled Upscaling via MultiDiffusion

MultiDiffusion is a fairly straightforward technique for tiled denoising. The gist is similar to other tiled upscaling methods - split the input image up in to tiles, process each independently, and stitch them back together. The main innovation for MultiDiffusion is to do this in latent space, blending the tensors together continually. This results in excellent consistency across the output image, with no seams.

This feature is exposed as a Tiled MultiDiffusion Denoise Latents node, currently classified as a beta version. It works much the same as the OG Denoise Latents node. Here's a workflow to get you started: sd15_multi_diffusion_esrgan_x2_upscale.json

We are still thinking about to expose this in the linear UI. Most likely, we expose this with very minimal settings. If you want to tweak it, use the workflow.

Thanks to @RyanJDick for designing and implementing MultiDiffusion.

How to use it

This technique is fundamentally the same as normal img2img. Appropriate use of conditioning and control will greatly improve the output. The one hard requirement is to use the Tile ControlNet model.

Besides that, here are some tips from our initial testing:

• Use a detail-adding or style LoRAs.
• Use a base model best suited for the desired output style.
• Prompts make a difference.
• The initial upscaling method makes a difference.
• Scheduler makes a difference. Some produce softer outputs.

VRAM Usage

This technique can upscale images to very large sizes without substantially increasing VRAM usage beyond what you'd see for a "normal" sized generation. The VRAM bottlenecks then become the first VAE encode (Image to Latents) and final VAE decode (Latents to Image) steps.

You may run into OOM errors during these steps. The solution is to enable tiling using the toggle on the Image to Latents and Latents to Image nodes. This allows the VAE operations to be done piecewise, similar to the tiled denoising process, without using gobs of VRAM.

There's one caveat - VAE tiling often introduces inconsistency across tiles. Textures and colors may differ from tile to tile. This is a function of the diffusers handling of VAE tiling, not the tiled denoising process introduced in v4.2.5. We are investigating ways to improve this.

Takeaway: If your GPU can handle non-tiled VAE encode and decode for a given output size, use that for best results.

Checkpoint models work without conversion to diffusers

The required conversion of checkpoint format models to diffusers format has long been a pain point. Diffusers now supports loading single-file (checkpoint) models directly, and we have removed the mandatory checkpoint-to-diffusers conversion step.

The main user-facing change is that there is no longer a conversion cache directory!

Major thanks to @lstein for getting this working.

📈 Patch Nodes for v4.2.5

Enhancements

• When downloading image metadata, graphs or workflows, the JSON file includes the image name and type of data. Thanks @jstnlowe!
• Add clear_queue_on_startup config setting to clear problematic queues. This is useful for a rare edge case where your queue is full of items that somehow crash the app. Set this to true, and the queue will clear before it has time to attempt to execute the problematic item. Thanks @steffy-lo!
• Performance and memory efficiency improvements for LoRA patching and model offloading.
• Addition of a simplified model installation methods to the Invocation API: download_and_cache_model, load_local_model and load_remote_model. These methods allow models to be used without needing them to be added to the model manager. For example, we are now using these methods to load ESRGAN models.
• Support for probing and loading SDXL VAE checkpoint.
• Updated gallery UI.
• Checkpoint models work without conversion to diffusers.
• When using a VAE in tiled mode, you may now select the tile size.

Fixes

• Fix handling handling of 0-step denoising process.
• If a control image's processed version is missing when the app loads, it is now re-processed.
• Fixed an issue where a model's size could be misreported as 0, possibly causing memory issues.

Performance improvements

• Improved LoRA patching.
• Improved RAM <-> VRAM model transfer performance.

Internal changes

• The DenoiseLatentsInvocation has had its internal methods split up to support tiled upscaling via MultiDiffusion. This included some amount of file shuffling and renaming. The invokeai package's exported classes should still be the same. Please let us know if this has broken an import for you.
• Internal cleanup, intending to eliminate circular import issues. There's a lot left to do for this issue, but we are making progress.

💾 Installation and Updating

To install or update to v4.2.6a1, download the installer and follow the installation instructions.

To update, select the same installation location. Your user data (images, models, etc) will be retained.

Missing models after updating from v3 to v4

See this FAQ.

Error during installation ModuleNotFoundError: No module named 'controlnet_aux'

See this FAQ

What's Changed

• Prefixed JSON filenames with the image UUID by @jstnlowe in #6486
• feat(ui): control layers internals cleanup by @psychedelicious in #6487
• LoRA patching optimization by @lstein in #6439
• fix(ui): re-process control image if processed image is missing on page load by @psychedelicious in #6494
• Split up latent.py (code reorganization, no functional changes) by @RyanJDick in #6491
• Add simplified model manager install API to InvocationContext by @lstein in #6132
• fix: Some imports from previous PR's by @blessedcoolant in #6501
• Improve RAM<->VRAM memory copy performance in LoRA patching and elsewhere by @lstein in #6490
• Fix DEFAULT_PRECISION handling by @RyanJDick in https://github.com/in...