Add ⚡️Sana: Efficient High-Resolution Image Synthesis with Linear Diffusion Transformer Support

[darkflare]

Feature Idea

🐱 Sana Model Card

Model

We introduce Sana, a text-to-image framework that can efficiently generate images up to 4096 × 4096 resolution.
Sana can synthesize high-resolution, high-quality images with strong text-image alignment at a remarkably fast speed, deployable on laptop GPU.

Source code is available at https://github.com/NVlabs/Sana.

Model Description

• Developed by: NVIDIA, Sana
• Model type: Linear-Diffusion-Transformer-based text-to-image generative model
• Model size: 1648M parameters
• Model resolution: This model is developed to generate 1024px based images with multi-scale heigh and width.
• License: CC BY-NC-SA 4.0 License
• Model Description: This is a model that can be used to generate and modify images based on text prompts.
  It is a Linear Diffusion Transformer that uses one fixed, pretrained text encoders (Gemma2-2B-IT)
  and one 32x spatial-compressed latent feature encoder (DC-AE).
• Resources for more information: Check out our GitHub Repository and the Sana report on arXiv.

Model Sources

For research purposes, we recommend our generative-models Github repository (https://github.com/NVlabs/Sana),
which is more suitable for both training and inference and for which most advanced diffusion sampler like Flow-DPM-Solver is integrated.
MIT Han-Lab provides free Sana inference.

• Repository: ttps://github.com/NVlabs/Sana
• Demo: https://nv-sana.mit.edu/

🧨 Diffusers

PR developing: Sana and DC-AE

Uses

Direct Use

The model is intended for research purposes only. Possible research areas and tasks include

• Generation of artworks and use in design and other artistic processes.

• Applications in educational or creative tools.

• Research on generative models.

• Safe deployment of models which have the potential to generate harmful content.

• Probing and understanding the limitations and biases of generative models.

Excluded uses are described below.

Out-of-Scope Use

The model was not trained to be factual or true representations of people or events, and therefore using the model to generate such content is out-of-scope for the abilities of this model.

Limitations and Bias

Limitations

• The model does not achieve perfect photorealism
• The model cannot render complex legible text
• fingers, .etc in general may not be generated properly.
• The autoencoding part of the model is lossy.

Bias

While the capabilities of image generation models are impressive, they can also reinforce or exacerbate social biases.

Existing Solutions

No response

Other

No response

[Amazon90]

Unfortunately, only the 512-pixel and 1024-pixel versions of the model have been released so far. According to the README, it seems there is a 4096-pixel version as well.

[jomcey]

Unfortunately, only the 512-pixel and 1024-pixel versions of the model have been released so far. According to the README, it seems there is a 4096-pixel version as well.

Pretty sure that's just referring to generating 4096x4096 images, not that there's a separate model.

[goodgutter]

is there a way to run this already in comfyui?

[ltdrdata]

There is an unofficial implementation if you are interested.
https://github.com/zmwv823/ComfyUI-Sana

[zmwv823]

Now use official custom-node ComfyUI_ExtraModels instead.

[Amazon90]

Now use official custom-node ComfyUI_ExtraModels instead.

Exciting news!

[ashrafbay]

Now use official custom-node ComfyUI_ExtraModels instead.

Exciting news!

Full of errors for windows I could not run it

[zmwv823]

Now use official custom-node ComfyUI_ExtraModels instead.

Exciting news!

Full of errors for windows I could not run it

Waiting for complete, it's in progress.
0.6b works.

[city96]

I'm currently doing a full rewrite of ExtraModels to try and get better support on both PixArt and Sana.

Progress is being tracked on this PR: city96/ComfyUI_ExtraModels#92

[ashrafbay]

I'm currently doing a full rewrite of ExtraModels to try and get better support on both PixArt and Sana.

Progress is being tracked on this PR: city96/ComfyUI_ExtraModels#92

This is so cool. Thanks for your work.