-
Notifications
You must be signed in to change notification settings - Fork 278
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
SDXL-Lightning support #198
Comments
any news on this? |
Update: Oops, I suddenly found that I had forgot to add Command: Result here: Tested on Google Colab (T4 GPU).
|
Seems like stable-diffusion.cpp does not support sdxl_lightning_1step.safetensors right now.
Log: Option:
n_threads: 2
mode: txt2img
model_path: SDXL-Lightning/sdxl_lightning_1step.safetensors
wtype: unspecified
vae_path: SDXL-Lightning/sdxl_vae.safetensors
taesd_path:
esrgan_path:
controlnet_path:
embeddings_path:
output_path: test.png
init_img:
control_image:
controlnet cpu: false
strength(control): 0.90
prompt: 1girl, extremely beautiful, miniskirt, nsfw
negative_prompt:
min_cfg: 1.00
cfg_scale: 1.00
clip_skip: -1
width: 1024
height: 1024
sample_method: euler_a
schedule: default
sample_steps: 1
strength(img2img): 0.75
rng: cuda
seed: 1768825772
batch_count: 1
vae_tiling: false
upscale_repeats: 1
System Info:
BLAS = 1
SSE3 = 1
AVX = 1
AVX2 = 1
AVX512 = 0
AVX512_VBMI = 0
AVX512_VNNI = 0
FMA = 1
NEON = 0
ARM_FMA = 0
F16C = 1
FP16_VA = 0
WASM_SIMD = 0
VSX = 0
[DEBUG] stable-diffusion.cpp:122 - Using CUDA backend
ggml_init_cublas: GGML_CUDA_FORCE_MMQ: no
ggml_init_cublas: CUDA_USE_TENSOR_CORES: yes
ggml_init_cublas: found 1 CUDA devices:
Device 0: Tesla T4, compute capability 7.5, VMM: yes
[INFO ] stable-diffusion.cpp:142 - loading model from 'SDXL-Lightning/sdxl_lightning_1step.safetensors'
[WARN ] model.cpp:682 - unknown format SDXL-Lightning/sdxl_lightning_1step.safetensors
[ERROR] stable-diffusion.cpp:148 - init model loader from file failed: 'SDXL-Lightning/sdxl_lightning_1step.safetensors'
new_sd_ctx_t failed This may be because SDXL Lightning 1step version needs "sample" prediction type. def generate_image(prompt, ckpt):
checkpoint = checkpoints[ckpt][0]
num_inference_steps = checkpoints[ckpt][1]
if num_inference_steps==1:
# Ensure sampler uses "trailing" timesteps and "sample" prediction type for 1-step inference.
pipe.scheduler = EulerDiscreteScheduler.from_config(pipe.scheduler.config, timestep_spacing="trailing", prediction_type="sample")
else:
# Ensure sampler uses "trailing" timesteps.
pipe.scheduler = EulerDiscreteScheduler.from_config(pipe.scheduler.config, timestep_spacing="trailing")
pipe.unet.load_state_dict(torch.load(hf_hub_download(repo, checkpoint), map_location="cuda"))
image = pipe(prompt, num_inference_steps=num_inference_steps, guidance_scale=0).images[0]
return image |
You can get my quantized results here: https://huggingface.co/mzwing/SDXL-Lightning-GGUF |
Update: I suddenly realized that I had made a mistake: I did not download the 1step model at that time! That's why I got the error However, the model's generated image is very very bad. I thought that's because SDXL Lightning 1step version needs "sample" prediction type. See my comment above. Generated by You can still get the quantized version of the 1step model in my huggingface repo, but not recommended to use them. |
Yea the 1step needs custom code. But the other lightning variants work like a charm. |
Looks like for even better lightning generation we need a custom scheduler (for ref AUTOMATIC1111/stable-diffusion-webui#15325 ) |
hi do these ggufs need any vae or are they good to go alone? thanks for making them! EDIT: nevermind worked fine without VAE for me! used q4_1. |
Yes, when I quantised them I added the fp16 fix vae in it too :) |
I'm so glad you made this because lightning is a perfect fit for the idea of quick lower requirement generation. I'm using it with koboldcpp since that has a chat and multimodal interface too. There, combining your 2 step model with LCM sampler looks really good to me. |
@mzwing sdxl_lightning_1step_x0.q8_0.gguf i will test. |
Maybe you also need to implement a prediction type if you use 1 step version to test ( The missing scheduler, I think, just uses for better quality of images generated by the SDXL-Lightning series. See #198 (comment) |
@mzwing could u please make some more 2step gguf q4.0 models? realistic unensored versions will be great |
hey guys, its my findings = |
Sorry for my previous neglect. However now I have some other things to do and so may not be able to help you about that. If you don't mind, I can help you quantize it a week later. BTW could you explain more clearly about the "realistic unensored versions"? I have limited knowledge on them. Edit: Seems that you have already quantized it? 👍 |
greetings, thank u for ur answer, yes i managed to quantize turbo to q5.0 and it worked greatly. yes i got the models i needed, the uncensored ones i meant a model that is capable of nsfw stuff, but it is ok cause im using pony realism now which is great |
Would you mind sharing links? |
i like to do it but sadly my upload speed is in a bad shape, sorry |
Hello,
I wonder if stable-diffusion.cpp supports ByteDance/SDXL-Lightning?
The text was updated successfully, but these errors were encountered: