-
Notifications
You must be signed in to change notification settings - Fork 5.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add a new community pipeline #5477
Add a new community pipeline #5477
Conversation
examples/community/latent_consistency_img2img.py which can be called like this import torch from diffusers import DiffusionPipeline pipe = DiffusionPipeline.from_pretrained( "SimianLuo/LCM_Dreamshaper_v7", custom_pipeline="latent_consistency_txt2img", custom_revision="main") # To save GPU memory, torch.float16 can be used, but it may compromise image quality. pipe.to(torch_device="cuda", torch_dtype=torch.float32) img2img=LatentConsistencyModelPipeline_img2img( vae=pipe.vae, text_encoder=pipe.text_encoder, tokenizer=pipe.tokenizer, unet=pipe.unet, #scheduler=pipe.scheduler, scheduler=None, safety_checker=None, feature_extractor=pipe.feature_extractor, requires_safety_checker=False, ) img = Image.open("thisismyimage.png") result = img2img(prompt,img,strength,num_inference_steps=4)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Cool addition! Could we maybe also add a small entry to the README here: https://github.com/huggingface/diffusers/tree/main/examples/community#latent-consistency-pipeline
The documentation is not available anymore as the PR was closed or merged. |
Fix name formatting for scheduler Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
7e76ac4
to
76f9f1e
Compare
76f9f1e
to
072e682
Compare
which formatter do I need to run? |
|
thanks! |
* Add a new community pipeline examples/community/latent_consistency_img2img.py which can be called like this import torch from diffusers import DiffusionPipeline pipe = DiffusionPipeline.from_pretrained( "SimianLuo/LCM_Dreamshaper_v7", custom_pipeline="latent_consistency_txt2img", custom_revision="main") # To save GPU memory, torch.float16 can be used, but it may compromise image quality. pipe.to(torch_device="cuda", torch_dtype=torch.float32) img2img=LatentConsistencyModelPipeline_img2img( vae=pipe.vae, text_encoder=pipe.text_encoder, tokenizer=pipe.tokenizer, unet=pipe.unet, #scheduler=pipe.scheduler, scheduler=None, safety_checker=None, feature_extractor=pipe.feature_extractor, requires_safety_checker=False, ) img = Image.open("thisismyimage.png") result = img2img(prompt,img,strength,num_inference_steps=4) * Apply suggestions from code review Fix name formatting for scheduler Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com> * update readme (and run formatter on latent_consistency_img2img.py) --------- Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
I tried: it is installed, but can't be imported in python - any ideas?
|
* Add a new community pipeline examples/community/latent_consistency_img2img.py which can be called like this import torch from diffusers import DiffusionPipeline pipe = DiffusionPipeline.from_pretrained( "SimianLuo/LCM_Dreamshaper_v7", custom_pipeline="latent_consistency_txt2img", custom_revision="main") # To save GPU memory, torch.float16 can be used, but it may compromise image quality. pipe.to(torch_device="cuda", torch_dtype=torch.float32) img2img=LatentConsistencyModelPipeline_img2img( vae=pipe.vae, text_encoder=pipe.text_encoder, tokenizer=pipe.tokenizer, unet=pipe.unet, #scheduler=pipe.scheduler, scheduler=None, safety_checker=None, feature_extractor=pipe.feature_extractor, requires_safety_checker=False, ) img = Image.open("thisismyimage.png") result = img2img(prompt,img,strength,num_inference_steps=4) * Apply suggestions from code review Fix name formatting for scheduler Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com> * update readme (and run formatter on latent_consistency_img2img.py) --------- Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
This PR adds an img2img pipeline for Latent Consistency Models: luosiallen/latent-consistency-model#8 (comment)
examples/community/latent_consistency_img2img.py
which can be called like this
import torch
from diffusers import DiffusionPipeline
pipe = DiffusionPipeline.from_pretrained(
"SimianLuo/LCM_Dreamshaper_v7", custom_pipeline="latent_consistency_txt2img", custom_revision="main")
pipe.to(torch_device="cuda", torch_dtype=torch.float32)
img2img=LatentConsistencyModelPipeline_img2img(
vae=pipe.vae,
text_encoder=pipe.text_encoder,
tokenizer=pipe.tokenizer,
unet=pipe.unet,
#scheduler=pipe.scheduler,
scheduler=None,
safety_checker=None,
feature_extractor=pipe.feature_extractor,
requires_safety_checker=False,
)
img = Image.open("thisismyimage.png")
result = img2img(prompt,img,strength,num_inference_steps=4)
What does this PR do?
Fixes # (issue)
Before submitting
documentation guidelines, and
here are tips on formatting docstrings.
Who can review?
Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.