LoRA support

[monstruosoft]

I recently noticed that the diffusers library natively supports loading LoRAs, so it's trivial to add LoRA support; these were made with FastSD CPU using an LCM model 3-step inference, same prompt, same seed, the first image is without LoRA, the rest are with LoRA weights 0.0 to 1.0 with a 0.1 weight increment each:

[rupeshs]

Latest release support it in CLI and WebUI
https://github.com/rupeshs/fastsdcpu/releases/tag/v1.0.0-beta.25

[evilblade666]

Hey folks. I'm a noob in all this so maybe it's just me doing something wrong.. I've got the latest version of fastsdcpu (which is v1.0.0 beta 33, right?), downloaded some LoRAs from civitai.com (the ones which have SD 1.5 as the base model) and put them into "lora_models" folder. After that they appeared in "Lora models" tab of fastsdcpu's webui and I can load them (btw, I didn't see such an option in GUI at all) .. But it has absolutely no effect on generated pictures :(
I've asked more experienced folks on a discord, they tried this same fastsdcpu UI and said that it seems like LoRAs don't load.
Is it true?.. Or am I just missing something?

UPD: Downloaded another UI (turns out ComfyUI can be run with --cpu key) and the same LoRA files immediately started to work.. so they are definitely not the problem.

[monstruosoft]

Are you using .safetensors base models from Civitai in LCM-LoRA mode? LoRA seems to work fine in plain LCM mode but not in LCM-LoRA mode with .safetensors models. Thanks for reporting this issue, it might need some work.

[evilblade666]

Are you using .safetensors base models from Civitai in LCM-LoRA mode? LoRA seems to work fine in plain LCM mode but not in LCM-LoRA mode with .safetensors models. Thanks for reporting this issue, it might need some work.

Oh-wow! It never even crossed my mind to try a mode other than the one that has a word "Lora" in it's name :D Thanks, will try..
UPD: YES! It seems to work now! Thank you!!!

[monstruosoft]

Looks like LoRA doesn't work at all in LCM-LoRA mode, either with .safetensors models or with diffusers format models. Weird, I thought it used to work well.

[monstruosoft]

I've just made a checkout on an older commit and can confirm that LoRA used to work fine in LCM-LoRA mode, then it must have got broken in a recent commit (probably by myself, even). Unfortunately, lately I've been kinda overwhelmed with IRL stuff and haven't been able to code at all.

[monstruosoft]

Apparently the problem is caused by always using fuse_lora() in LCM-LoRA mode. I thought it was safe to always fuse the LCM LoRA and then use the Adapters approach for normal LoRAs but apparently you can't mix fuse_lora() with Adapters. Simply commenting out the fuse_lora() line in src/backend/pipelines/lcm_lora.py solves this issue.