Resizing of extracted lora giving error IndexError: index 320 is out of bounds for dimension 0 with size 320

[fractal-fumbler]

hello :) thanks for your efforts and work :)

first of all i am extracting LoRA from custom checkpoint (SD-2.x) using extract_lora_from_models.py

python extract_lora_from_models.py --save_to model_r512.safetensors \
--model_org v2-1_768-ema-pruned.safetensors --model_tuned model.safetensors \
--device cuda --dim 512 --conv_dim 512 --save_precision fp16 --v2

after i am trying to resize LoRA for purpose of reducing "useless noise" using resize_lora.py

python resize_lora.py --save_precision fp16 --save_to model_r512_sv20.safetensors \
--model model_r512.safetensors --device cuda \
--dynamic_method sv_ratio --dynamic_param 20 --new_rank 512

after executing command i am getting error
IndexError: index 320 is out of bounds for dimension 0 with size 320

Dynamically determining new alphas and dims based off sv_ratio: 20.0, max rank is 512                                   
 97%|████████████████████████████████████████████████████████████████████████████▊  | 1213/1248 [03:24<00:05,  5.94it/s]
Traceback (most recent call last):                                                                                      
  File "/tmp/yay/extract_lora_resize/resize_lora.py", line 350, in <module>                                             
    resize(args)                                                                                                        
  File "/tmp/yay/extract_lora_resize/resize_lora.py", line 299, in resize                                               
    state_dict, old_dim, new_alpha = resize_lora_model(lora_sd, args.new_rank, save_dtype, args.device, args.dynamic_met
hod, args.dynamic_param, args.verbose)                                                                                  
  File "/tmp/yay/extract_lora_resize/resize_lora.py", line 236, in resize_lora_model                                    
    param_dict = extract_conv(full_weight_matrix, new_rank, dynamic_method, dynamic_param, device, scale)               
  File "/tmp/yay/extract_lora_resize/resize_lora.py", line 68, in extract_conv                                          
    param_dict = rank_resize(S, lora_rank, dynamic_method, dynamic_param, scale)                                        
  File "/tmp/yay/extract_lora_resize/resize_lora.py", line 175, in rank_resize                                          
    param_dict["max_ratio"] = S[0]/S[new_rank]                                                                          
IndexError: index 320 is out of bounds for dimension 0 with size 320 

what i am doing wrong?

environment
OS: archlinux
pytorch: 1.13.1
python: 3.10.10
custom checkpoint: https://civitai.com/models/3452/ultraskin

[mgz-dev]

Oops. This is a mistake on my part.

It is because none of the dims in one or more of the layer can satisfy the "sv_ratio = 20" requirement so the index goes out of bounds since the dimension of that layer is only 320. It looks like I had meant to wrap a min function at line 135 but applied it at line 148 instead.

As a quick fix, it should run if you insert the following line below line 135 in resize_lora.py:

new_rank = min(new_rank, len(S)-1)

the updated if statement block should look like this:

    if dynamic_method=="sv_ratio":
        # Calculate new dim and alpha based off ratio
        max_sv = S[0]
        min_sv = max_sv/dynamic_param
        new_rank = max(torch.sum(S > min_sv).item(),1)
        new_rank = min(new_rank, len(S)-1)
        new_alpha = float(scale*new_rank)

I'll also get together a PR to fix this.

As a side note, I tested on the same models that you did to confirm the error and noticed that the output file sizes are still quite large. I'd recommend trying a lower sv_ratio (5-15 range) or capping the max dim to see if you can reduce the size more aggressively. If you want a numerical comparison on how "well" it was able to replicate the original, you can add the flag --verbose to see the frobenius norm ratio (you can think of this as a proxy for how accurate the reduced model replicates the original model).

For example at sv_ratio = 20, the result is 97.7%, and sv_ratio = 10, the result is 93.7% Of course, visual test at the end is needed to see if the result is satisfactory. I've also noticed though that extracted LoRA tend to require larger sizes than resizing natively trained LoRA so in the end it may not be possible to retain the level of information you want without having a large file.

[fractal-fumbler]

#328 fixed issue for me - lora successfully resized.

i was able to extract more than 320 dimension after recent update, so decided to try out resizing :)
and started with sv_ration = 20.

thanks a lot! i'll try with 5-15 to compare result

[pandora523]

activating extra network lora with arguments [<modules.extra_networks.ExtraNetworkParams object at 0x292773760>]: RuntimeError
Traceback (most recent call last):
File "/Users/rei/stable-diffusion-webui/modules/extra_networks.py", line 75, in activate
extra_network.activate(p, extra_network_args)
File "/Users/rei/stable-diffusion-webui/extensions-builtin/Lora/extra_networks_lora.py", line 23, in activate
lora.load_loras(names, multipliers)
File "/Users/rei/stable-diffusion-webui/extensions-builtin/Lora/lora.py", line 214, in load_loras
lora = load_lora(name, lora_on_disk.filename)
File "/Users/rei/stable-diffusion-webui/extensions-builtin/Lora/lora.py", line 176, in load_lora
module.weight.copy_(weight)
RuntimeError: output with shape [128, 320, 1, 1] doesn't match the broadcast shape [128, 320, 3, 3]

im having the same problem I did extract a Lora from (My own face)x(1.5) trained checkpoint and its so big so idid do the extract method but when I load the Lora im having this problem.