lora 训练参数

[MonkeyTB]

trainable params: 22021440 || all params: 6195307952 || trainable%: 0.35545351692954874

请教一下，更新之后的lora为啥训练参数占比这么高了，之前只有0.05左右

[MonkeyTB]

trainable params: 22021440 || all params: 6195307952 || trainable%: 0.35545351692954874

请教一下，更新之后的lora为啥训练参数占比这么高了，之前只有0.05左右

原始老版本的lora
trainable params: 3670016 || all params: 6176956416 || trainable%: 0.05941463324063059

[shibing624]

参考qlora的论文，改了lora_target_module默认参数，lora_target_modules= all linear layers, 全部dense层参与训练，效果最好。

想改回之前的，设置lora_target_modules参数即可。

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.(由于长期不活动，机器人自动关闭此问题，如果需要欢迎提问)