How to use peft for LoRA ensemble #1945
-
|
Can we use peft for LoRA ensemble? i.e., train multiple LoRA adapters and generate one output for each adapter in a single forward pass? |
Beta Was this translation helpful? Give feedback.
Answered by
BenjaminBossan
Jul 23, 2024
Replies: 1 comment
-
|
No, this is not possible. You can load multiple LoRA adapters and then cycle through each of them to get the same outcome, but it will be more expensive to call. What you describe cannot be easily achieved as that would require to change the output signature of each module's |
Beta Was this translation helpful? Give feedback.
0 replies
Answer selected by
DongxiaW
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
No, this is not possible. You can load multiple LoRA adapters and then cycle through each of them to get the same outcome, but it will be more expensive to call. What you describe cannot be easily achieved as that would require to change the output signature of each module's
forwardmethod.