Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support Rank Stabilized LoRA in the ModelConfig/LoraConfig #1877

Merged
merged 12 commits into from
Aug 6, 2024
11 changes: 11 additions & 0 deletions trl/trainer/model_config.py
Original file line number Diff line number Diff line change
@@ -64,6 +64,17 @@ class ModelConfig:
lora_task_type: str = field(
default="CAUSAL_LM", metadata={"help": "The task_type to pass for LoRA (use SEQ_CLS for reward modeling)"}
)
use_rslora: bool = field(
default=False,
metadata={
"help": (
"When set to True, uses <a href='https://doi.org/10.48550/arXiv.2312.03732'>Rank-Stabilized LoRA</a>"
" which sets the adapter scaling factor to `lora_alpha/math.sqrt(r)`, since it"
" was proven to work better. Otherwise, it will use the original default"
qgallouedec marked this conversation as resolved.
Show resolved Hide resolved
" value of `lora_alpha/r`."
)
},
)
load_in_8bit: bool = field(
default=False, metadata={"help": "use 8 bit precision for the base model - works only with LoRA"}
)
1 change: 1 addition & 0 deletions trl/trainer/utils.py
Original file line number Diff line number Diff line change
@@ -791,6 +791,7 @@ def get_peft_config(model_config: ModelConfig) -> "Optional[PeftConfig]":
lora_dropout=model_config.lora_dropout,
bias="none",
task_type=model_config.lora_task_type,
use_rslora=model_config.use_rslora,
target_modules=model_config.lora_target_modules,
modules_to_save=model_config.lora_modules_to_save,
)