A minimal pytorch implementation for lora via pytorch parametrizations
pip install pytora
from pytora import apply_lora
apply_lora(
model,
lora_r = 4,
lora_alpha = 1,
lora_dropout = 0.0
)
A simple example use case with a pretrained huggingface model can be found in demo.ipynb