You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Yes actually! We are currently working or rolling out some support for quantization generally and quantize lora.
We will work on some actual docs for this soon, but for now you can check out the qlora.py in the description of this PR to see what we are working on -> keras-team/keras#19356
I believe the 4 bit normal float from the qlora paper is not currently available, but int8 quantized weights layered in with lora should be good to go.
Early days on the feature, so any feedback appreciated as we build it out.
Hi there!
keras-nlp
supports Lora, e.g. from https://ai.google.dev/gemma/docs/lora_tuningJust wondering are there any plans to implement QLora finetuning, to help make finetuning even more accessible on smaller GPUs?
https://arxiv.org/abs/2305.14314
https://huggingface.co/blog/4bit-transformers-bitsandbytes
Many thanks for any help, and this lib!
The text was updated successfully, but these errors were encountered: