Code

Add lora+ implementation#1915

Code

Merged

BenjaminBossan merged 34 commits intohuggingface:mainhuggingface/peft:mainfrom kallewoof:202407-lorapluskallewoof/peft:202407-loraplusCopy head branch name to clipboardJul 29, 2024

+271

Commits

Commits on Jul 29, 2024

• Add lora+ implentation

  

  moghadas76

  authored and

  kallewoof

  committed
  5f54802

  Copy full SHA for 5f54802

• Support LoraPlus cfg

  

  moghadas76

  authored and

  kallewoof

  committed
  1703b91

  Copy full SHA for 1703b91

• Fix QA comments

  

  moghadas76

  authored and

  kallewoof

  committed
  105bb9b

  Copy full SHA for 105bb9b

• Fix test

  

  moghadas76

  authored and

  kallewoof

  committed
  77e04ff

  Copy full SHA for 77e04ff

• Fix tests

  

  moghadas76

  authored and

  kallewoof

  committed
  e6e3979

  Copy full SHA for e6e3979

• Fix comments

  

  moghadas76

  authored and

  kallewoof

  committed
  7b64437

  Copy full SHA for 7b64437

• Add unit test

  

  moghadas76

  authored and

  kallewoof

  committed
  1ceab2b

  Copy full SHA for 1ceab2b

• Decouple file structures

  

  moghadas76

  authored and

  kallewoof

  committed
  27b375b

  Copy full SHA for 27b375b

• Fix clean code issues

  

  moghadas76

  authored and

  kallewoof

  committed
  4a385d4

  Copy full SHA for 4a385d4

• Fix styling problem

  

  moghadas76

  authored and

  kallewoof

  committed
  6228919

  Copy full SHA for 6228919

• Fix formatter

  

  moghadas76

  authored and

  kallewoof

  committed
  083b0b9

  Copy full SHA for 083b0b9

• Fix QA comments

  

  moghadas76

  authored and

  kallewoof

  committed
  1a75154

  Copy full SHA for 1a75154

• Fix docs

  

  moghadas76

  authored and

  kallewoof

  committed
  132dfac

  Copy full SHA for 132dfac

• style fixes

  kallewoof

  committed
  1d35a72

  Copy full SHA for 1d35a72

• move loraplus_lr_ratio out of opt_kwargs, and other fixes

  kallewoof

  committed
  65069de

  Copy full SHA for 65069de

• add support for other 8 bit optimizers

  Show description for 9146816
  

  kallewoof

  and

  stillmatic

  committed
  9146816

  Copy full SHA for 9146816

• revert unneeded import

  kallewoof

  committed
  4bca4a7

  Copy full SHA for 4bca4a7

• clean out old code

  kallewoof

  committed
  a5df785

  Copy full SHA for a5df785

• add 'LORAPLUS' to peft config mapping

  kallewoof

  committed
  af584cb

  Copy full SHA for af584cb

• conditional bnb in lora+ tests

  kallewoof

  committed
  56a5227

  Copy full SHA for 56a5227

• added compat for py 3.8

  kallewoof

  committed
  b6d2d6a

  Copy full SHA for b6d2d6a

• license header

  kallewoof

  committed
  95b9e80

  Copy full SHA for 95b9e80

• review fixes

  kallewoof

  committed
  0460dcd

  Copy full SHA for 0460dcd

• review fixes

  kallewoof

  committed
  80f7392

  Copy full SHA for 80f7392

• make lr and loraplus_lr_ratio required forced kw args

  kallewoof

  committed
  52d8e0b

  Copy full SHA for 52d8e0b

• lora+: do not propagate weight_decay to optimizer

  kallewoof

  committed
  f550e51

  Copy full SHA for f550e51

• doc: add LoRA+ optimizer example

  kallewoof

  committed
  b2f802b

  Copy full SHA for b2f802b

• remove LoraPlusConfig

  kallewoof

  committed
  29b3f5d

  Copy full SHA for 29b3f5d

• rename weight_decay to loraplus_weight_decay to avoid potential ambiguity

  kallewoof

  committed
  d445dad

  Copy full SHA for d445dad

• review fixes LoRA+ documentation

  kallewoof

  committed
  bf54d0a

  Copy full SHA for bf54d0a

• specify optimizer params

  kallewoof

  committed
  5981ecc

  Copy full SHA for 5981ecc

• assumption about lora set up

  kallewoof

  committed
  afe07e1

  Copy full SHA for afe07e1

• add lora hint to lora+ example code

  kallewoof

  committed
  c60ca71

  Copy full SHA for c60ca71

• missing LoraConfig import

  kallewoof

  committed
  babd89b

  Copy full SHA for babd89b