[advanced dreambooth lora sdxl] add DoRA training feature

[linoytsaban]

Following the integration of DoRA: Weight-Decomposed Low-Rank Adaptation into peft in: huggingface/peft#1474

This PR adds a use_dora arg to allow training a DoRA with the current advanced dreambooth script

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[sayakpaul]

I think we should till the peft PR is merged so that we can suggest folks to install peft from the main. Also, let's propagate to the SD v1.5 advanced script?

[nbasyl]

@linoytsaban, appreciate your effort on pushing this PR through. The DoRA PR has been successfully merged into the PEFT main branch. Since, I'm not very familiar with stable diffusion at all, it would be great if you could continue with the PR, and I can get to try DoRA on fine-tuning stable diffusion model and see if I can think of any other further improvement I can make specifically for SD finetuning.

[linoytsaban]

@sayakpaul @apolinario I added sd 1.5 and peft pr is now merged too so I can merge if ok with you :)

[sayakpaul]

@sayakpaul @apolinario I added sd 1.5 and peft pr is now merged too so I can merge if ok with you :)

Let’s mention that using DoRA requires installing peft from the source. Let’s also include some notes about DoRA from your experiments to guide our users. WDYT?

Once done, LMK, I would like to do a review before finally merging.

[linoytsaban]

Added some notes about peft installation and the 2 main conclusions so far from DoRA training. WDYT?

[sayakpaul]

Let's go!