Add docs and examples for X-LoRA

[EricLBuehler]

Adds docs for X-LoRA as well as examples:

• Add docs
  • Package reference
  • Conceptual guide
  • Task guide
  • Developer guide
• Add examples
  • Training example
  • Inference example

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[BenjaminBossan]

Looks good so far: https://moon-ci-docs.huggingface.co/docs/peft/pr_1970/en/package_reference/xlora

Btw. did you join HF?

[EricLBuehler]

Btw. did you join HF?

Yes!

More docs: https://moon-ci-docs.huggingface.co/docs/peft/pr_1970/en/conceptual_guides/adapter#mixture-of-lora-experts-x-lora

[BenjaminBossan]

Btw. did you join HF?

Yes!

Ha, very nice! Didn't see any announcement yet. Welcome on board.

More docs:

Nice. LMK when this is ready for review (won't have time today but will check tomorrow).

[EricLBuehler]

Hi @BenjaminBossan! I think this should be ready for a review of the docs. I haven't added training examples yet, but I added one for mistralrs - do you think that's good or should we just add a link?

[BenjaminBossan]

Thanks a lot for the detailed docs. There isn't much to add from my side, just a few smaller comments.

I think it would still be great to have a training example to accompany the mistralrs inference example.

[BenjaminBossan]

Please always ping me when the PR is ready for review.

[EricLBuehler]

@BenjaminBossan it should be ready now!

[BenjaminBossan]

Thanks for providing the documentation for X-LoRA. LGTM.

[EricLBuehler]

Thanks! I'll add the training example in a later PR.

[cyx96]

May I ask when will the training example be published?