This folder contains examples of running different training mode with IPEX-LLM on Intel GPU:
- LoRA: examples of running LoRA finetuning
- QLoRA: examples of running QLoRA finetuning
- QA-LoRA: examples of running QA-LoRA finetuning
- ReLora: examples of running ReLora finetuning
- DPO: examples of running DPO finetuning
- common: common templates and utility classes in finetuning examples
- HF-PEFT: run finetuning on Intel GPU using Hugging Face PEFT code without modification
- axolotl: LLM finetuning on Intel GPU using axolotl without writing code
| Model | Finetune mode | Frameworks Support |
|---|---|---|
| LLaMA 2/3 | LoRA, QLoRA, QA-LoRA, ReLora | HF-PEFT, axolotl |
| Mistral | LoRA, QLoRA | DPO |
| ChatGLM 3 | LoRA, QLoRA | HF-PEFT |
| Qwen-1.5 | QLoRA | HF-PEFT |
| Baichuan2 | QLoRA | HF-PEFT |
-
If you fail to finetune on multi cards because of following error message:
RuntimeError: oneCCL: comm_selector.cpp:57 create_comm_impl: EXCEPTION: ze_data was not initialized
Please try
sudo apt install level-zero-devto fix it. -
Please raise the system open file limit using
ulimit -n 1048576. Otherwise, there may exist errorToo many open files. -
If application raise
wandb.errors.UsageError: api_key not configured (no-tty). Please login wandb or disable wandb login with this command:
export WANDB_MODE=offline- If application raise Hugging Face related errors, i.e.,
NewConnectionErrororFailed to downloadetc. Please download models and datasets, set model and data path, then setHF_HUB_OFFLINEwith this command:
export HF_HUB_OFFLINE=1