LoRA微调的损失为什么一直是0啊？

[LiangYong1216]

我的Lora配置：
data_config:
train_file: train.json
val_file: dev.json
test_file: dev.json
num_proc: 16
max_input_length: 128
max_output_length: 256
training_args:

see transformers.Seq2SeqTrainingArguments

output_dir: ./output
max_steps: 5000

settings for data loading

per_device_train_batch_size: 6
dataloader_num_workers: 16
remove_unused_columns: false

settings for saving checkpoints

save_strategy: steps
save_steps: 300

settings for logging

log_level: info
logging_strategy: steps
logging_steps: 10

settings for evaluation

per_device_eval_batch_size: 16
evaluation_strategy: steps
eval_steps: 5001

settings for optimizer

adam_epsilon: 1e-6

uncomment the following line to detect nan or inf values

debug: underflow_overflow

predict_with_generate: true

see transformers.GenerationConfig

generation_config:
max_new_tokens: 256

set your absolute deepspeed path here

#deepspeed: ds_zero_2.json

set to true if train with cpu.

use_cpu: false
peft_config:
peft_type: LORA
task_type: CAUSAL_LM
r: 8
lora_alpha: 32
lora_dropout: 0.1
这是为什么啊？

[LiangYong1216]

使用的数据集是AdvertiseGen,就是广告开源的哪个数据集

[zRzRzRzRzRzRzR]

查看一下你的依赖，大概率是依赖没对，一定要严格使用peft =0.7.1