"RuntimeError: Found dtype Half but expected Float" when trying to train unconditional_image_generation using `--mixed_precision=fp16'

[linhduongtuan]

Describe the bug

"RuntimeError: Found dtype Half but expected Float" when trying to train unconditional_image_generation using `--mixed_precision=fp16'

But it works if I set --mixed_precision=no

Reproduction

interactive --gpus=4 -t 2:20:00
accelerate config defaults

accelerate launch --mixed_precision="fp16" --multi_gpu train_unconditional.py
--dataset_name="huggan/flowers-102-categories"
--resolution=64 --center_crop --random_flip
--output_dir="ddpm-ema-flowers-64"
--train_batch_size=16
--num_epochs=100
--gradient_accumulation_steps=1
--use_ema
--learning_rate=1e-4
--lr_warmup_steps=500
--mixed_precision="fp16"
--prediction_type="epsilon"
--logger="wandb"

Logs

No response

System Info

Ubuntu 20.4
Pytorch 2.0.1
Diffusers 0.22
Transformers 4.33
Accelerate 0.23
GPUs A100

Who can help?

@sayakpaul
I encounter with this error "RuntimeError: Found dtype Half but expected Float" when trying to train unconditional_image_generation using `--mixed_precision=fp16'

But it works if I set --mixed_precision=no
Here is my tracebacks

Thank in advance,
Linh

[linhduongtuan]

Hi @sayakpaul
I hope this report finds you well. Would you be able to take a look into my bug report and help me?
Linh

[sayakpaul]

Hi @linhduongtuan.

#5407 should fix it.

[linhduongtuan]

Hi @sayakpaul
Thank you so much for supporting.
Linh