the usage of ema

[cb6e310]

hi, great work! i just would like to ask what is the role of ema in cifar10 example?

[atong01]

ema improves the performance by quite a bit on cifar (2-3 FID if I remember correctly). Would say this is fairly standard in diffusion models due to large variance in the gradients.

[kilianFatras]

Hi. Alex is right. Furthermore, without EMA, the FID is around 7.5. With EMA, the FID is around 3.7.