You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hey, thanks for this cool diffusion repo, it's really interesting.
I am playing around with the model and trying to use it to generate different types of non-audio distribution. My input is a random distribution of size (1,18,512). However, when I train the model and then perform inference, two things happen - the model loss doesn't seem to converge and the outputs don't match the original distribution.
The training code is as follows:
for epoch in range(epochs):
l = 0.0
for i,x in enumerate(train_loader):
optimizer.zero_grad()
loss = diffusion(x)
loss.backward() # Do this many times
optimizer.step()
#scheduler.step()
l += loss.detach().cpu().item()
Any suggestions for how I can get the loss to converge?
My optimizer is RMSProp and learning rate is 1e-4.
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
Hey, thanks for this cool diffusion repo, it's really interesting.
I am playing around with the model and trying to use it to generate different types of non-audio distribution. My input is a random distribution of size (1,18,512). However, when I train the model and then perform inference, two things happen - the model loss doesn't seem to converge and the outputs don't match the original distribution.
The training code is as follows:
Any suggestions for how I can get the loss to converge?
My optimizer is RMSProp and learning rate is 1e-4.
Beta Was this translation helpful? Give feedback.
All reactions