Generating random outputs after some epochs of Training

[jackwardphines]

I am training a CycleGAN on my own data set of 10000 tables. Input is a natural table and output should be the skeleton of the table. For the skeleton I'm generating random table skeletons using a python script. The training went OK through first 5-8 epochs and after that it is generating some random tables with random noise. Also the GAN_A loss is oscillating so much and my batch size is 4. Does cycle GAN become worse after few epochs of training or Am I doing something wrong?. Attached the loss function with respect to the number of epochs(25).

[junyanz]

It's hard to tell. Could you try batch_size=1? Also, not sure if you can transfer a table into a randomly generated skeleton. The quality of the translation will depend on how you generate random skeletons.

[Vaishnvi]

Hi @junyanz ,

I have been training cyclegan on my own dataset of around 500 images of A and 500 images of B.
But after few epochs I see that there’s random noise added to the fake images being generated while training and after 60 epochs now it’s still having some noise in generated images.
But I see that g_loss is minimising well for 60 epochs as of now.
Is there anything that’s going wrong in training ?

[junyanz]

The training looks normal to me. The noise might go away if the model is trained longer. If not, you can also look at #64

[Vaishnvi]

The training looks normal to me. The noise might go away if the model is trained longer. If not, you can also look at #64

@junyanz Thank you so much for the response !
#64 seems very similar to the issue I am facing in generated images as the checkerboard effect.
Will check with that solution :)

Also can you please answer one more query , do I need to change no of epochs if dataset is small ? As in original implementation suppose for horse2zebra dataset 200 epochs are used but my dataset is around 500 images for A(underwater images) and 500 for B(plastic bottles and plastic bags - not underwater).

Also since I was getting more noise in generated images I just tried testing with weights for first 10 epochs and generated images were quite smooth and identifiable compared to 100th or 200th epochs weights which were not easily identifying the plastic bottle/bag as in my dataset for B.

[junyanz]

Feel free to change the number of epochs. I think the number of epochs depends on your datasets and applications, and it is hard to tell in advance.

[Vaishnvi]

Yeah will check. Thanks a lot for help :)
Also your work and repo is really very helpful !
Thanks :)