Can't resume training from a preview checkpoint

[oxmah]

Hello guys !

It maybe an easy question. When I close then try to relaunch the training, it's start from 0 again. Is that a bug or is there something I'm missing ? I didn't find a command from (re)pointing a checkpoint to start from.

Thanks for your answer and sorry if it's a silly question.

[rjpeart]

@mathieuarbezhermoso did you resolve this? I'd also like to know how to continue from a particular checkpoint.

[danieleghisi]

@mathieuarbezhermoso I'd like to know this too!

[danieleghisi]

I think I managed to do that, it should be relatively simple. I've added netD and netG options in the opt = { ... } portion of main.lua, and then simply

if (opt.netD ~= '') then
  print('Initializing discriminator network from' .. opt.netD)
  netD = torch.load(opt.netD)
else 
  netD = nn.Sequential()
 ....
end

and same thing for opt.netG.
That should work, but you should hard code the epoch count by hand if you need to change it; also not sure about how the adam learning rates will behave, but I assume they'll be just reset as if you started from scratch.

[rjpeart]

@danieleghisi oh great! I'm gonna try this tonight. Thanks for sharing your findings!

[maryam089]

@rjpeart did you able to update that solution in the code ? If yesh then kindly help me too

[rjpeart]

Sadly no, I didn’t try this in the end.

…

On Mon, 25 Dec 2017 at 14:42, Maryam Shah ***@***.***> wrote: @rjpeart <https://github.com/rjpeart> did you able to update that solution in the code ? If yesh then kindly help me too — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#62 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACy3y8k0kqIKZI5K5NCLy9t2cziYI-DEks5tD7RNgaJpZM4LiL6W> .

[maryam089]

Thanks so much for quick response. Well i am stuck at resuming epoch from a certain value. Could you explain me the solution mentioned above? I would be extremely thankful to you.

[rjpeart]

I would love to be able to help, but I’m not skilled at this! It would take me a long time to work it out through trial-and-error. Hope you find your way through.

…

On Mon, 25 Dec 2017 at 15:15, Maryam Shah ***@***.***> wrote: Thanks so much for quick response. Well i am stuck at resuming epoch from a certain value. Could you explain me the solution mentioned above? I would be extremely thankful to you. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#62 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACy3yy81t11hv6iuTQPIPSD0CLTLstJUks5tD7wegaJpZM4LiL6W> .

[maryam089]

Yeah sure well I found one solution... will check it first thing in the morning (since its 2:30 am at my place).Anyway I truly appreciate your concern and kind words. Thank you so much !