Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make Word2vec multiple passes more explicit. Fix #1052 #1183

Closed
wants to merge 0 commits into from

Conversation

robotcator
Copy link
Contributor

I modify the method 'train()' by adding some parameters --iteration=None, epochs=1, start_alpha=None, end_alpha=None, cur_epoch=None. And the default epochs is one, others are None. If the user trains the model with a loop, the epochs and cur_epoch(the current epoch) must be specified. So if the user don't specify the start_alpha and end_alpha(the usage is the same as alpha and min_alpha), we can recaculate the start_alpha and end_alpha according to the current epoch to support linear learning-rate decay from (initial) alpha to min_alpha.

@robotcator
Copy link
Contributor Author

I also have another idea that we can force the user to pass those parameters(iter, alpha, min_alpha) explicitly. If the user train the model using a loop, they can carefully pass the start_alpha and end_alpha to support one long smooth descent from alpha to min_alpha.

@gojomo
Copy link
Collaborator

gojomo commented Mar 6, 2017

Another pending PR with same general idea is #1139 - forcing users to be aware of train()'s parameters by making them required.

@tmylk tmylk changed the title Make Word2vec multiple passes more explicit Make Word2vec multiple passes more explicit. Fix #1052 Mar 8, 2017
@tmylk
Copy link
Contributor

tmylk commented Mar 8, 2017

@robotcator Thanks for this contribution but it is best to close this PR and finish #1139 in particular add tests for assertRaises if the new epochs param is ommiteed; update the ipynbs that use train; implement epoch_callback,

@robotcator
Copy link
Contributor Author

@tmylk Thank you for your suggestion. But can you give more details about implementing epoch_callback ?
@gojomo Would you mind let me to finish the #1139 to do list.

@gojomo
Copy link
Collaborator

gojomo commented Mar 9, 2017

@robotcator - Please feel free to continue/finish #1139. In my opinion the most important thing is to ensure notebooks/docs reflect the change & current best-practices, and resolve any conflicts with the main develop branch.

Implementing epoch_callback would require some careful refactoring of the existing loops and multi-threading; I'm not sure of the best way to proceed yet, which is one of the reasons I put #1139 aside.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants