Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rprop improvements #11

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Rprop improvements #11

wants to merge 2 commits into from

Conversation

abrock
Copy link

@abrock abrock commented Sep 7, 2016

I added some functionality to the maximize() function in RProp:

  • Print the current set of hyperparameters in every iteration if the user wants that. Default behaviour is unchanged.

I checked the behaviour of the function, especially what happens if I allow a large number of iterations (200). I found that after the first 50-100 iterations the method gets stuck and no updates to the best_params vector occur.
I therefore implemented a cooldown strategy:

Merging this would make #9 and #10 obsolete.

  • If the new value of the target function is not better than the old decrease stepsize_factor which is multiplied with the calculated gp->log_likelihood_gradient. Initial value for stepsize_factor is 1.0 so for the first iterations the behaviour is unchanged.
  • If the stepsize is smaller than a threshold (default: 1e-2) stop

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant