-
-
Notifications
You must be signed in to change notification settings - Fork 609
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add no improvement handler (similar to early stopping handler) #2314
Comments
@arisliang Thank you for this FR ! Could we imagine that the actual |
@sdesrozis Thanks for the response. Yes exactly. I'd imagine I haven't thought about other trending policies. It's certainly possible as another enhancement to make things even more general. |
Not sure which is more suitable, |
@arisliang I will try to write a piece of code to illustrate my thoughts. |
Ok so the actual class EarlyStopping:
def __init__(self, patience, score_function, trainer):
self.score_function = score_function
self.patience = patience
self.trainer = trainer
self.counter = 0
self.best_score = float('-inf')
def __call__(self, engine):
score = self.score_function(engine)
if score <= self.best_score:
self.best_score = score
self.counter += 1
print("EarlyStopping: %i / %i" % (self.counter, self.patience))
if self.counter >= self.patience:
print("EarlyStopping: Stop training")
self.trainer.terminate()
else:
self.best_score = score
self.counter = 0 We could think about a more flexible implementation based on a class TriggerHandlerState:
def __init__(self):
self.best_score = float('-inf')
self.counter = 0
class TriggerHandler:
def __init__(self, score_function, pass_function, stop_function):
self.score_function = score_function
self.pass_function= pass_function
self.stop_function = stop_function
self.state = TriggerHandlerState()
def __call__(self, engine):
score = self.score_function(engine, self.state)
if self._update_and_check_state(score):
self.stop_function(self.state)
else:
self.pass_function(self.state)
def _update_and_check_state(self, score):
if score <= self.state.best_score:
self.state.best_score = score
self.state.counter += 1
return True
else:
return False What do you think ? |
Looks great for both current need and flexible. What is |
Yeah, a typo. Fixed. Let's see whether this design works although the naming, concepts and scope need to be carefully considered. |
Hi @sdesrozis and @arisliang, I am interested in taking this up. I was wondering, should we also keep the decision function of the Base class fixed as |
🚀 Feature
To add a 'no-improvement' handler. It's similar to early stopping, except that early stopping calls the training to terminate, 'no-improvement' handler would've calls a function that user pass in. User could pass in different functions that they want to execute if they find no improvements is made. They may want to try various things before terminate the training.
The text was updated successfully, but these errors were encountered: