Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Evaluate model on best model when doing early stopping #524

Merged
merged 3 commits into from
Sep 7, 2020
Merged

Evaluate model on best model when doing early stopping #524

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
990ed06
Evaluate model on best model when doing early stopping
PhilipMay Sep 6, 2020
994155f
Add doc for return value of train
PhilipMay Sep 6, 2020
cda007f
Evaluate model on test
PhilipMay Sep 6, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions farm/train.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -240,6 +240,9 @@ def train(self):

If trainer evaluates the model with a test set the result of the
evaluation is stored in ``test_result``.

:return: Returns the model after training. When you do ``early_stopping``
with a ``save_dir`` the best model is loaded and returned.
"""

# connect the prediction heads with the right output from processor
Expand Down Expand Up @@ -352,8 +355,8 @@ def train(self):
if self.early_stopping and self.early_stopping.save_dir:
logger.info("Restoring best model so far from {}".format(self.early_stopping.save_dir))
lm_name = self.model.language_model.name
model = AdaptiveModel.load(self.early_stopping.save_dir, self.device, lm_name=lm_name)
model.connect_heads_with_processor(self.data_silo.processor.tasks, require_labels=True)
self.model = AdaptiveModel.load(self.early_stopping.save_dir, self.device, lm_name=lm_name)
self.model.connect_heads_with_processor(self.data_silo.processor.tasks, require_labels=True)

# Eval on test set
if self.evaluator_test:
Expand Down