Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Which f1 should we report? #25

Open
soodeh-nilforoushan opened this issue Dec 11, 2021 · 1 comment
Open

Which f1 should we report? #25

soodeh-nilforoushan opened this issue Dec 11, 2021 · 1 comment

Comments

@soodeh-nilforoushan
Copy link

When I run the code I got three f1 from different epochs. Which f1 should we report as a final f1 accuracy based on the paper?
this is the example of out put: epoch 5: dev_f1=0.8317046688382194, f1=0.818146568437379, best_f1=0.8185719859539602

@rinkstiekema
Copy link

The dev_f1 uses the validation dataset for evaluation, while f1 uses the test dataset. Lastly, best_f1 indicates the best f1 score evaluated against the test dataset.

Eventually, the model that is written to disk is simply the last checkpoint. Since you should report the f1 of this model, it's best to use f1.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants