Skip to content
This repository has been archived by the owner on Nov 1, 2024. It is now read-only.

Randomly initialize multiclass label selection when switching to a new context. Make it stay the same otherwise. #904

Open
TristanThrush opened this issue Apr 21, 2022 · 0 comments

Comments

@TristanThrush
Copy link
Contributor

Right now, after you submit an example, the label selection randomly changes: (e.g. "contradictory" has a 1/3 chance of changing to neutral, etc.). See here: https://dynabench.org/tasks/nli/create

We want the label to be randomly initialized after switching to a new context, but we don't want it to be randomly initialized after every example submission. The label button is defined here:

https://github.com/facebookresearch/dynabench/blob/main/frontends/web/src/common/Annotation/AnnotationComponent.js#L67

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant