You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
My own task or dataset (give details below)
Reproduction
It is shown in the OPT model documentation (in Tips) that it is required to pass use_fast=False when loading a OPT tokenizer, as OPT tokenizer will add </s> to the beginning of every prompt.
System Info
platform==Ubuntu 18.04.01
python==3.10
transformers==4.29.1
Who can help?
@sgugger @stevhliu @mk
Information
Tasks
examples
folder (such as GLUE/SQuAD, ...)Reproduction
It is shown in the OPT model documentation (in Tips) that it is required to pass
use_fast=False
when loading a OPT tokenizer, as OPT tokenizer will add</s>
to the beginning of every prompt.I made a trial:
</s>
is correctly added and no difference is observed.Expected behavior
Is the tips wrong or in some other cases
use_fast=Fast
is actually required?The text was updated successfully, but these errors were encountered: