Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Improve
PreTrainedTokenizerFast
loading time when there are many ad…
…ded tokens (#31404) * use hash * use hash * update --------- Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
- Loading branch information