Code

fix pylist#1673

Code

Merged

ArthurZucker merged 7 commits intomainhuggingface/tokenizers:mainfrom fix-pylisthuggingface/tokenizers:fix-pylistCopy head branch name to clipboardNov 5, 2024

+30-28

Commits

Commits on Nov 5, 2024

• fix pylist

  ArthurZucker

  committed
  d50f9ae

• add comment about why we use PySequence

  ArthurZucker

  committed
  c2dc6b0

• style

  ArthurZucker

  committed
  495d430

• fix encode batch fast as well

  ArthurZucker

  committed
  8c96f47

• Update bindings/python/src/tokenizer.rs

  Show description for e1169ee
  

  ArthurZucker

  and

  Narsil

  authored
  e1169ee

• fix with capacity

  ArthurZucker

  committed
  6056e0c

• stub :)

  ArthurZucker

  committed
  2da036e