Skip to content

v0.9.6 - fix Llama 3 tokenizer

Latest
Compare
Choose a tag to compare
@kddubey kddubey released this 02 Nov 02:45
· 1 commit to main since this release
cdf8275

Breaking changes

None

New features

None

Bug fixes

  • cappr.huggingface is compatible with Llama 3/3.1's tokenizer. It works around this issue using code from this PR (with small modifications). See the updated list of supported architectures here.