Python: v0.2.3 - Update to latest tokenizer crates

benbrandt released this 11 Sep 09:51

· 522 commits to main since this release

f92d9c3

What's New

Update to v0.4.4 of text-splitter to support tokenizers@0.14.0
Update tokenizers and tiktoken-rs to latest versions

Full Changelog: python-v0.2.2...python-v0.2.3

Assets 2

0 Join discussion