Python: v0.2.3 - Update to latest tokenizer crates #47

benbrandt announced in Announcements

benbrandt
Sep 11, 2023
Maintainer

What's New

Update to v0.4.4 of text-splitter to support tokenizers@0.14.0
Update tokenizers and tiktoken-rs to latest versions

Full Changelog: python-v0.2.2...python-v0.2.3

This discussion was created from the release Python: v0.2.3 - Update to latest tokenizer crates.

Replies: 0 comments

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment