This project implements token calculation for OpenAI's gpt-4 and gpt-3.5-turbo model, specifically using `cl100k_base` encoding.
encoding ai csharp tokens openai gpt4 chatgpt langchain tiktoken gpt35turbo cl100kbase tiktoken-sharp p50kbase langchain-dotnet
-
Updated
Dec 23, 2024 - C#