How can i compute the token count？

[krisice]

Your issue may already be reported!
Please search on the issue tracker before creating one.

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

[vvatanabe]

@krisice It seems that compute the token count has already been discussed. Please check #62.

[FYI] Good example of how to count tokens:

• https://github.com/pkoukk/tiktoken-go#counting-tokens-for-chat-api-calls
• https://github.com/openai/openai-cookbook/blob/main/examples/How_to_count_tokens_with_tiktoken.ipynb

[vvatanabe]

Please continue the discussion here. #62.