Failed to calculate number of tokens, falling back to approximate count TypeError: fetch failed

[sapana-k]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

const promptTokens = await this.llm.getNumTokens(prompt);

Description

llm.getNumTokens() function is causing error in call stack, any idea how this can be avoided?
I cant update langchain versions because there are lots of dependencies
I've tried -
updating js-tiktoken version, didnt work
using js-tiktoken directly to get token length is not feasible

System Info

"langchain": "0.1.32"
Node.js v22.9.0
"js-tiktoken": "^1.0.14"

[dosubot[bot]]

I found several similar issues related to the error "Failed to calculate number of tokens, falling back to approximate count TypeError: fetch failed" in the LangChain library.

1. Issue #2668: This issue suggests upgrading LangChainJS to version 0.0.76 or newer, which replaces the WASM tiktoken package with a pure-JS version. This change should resolve the token calculation issue ^[1].

2. Issue #2493: This issue also recommends ensuring that the model name passed to the encodingForModel function is valid and that the js-tiktoken library is properly installed and imported. Upgrading to LangChainJS version 0.0.76 or newer is suggested as a solution ^[2].

Since you mentioned that you cannot update LangChain due to dependencies, you might want to check if you can update just the js-tiktoken package to ensure it's properly installed and configured. However, if updating LangChain is not an option, you may need to explore other workarounds, such as manually handling token calculations or adjusting your code to handle the fallback more gracefully.

To continue talking to Dosu, mention @dosu.

[sapana-k]

I have tried updating js-tiktoken but it doesnt work, manually handling token calculations using js-tiktoken is not feasible, any other way to resolve this issue? or any insights on why this is failing?

[dosubot[bot]]

The llm.getNumTokens() function in LangChainJS attempts to use the Tiktoken library for tokenization. If it fails, it falls back to an approximate calculation by dividing the content length by 4 ^[1][2]. The error you're encountering might be due to issues with the Tiktoken library or its integration.

Since updating js-tiktoken didn't work and manual handling isn't feasible, consider these options:

1. Check Network and Environment: Ensure there are no network issues or environment restrictions that might be causing the fetch operation to fail.

2. Verify Installation: Double-check that js-tiktoken is correctly installed and that there are no version mismatches or conflicts with other dependencies.

3. Fallback Handling: Enhance your code to handle the fallback scenario more gracefully. This might involve logging the error and using the approximate count as a temporary measure.

4. Alternative Tokenization: If possible, explore other tokenization libraries that might be compatible with your setup and can be used as a temporary workaround.

These steps might help mitigate the issue until you can update LangChain or find a more permanent solution.

To continue talking to Dosu, mention @dosu.