Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Token usage is not counting llm nodes in iteration node #11221

Closed
5 tasks done
fdb02983rhy opened this issue Nov 29, 2024 · 2 comments
Closed
5 tasks done

Token usage is not counting llm nodes in iteration node #11221

fdb02983rhy opened this issue Nov 29, 2024 · 2 comments
Labels
🐞 bug Something isn't working

Comments

@fdb02983rhy
Copy link
Contributor

Self Checks

  • This is only for bug report, if you would like to ask a question, please head to Discussions.
  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
  • [FOR CHINESE USERS] 请务必使用英文提交 Issue,否则会被关闭。谢谢!:)
  • Please do not modify this template :) and fill in all the required fields.

Dify version

0.12.1

Cloud or Self Hosted

Cloud, Self Hosted (Docker)

Steps to reproduce

test.yml.zip

✔️ Expected Behavior

No response

❌ Actual Behavior

image
@dosubot dosubot bot added the 🐞 bug Something isn't working label Nov 29, 2024
Copy link

dosubot bot commented Nov 29, 2024

I found a similar closed issue titled "Upgrade to 0.8.3, metadata still lacks the return of token usage information" which was resolved by a pull request that fixed an assignment exception [1].

In the Dify codebase, token usage tracking within iteration nodes is handled by including the graph_engine.graph_runtime_state.total_tokens attribute in the metadata of events like IterationRunSucceededEvent and IterationRunFailedEvent. This ensures that the total number of tokens used is recorded as part of the node's execution metadata [2].

If token usage is not being counted for LLM nodes in iteration nodes, it might be worth checking if the implementation aligns with this approach and if there are any discrepancies in how the events are being generated or handled.

To continue talking to Dosu, mention @dosu.

yihong0618 added a commit to yihong0618/dify that referenced this issue Nov 30, 2024
@laipz8200
Copy link
Member

@Nov1c444 Could you please review this issue and the refered solution?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🐞 bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants