🗼 Feature request: OpenAI tokens usage information returned in add and query methods.

[Panczo]

🚀 The feature

I was looking at docs and previous issues but couldn't find any information about consumed OpenAI tokens.
According to the OpenAI documentation, they return following information in they response:

  "usage": {
    "prompt_tokens": 5,
    "completion_tokens": 7,
    "total_tokens": 12
  }

It'd be very useful to have such information, as right now I cannot track/measure programmatically consumed OpenAI tokens.
So I imagine that if I'll put some option to query or add config, data about usage will be returned in the response, something like below:

result = naval_chat_bot.add("https://www.youtube.com/watch?v=3qHkcs3kG44", config=add_config)
result['source'] => some source id
result['usage'] => usage data info

result = test_chat_bot.query('some happy little query', )
result['response_message'] => 'generated response'
result['usage'] => usage data info

It can be even simplified using some kind of proxy flag, so the whole OpenAI response will be returned.

Motivation, pitch

I'm working on a web app, where each user can have his own sources db. I don't want to force someone to put/use his own OpenAI api key, instead I want to control user tokens usage programmatically.

[cachho]

Good idea, this should definitely be on the list of things to add, however this isn't a quick thing to introduce so I must ask for some patience. thanks.

[Panczo]

Hi @cachho any news about this feature?

[Panczo]

👍