[Feature] Return token usage

[kieranx]

In the official OpenAI API, token usage is returned, e.g. "total_tokens": 1340

There seems to be no way to get this same info in LLPhant -- this is very basic stuff and I feel like it should be able to be accessed, given OpenAI do return that info? We can't figure out a way to do that in LLPhant currently however.

Edit: to be clear, this is for the questionanswering class, I know we can get it for the general chat class.

[f-lombardo]

What about

LLPhant/src/Chat/OpenAIChat.php

Line 74 in fea142b

public function getTotalTokens(): int

? Does it fit your needs?

[Vardan9898]

Is there a way to retrieve the total number of tokens used with Question Answering?

$qa = new QuestionAnswering(
$memoryVectorStore,
$embeddingGenerator,
new OpenAIChat()
);

$answer = $qa->answerQuestion('What is the secret of Alice?');

If not, is it possible to use OpenAIChat with vector-stored embeddings?

[f-lombardo]

Maybe you can try this:

    $chat = new OpenAIChat(new OpenAIConfig());
    $qa = new QuestionAnswering(
        $filesVectorStore,
        $embeddingGenerator,
        $chat
    );

    $response = $qa->answerQuestion('What is the secret of Alice?');

    echo('Total tokens: ' .$chat->getTotalTokens());

[MaximeThoonsen]

You want the total of token for the question embeddings + all generations?

[MaximeThoonsen]

I started a small PR here.
We need to add the count of embeddings to make it correct