Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(openai): add token usage and cost tracking callback #349

Merged
merged 3 commits into from
Jul 10, 2023

Conversation

mspronesti
Copy link
Contributor

@mspronesti mspronesti commented Jul 9, 2023

Hi @gventuri,
this PR aims at supporting token usage count and cost tracking for OpenAI llms, adapting and simplifying langchain's implementation.

Supposed usage:

"""Example of using PandasAI with a Pandas DataFrame"""

import pandas as pd
from data.sample_dataframe import dataframe

from pandasai import PandasAI
from pandasai.llm.openai import OpenAI
from pandasai.helpers.openai_info import get_openai_callback

df = pd.DataFrame(dataframe)

llm = OpenAI()

# conversational=False is supposed to display lower usage and cost
pandas_ai = PandasAI(llm, enable_cache=False, conversational=True)

with get_openai_callback() as cb:
    response = pandas_ai(df, "Calculate the sum of the gdp of north american countries")

    print(response)
    print(cb)
#  The sum of the GDP of North American countries is 19,294,482,071,552.
#  Tokens Used: 375
#	Prompt Tokens: 210
#	Completion Tokens: 165
# Total Cost (USD): $ 0.000750

Of course it also works with AzureOpenAI.
Missing tests for now.

@mspronesti mspronesti changed the title feat(openai): add token and cost tracking callback feat(openai): add token usage and cost tracking callback Jul 9, 2023
@mspronesti mspronesti marked this pull request as ready for review July 10, 2023 14:36
@mspronesti
Copy link
Contributor Author

@gventuri Ready to go :)

@gventuri gventuri merged commit 24ac098 into Sinaptik-AI:main Jul 10, 2023
9 checks passed
@gventuri
Copy link
Collaborator

@mspronesti thanks a lot, just merged!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add support for token usage count
2 participants