[Feature Request]: llm_config with input_price and output_price

[WebsheetPlugin]

Is your feature request related to a problem? Please describe.

#2424

As mentioned above, the issue is that Autogen has issues with new models. As all the model pricings are hardcoded. Curently you need to get the latest version to fix this. But you might not always want to get the latest version.

Describe the solution you'd like

Solution Idea A) Propose to allow setting prompt/completion
prices via llm_settings. See here:

{
"model": "gpt-3.5-turbo-1106",
"api_key": "sk-xxx",
"max_tokens": 4000,
"timeout": 100,
"max_retries": 2,
"input_cost" : 10, # USD per mil; alt naming prompt_tokens_cost
"output_cost" : 30 # USD per mil; alt naming completion_tokens_cost
}

It might make sense to also add vision pricing this way then:

Solution Idea B) It seems that curently Autogen is hardcoding the price for each model type
separately. But if it's not present, then you should simply compare the
prefix. So in this case, I believe the prefix "gpt-4-turbo" is enough to
understand what price range it is.

Additional context

No response

[Hk669]

Feel free to make a PR. Excited to see it! @WebsheetPlugin

[WebsheetPlugin]

@Hk669 I can do B) as it's very easy to do and in my opinion a better solution compared to how it's handled now, but @sonichi says it's a temporary or hacky solution. So not sure if I should proceed.

[Hk669]

@Hk669 I can do B) as it's very easy to do and in my opinion a better solution compared to how it's handled now, but @sonichi says it's a temporary or hacky solution. So not sure if I should proceed.

@WebsheetPlugin sure give it a try. we'll have support from @sonichi too.

[afourney]

So PR #1682 has a partial implementation of this. Can you have a look and see if it is useful here? Maybe revive it?