Support for OpenAI's cached input tokens

[anatolec]

Describe the feature or potential improvement

Since Oct 1st 2024, OpenAI introduced a caching feature where common prefixes of successive prompts are cached resulting in a cost and latency decrease. The usage field in OpenAI's completion API response contains the detail of the number of cached tokens.

It would be ideal that LangFuse takes into account this cost discount to reflect the real cost.

Additional information

No response

[marcklingen]

Thanks for raising this! This is work in progress as we overhaul the cost/usage tracking in Langfuse to be very flexible and adapt to new pricing models, cc @hassiebp who is working on this