Use callbacks when using LLM with cache

[gramliu]

Integrations with observability platforms like LangSmith have been great for tracing calls to OpenAI. However, when a cache is specified, the callbacks (handleLLMStart, handleLLMEnd, and handleLLMError) aren't called unlike in their uncached counterparts.

I presume this is the case because the callbacks don't currently handle the case of cached responses, so there wouldn't be a way to tell if tokens were actually consumed or not. That being said, I think it would be pretty worthwhile to add this and even initially just specify that it's cached in the callback metadata. I would very much like to see the messages being generated on LangSmith and not just the calls to the tools.

Uncached

Cached

[dosubot]

🤖

It seems like you've got this under control, if you want help or have specific questions, let me know what I can do for you!

[gramliu]

This could likely be handled well by creating a _generateCached counterpart to the _generateUncached callbacks