[Enhancement]: add integration with litellm for observability

[aybruhm]

Description

This PR allows litellm integration for observability.

Related Issue

Closes #1582

Additional Information

The example demonstrating LiteLLM integration for observability is titled "litellm_integration". This example is included in the pull request.

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
agenta	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Jun 6, 2024 2:13pm

[mmabrouk]

Closes #1569

[aybruhm]

The input for this span is totally wrong. Let's look at how others process the data from litellm and copy it ( a note here, everyone seems to be using the response_obj while we are using kwargs, why?)

They're not. Check out this and this. The kwargs in log_pre_api_call contain the prepared data for the LLM provider. We only have access to response_obj after receiving a response from the LLM provider.

re: @mmabrouk

[mmabrouk]

The input for this span is totally wrong. Let's look at how others process the data from litellm and copy it ( a note here, everyone seems to be using the response_obj while we are using kwargs, why?)

They're not. Check out this and this. The kwargs in log_pre_api_call contain the prepared data for the LLM provider. We only have access to response_obj after receiving a response from the LLM provider.

re: @mmabrouk

You are right. Sorry for missing that. However, the input for the span in the screenshot were wrong. It could be that the format that we are logging then is wrong (related to the issue we discussed this morning).

[aybruhm]

The input for this span is totally wrong. Let's look at how others process the data from litellm and copy it ( a note here, everyone seems to be using the response_obj while we are using kwargs, why?)

They're not. Check out this and this. The kwargs in log_pre_api_call contain the prepared data for the LLM provider. We only have access to response_obj after receiving a response from the LLM provider.
re: @mmabrouk

You are right. Sorry for missing that. However, the input for the span in the screenshot were wrong. It could be that the format that we are logging then is wrong (related to the issue we discussed this morning).

Yup, correct. This has been resolved.

[mmabrouk]

Thanks @aybruhm ! Great work!