Improve error handling in evaluation

[bekossy]

No description provided.

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
agenta	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Jun 27, 2024 3:05pm

[bekossy]

When the output returns an OpenAi exception, it should be returned as anerrorobject with message and/or stacktrace properties and not text. The frontend will handle the error by displaying the information in a modal. The message property will be shown in red text, and the stacktrace will be displayed in a code block.
(cc @aybruhm @mmabrouk)

For Instance

[bekossy]

Ways to reproduce errors in the Evaluation:
(cc @mmabrouk @aybruhm)

• Create a testset with very long context

• Create Code evaluator with bad code

• Create App with invalid/incorrect API Key

• Enabling Force json
  Backend returned 500 Internal Server error
  Response: {detail: "'NoneType' object is not iterable"}

[aakrem]

@bekossy can we fix the conflicts please

[jp-agenta]

[Not blocking]
There is a change that this could be used for LLM App errors too, not just Evaluations errors, right ?