Fixes to JSON evaluators

[mmabrouk]

Closes: AGE-1016 and AGE-1017

Both contains_json and json_diff were not functioning correctly. I refactored the code, removing validate_json (which was unnecessary—more on that later), and fixed two bugs:

1. We were throwing an exception if the output wasn't JSON, contradicting the evaluator's purpose of scoring based on whether the output is JSON.
2. We weren't parsing strings as dictionaries for the evaluator.

On validate_json:

• Methods should do one thing and one thing only. In this case it was a function that created a string json if the input is a dict and validated the input if it is a string
• Methods should have a name that describe what they do. The method did not validate jsons, it did something else
• Whenever we have a method that take two possible data types (string or dict), usually the code is not well written.

QA

• This PR fixes the evaluators contains_json and diff_json. It does not touch the logic of the second but only the inputs.
• Please test the following:
  -- Whether the two evaluators work both in no-code evaluation and in the evaluator playground. Please make sure to look very well at the results and see whether they make sense at all configuration
  -- Please test the evaluators with different input mapping (using testset.column for correct answer and correct answer directly) and both from the evaluation playground and from no-code evaluation

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
agenta	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Oct 7, 2024 9:08pm
agenta-documentation	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Oct 7, 2024 9:08pm

[aybruhm]

Thank you for the fix (and the awesome work), @mmabrouk!