Bugfix/code-evaluators

[mmabrouk]

Closes #1749

Issues:

• We were incorrectly calling the execute_code_safely function with the wrong parameters (data_point instead of correct_answer).

Solutions:

• The code should ideally have access to the whole row, not just the correct_answer. However, there are pre-existing eval functions with a signature that expects the correct_answer.
• To address this, I created a solution that is backward compatible.
• I have updated the example to demonstrate a snippet with the new signature that expects data_point.

Another issue was that the frontend had no way to know which ground truth to display in the new architecture. To resolve this:

• I added a correct_answer parameter in the evaluator configuration. This parameter is used solely to inform the frontend which column to display on the results page.
• In the future, this could be improved to enable the display of multiple or a list of ground truth columns.

Quality Assurance:

• I tested the PR locally using both an old evaluator with the old configuration and a new evaluator with the new signature.
• I also tested a failing evaluator to verify the changes made to the results display.

Additional Testing Requirements:

• Test this in the staging environment with both settings.
• Ensure the correct_answer column is correctly displayed in the results.
• Test changing the name of the correct_column, using a code that uses the new name, and verifying whether it is correctly displayed in the results.

Note: Migration scripts will need to be updated to modify the code evaluator and add correct_answer.

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
agenta	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Jun 3, 2024 1:02pm

[aakrem]

lgtm thanks for the fix @mmabrouk