adding the end2end rag task #953
Conversation
| @@ -416,3 +416,144 @@ | |||
| add_to_catalog( | |||
| metric, f"metrics.rag.response_generation.{axis}.{base_metric}", overwrite=True | |||
| ) | |||
|
|
|||
| ############################################# | |||
There was a problem hiding this comment.
i think it would be nice to have any new answer correctness and context correctness metrics in these modules:
https://github.com/IBM/unitxt/blob/main/prepare/metrics/rag_answer_correctness.py
https://github.com/IBM/unitxt/blob/main/prepare/metrics/rag_context_correctness.py
splitting up the metrics across modules makes things easier to follow, and allows reuse of the testing code that exists in the above modules.
| ############################################# | ||
|
|
||
| end_to_end_artifact_names = [ | ||
| TaskRagEndToEndConstants.METRICS_RAG_END_TO_END_ANSWER_CORRECTNESS, |
There was a problem hiding this comment.
Can these names be shortened?
For example, consider:
RagEndToEnd.ANSWER_CORRECTNESS
so RagEndToEnd is not repeated twice, being a constant is implied from the use of UPPER_CASE, Task is implied from RagEndToEnd and Metric is implied from ANSWER_CORRECTNESS.
Would makes things much more readable IMO.. WDYT?
No description provided.