-
Notifications
You must be signed in to change notification settings - Fork 408
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
chore(llmobs): ragas faithfulness evaluator #10748
Conversation
…py into evan.li/ragas-skeleton
) | ||
|
||
|
||
def test_evaluator_runner_periodic_enqueues_eval_metric(LLMObs, mock_llmobs_eval_metric_writer): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
⚪ Code Quality Violation
def test_evaluator_runner_periodic_enqueues_eval_metric(LLMObs, mock_llmobs_eval_metric_writer): | |
def test_evaluator_runner_periodic_enqueues_eval_metric(l_l_m_obs, mock_llmobs_eval_metric_writer): |
use snake_case and not camelCase (...read more)
Ensure that function use snake_case
.
This rule is not valid for tests files (prefixed by test_
or suffixed by _test.py
) because testing requires some camel case methods, such as, tearDown
, setUp
, and more.
Learn More
|
||
|
||
@pytest.fixture | ||
def LLMObsWithRagas(monkeypatch, mock_llmobs_span_writer, mock_llmobs_eval_metric_writer, ddtrace_global_config): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
⚪ Code Quality Violation
def LLMObsWithRagas(monkeypatch, mock_llmobs_span_writer, mock_llmobs_eval_metric_writer, ddtrace_global_config): | |
def l_l_m_obs_with_ragas(monkeypatch, mock_llmobs_span_writer, mock_llmobs_eval_metric_writer, ddtrace_global_config): |
use snake_case and not camelCase (...read more)
Ensure that function use snake_case
.
This rule is not valid for tests files (prefixed by test_
or suffixed by _test.py
) because testing requires some camel case methods, such as, tearDown
, setUp
, and more.
Learn More
|
||
|
||
@pytest.mark.vcr_logs | ||
def test_ragas_faithfulness_evaluator_timed_enqueues_eval_metric(LLMObs, mock_llmobs_eval_metric_writer): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
⚪ Code Quality Violation
def test_ragas_faithfulness_evaluator_timed_enqueues_eval_metric(LLMObs, mock_llmobs_eval_metric_writer): | |
def test_ragas_faithfulness_evaluator_timed_enqueues_eval_metric(l_l_m_obs, mock_llmobs_eval_metric_writer): |
use snake_case and not camelCase (...read more)
Ensure that function use snake_case
.
This rule is not valid for tests files (prefixed by test_
or suffixed by _test.py
) because testing requires some camel case methods, such as, tearDown
, setUp
, and more.
Learn More
@@ -1525,6 +1530,13 @@ def process_trace(self, trace): | |||
llmobs_service.disable() | |||
|
|||
|
|||
def test_llm_span_ragas_evaluator(LLMObsWithRagas): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
⚪ Code Quality Violation
def test_llm_span_ragas_evaluator(LLMObsWithRagas): | |
def test_llm_span_ragas_evaluator(l_l_m_obs_with_ragas): |
use snake_case and not camelCase (...read more)
Ensure that function use snake_case
.
This rule is not valid for tests files (prefixed by test_
or suffixed by _test.py
) because testing requires some camel case methods, such as, tearDown
, setUp
, and more.
Checklist
Reviewer Checklist