Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AgentEval Offline Integration #2345

Closed
wants to merge 28 commits into from
Closed

Conversation

jluey1
Copy link
Collaborator

@jluey1 jluey1 commented Apr 10, 2024

Why are these changes needed?

Related issue number

See issue #2162

Checks

autogen/agenteval/task.py Outdated Show resolved Hide resolved
autogen/agenteval/agent_eval.py Outdated Show resolved Hide resolved
autogen/agentchat/contrib/critic_agent.py Outdated Show resolved Hide resolved


@pytest.mark.skipif(
skip_openai,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To run this test in CI, we need to (1) create a new PR from a branch in the upstream repo; (2) add a section in contrib-openai.yml to run the test.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sonichi can you clarify what you mean by (1)? is this a new PR entirely or just merging in changes from a different branch?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A new PR with the same changes as in this PR.

test/agentchat/contrib/agent_eval/test_criterion.py Outdated Show resolved Hide resolved
autogen/agentchat/contrib/critic_agent.py Outdated Show resolved Hide resolved
autogen/agentchat/contrib/quantifier_agent.py Outdated Show resolved Hide resolved
autogen/agentchat/contrib/critic_agent.py Outdated Show resolved Hide resolved
notebook/agenteval_cq_math.ipynb Outdated Show resolved Hide resolved
autogen/agenteval/agent_eval.py Outdated Show resolved Hide resolved
autogen/agenteval/agent_eval.py Outdated Show resolved Hide resolved


@pytest.mark.skipif(
skip_openai,
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sonichi can you clarify what you mean by (1)? is this a new PR entirely or just merging in changes from a different branch?

Copy link
Contributor

@julianakiseleva julianakiseleva left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me

@jluey1
Copy link
Collaborator Author

jluey1 commented May 13, 2024

Closing in order to make a new PR from agenteval_integration into main

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
evaluation Issues related to evaluating the framework
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants