Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Graph group chat #753

Closed
wants to merge 30 commits into from
Closed

Graph group chat #753

wants to merge 30 commits into from

Conversation

joshkyh
Copy link
Contributor

@joshkyh joshkyh commented Nov 24, 2023

@BeibinLi, @thinkall, @afourney, @sonichi because of they have previously reviewed in the proof-of-concept in #720.

Why are these changes needed?

Implemented a graph mode for GroupChat, this helps having more agents working together, since the number of transition pairs increases rapidly (N Choose 2).

Functionalities

Graph Validity Checks

  • Ensures that the graph meets certain criteria:
  • Has at least one node and one edge.
  • Contains at least one node marked as 'first_round_speaker'.
  • Warns if there are isolated agent nodes (agents with no connections).
  • Warns if there are agents listed in self.agents who are not present in the graph.
  • Prohibits self-loops in the graph if allow_repeat_speaker is set to False.

Dynamic Speaker Selection

  • Determines the first round speaker based on the first_round_speaker attribute in the graph.
  • Chooses the next speaker based on the graph's successors of the previous speaker.
  • Incorporates suggestions from the last message using a special "NEXT:" keyword.
  • Employs language models (LLM) for speaker selection, if there is context in the messages.
  • Defaults to random selection as a fallback method.
  • Error Handling: Raises appropriate exceptions if the graph does not meet the required criteria or if no eligible speakers are found based on the graph constraints.

Related issue number

Closes #743

Checks

@codecov-commenter
Copy link

codecov-commenter commented Nov 24, 2023

Codecov Report

Attention: 65 lines in your changes are missing coverage. Please review.

Comparison is base (f654946) 27.79% compared to head (810faa5) 38.45%.
Report is 15 commits behind head on main.

Files Patch % Lines
autogen/agentchat/contrib/graphgroupchat.py 0.00% 59 Missing and 1 partial ⚠️
autogen/agentchat/groupchat.py 80.00% 4 Missing and 1 partial ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##             main     #753       +/-   ##
===========================================
+ Coverage   27.79%   38.45%   +10.66%     
===========================================
  Files          27       28        +1     
  Lines        3497     3643      +146     
  Branches      793      841       +48     
===========================================
+ Hits          972     1401      +429     
+ Misses       2454     2119      -335     
- Partials       71      123       +52     
Flag Coverage Δ
unittests 38.40% <23.52%> (+10.66%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Co-authored-by: Chi Wang <wang.chi@microsoft.com>
Copy link
Contributor

@sonichi sonichi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For testing, let's follow the same steps I suggested in #846: Create a PR from a branch in microsoft/autogen, disable other tests temporarily in this workflow and change pull_request_target to pull_request.

@joshkyh
Copy link
Contributor Author

joshkyh commented Dec 4, 2023

Create a PR from a branch in microsoft/autogen, disable other tests temporarily in this workflow and change pull_request_target to pull_request.

Thanks @sonichi for the guidance. Please bear with me, can I clarify that for this PR, I need to:
A1. Change this PR so that it is joshkyh:GraphGroupChat -> microsoft:GraphGroupChat.
A2. Change my .github/workflows/contrib-openai.yml so that I (a) comment out non-GraphGroupChat related tests. (b) change line 7 from pull_request_target -> pull_request.

Or is it:
B1. Close this PR. Start a new branch microsoft:GraphGroupChat.
B2: Bring my changes into microsoft:GraphGroupChat. Start a new PR from microsoft:GraphGroupChat -> microsoft:main
B3. Change my .github/workflows/contrib-openai.yml so that I (a) comment out non-GraphGroupChat related tests. (b) change line 7 from pull_request_target -> pull_request.

Thank you.

@sonichi
Copy link
Contributor

sonichi commented Dec 4, 2023

It's plan B.

@joshkyh joshkyh closed this Dec 4, 2023
joshkyh added a commit that referenced this pull request Dec 4, 2023
@joshkyh joshkyh mentioned this pull request Dec 4, 2023
3 tasks
github-merge-queue bot pushed a commit that referenced this pull request Feb 6, 2024
* Move contrib-openai.yml

* Moved groupgroupchat

* From #753

* Removed local test references

* Added ignore=test/agentchat/contrib

* Trying to pass contrib-openai tests

* More specific in unit testing.

* Update .github/workflows/contrib-tests.yml

Co-authored-by: Li Jiang <lijiang1@microsoft.com>

* Remove coverage as it is included in test dependencies

* Improved docstring with overview of GraphGroupChat

* Iterate on feedback

* Precommit pass

* user just use pip install pyautogen[graphs]

* Pass precommit

* Pas precommit

* Graph utils an test completed

* Added inversion tests

* Added inversion util

* allow_repeat_speaker can be a list of Agents

* Remove unnessary imports

* Expect ValueError with 1 and 0 agents

* Check that main passes all tests

* Check main

* Pytest all in main

* All done

* pre-commit changes

* noqa E402

* precommit pass

* Removed bin

* Removed old unit test

* Test test_graph_utils

* minor cleanup

* restore tests

* Correct documentation

* Special case of only one agent remaining.

* Improved pytest

* precommit pass

* Delete OAI_CONFIG_LIST_sample copy

* Returns a filtered list for auto to work

* Rename var speaker_order_dict

* To write test cases

* Added check for a list of Agents to repeat

* precommit pass

* Update documentation

* Extract names in allow_repeat_speaker

* Post review changes

* hange "pull_request_target" into "pull_request" temporarily.

* 3 return values from main

* pre-commit changes

* PC edits

* docstr changes

* PC edits

* Rest of changes from main

* Update autogen/agentchat/groupchat.py

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* Remove unnecessary script files from tracking

* Non empty scripts files from main

* Revert changes in script files to match main branch

* Removed link from website as notebook is removed.

* test/test_graph_utils.py is tested as part of L52 of build.yml

* GroupChat ValueError check

* docstr update

* More clarification in docstr

* Update autogen/agentchat/groupchat.py

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* Update autogen/agentchat/groupchat.py

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* Update autogen/agentchat/groupchat.py

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* Update autogen/agentchat/groupchat.py

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* 1.add commit to line138 in groupchat.py;2.fix bug if random choice [];3.return selected_agent if len(graph_eligible_agents) is 1;4.replace all speaker_order to speaker_transitions;5.format

* fix graph_modelling notebook in the last cell

* fix failure in test_groupchat.py

* fix agent out of group to initiate a chat like SocietyOfMind

* add a warning rule in graph_utils to check duplicates in any lists

* refactor allowed_or_disallowed_speaker_transitions to Dict[Agent, List[Agent]] and modify the tests and notebook

* delete Rule 4 in graph_utils and related test case. Add a test to resolve https://github.com/microsoft/autogen/pull/857/files/993fd006e922c8efe5e50bd0700e355994c6d337#r1460726831

* fix as the final comments

* modify setup option from graphs to graph and add texts in optional-dependencies.md

* Update autogen/graph_utils.py

---------

Co-authored-by: Li Jiang <lijiang1@microsoft.com>
Co-authored-by: Beibin Li <BeibinLi@users.noreply.github.com>
Co-authored-by: Chi Wang <wang.chi@microsoft.com>
Co-authored-by: Qingyun Wu <qingyun0327@gmail.com>
Co-authored-by: Yishen Sun <freedeaths@FREEDEATHS-XPS>
Co-authored-by: freedeaths <register917@gmail.com>
whiskyboy pushed a commit to whiskyboy/autogen that referenced this pull request Apr 17, 2024
* Move contrib-openai.yml

* Moved groupgroupchat

* From microsoft#753

* Removed local test references

* Added ignore=test/agentchat/contrib

* Trying to pass contrib-openai tests

* More specific in unit testing.

* Update .github/workflows/contrib-tests.yml

Co-authored-by: Li Jiang <lijiang1@microsoft.com>

* Remove coverage as it is included in test dependencies

* Improved docstring with overview of GraphGroupChat

* Iterate on feedback

* Precommit pass

* user just use pip install pyautogen[graphs]

* Pass precommit

* Pas precommit

* Graph utils an test completed

* Added inversion tests

* Added inversion util

* allow_repeat_speaker can be a list of Agents

* Remove unnessary imports

* Expect ValueError with 1 and 0 agents

* Check that main passes all tests

* Check main

* Pytest all in main

* All done

* pre-commit changes

* noqa E402

* precommit pass

* Removed bin

* Removed old unit test

* Test test_graph_utils

* minor cleanup

* restore tests

* Correct documentation

* Special case of only one agent remaining.

* Improved pytest

* precommit pass

* Delete OAI_CONFIG_LIST_sample copy

* Returns a filtered list for auto to work

* Rename var speaker_order_dict

* To write test cases

* Added check for a list of Agents to repeat

* precommit pass

* Update documentation

* Extract names in allow_repeat_speaker

* Post review changes

* hange "pull_request_target" into "pull_request" temporarily.

* 3 return values from main

* pre-commit changes

* PC edits

* docstr changes

* PC edits

* Rest of changes from main

* Update autogen/agentchat/groupchat.py

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* Remove unnecessary script files from tracking

* Non empty scripts files from main

* Revert changes in script files to match main branch

* Removed link from website as notebook is removed.

* test/test_graph_utils.py is tested as part of L52 of build.yml

* GroupChat ValueError check

* docstr update

* More clarification in docstr

* Update autogen/agentchat/groupchat.py

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* Update autogen/agentchat/groupchat.py

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* Update autogen/agentchat/groupchat.py

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* Update autogen/agentchat/groupchat.py

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* 1.add commit to line138 in groupchat.py;2.fix bug if random choice [];3.return selected_agent if len(graph_eligible_agents) is 1;4.replace all speaker_order to speaker_transitions;5.format

* fix graph_modelling notebook in the last cell

* fix failure in test_groupchat.py

* fix agent out of group to initiate a chat like SocietyOfMind

* add a warning rule in graph_utils to check duplicates in any lists

* refactor allowed_or_disallowed_speaker_transitions to Dict[Agent, List[Agent]] and modify the tests and notebook

* delete Rule 4 in graph_utils and related test case. Add a test to resolve https://github.com/microsoft/autogen/pull/857/files/993fd006e922c8efe5e50bd0700e355994c6d337#r1460726831

* fix as the final comments

* modify setup option from graphs to graph and add texts in optional-dependencies.md

* Update autogen/graph_utils.py

---------

Co-authored-by: Li Jiang <lijiang1@microsoft.com>
Co-authored-by: Beibin Li <BeibinLi@users.noreply.github.com>
Co-authored-by: Chi Wang <wang.chi@microsoft.com>
Co-authored-by: Qingyun Wu <qingyun0327@gmail.com>
Co-authored-by: Yishen Sun <freedeaths@FREEDEATHS-XPS>
Co-authored-by: freedeaths <register917@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Adding Graph Mode to GroupChat [GroupChat] [Enhancement]
5 participants