Resuming a GroupChat

[marklysze]

2024-05-11 - Update: The resume functionality no longer incorporates initiate_chat and, instead, sets up the group chat objects to be ready for resuming using initiate_chat afterwards.

Why are these changes needed?

It's not possible to currently resume a group chat. This PR includes the ability to resume a group chat using a list of messages.

The group chat can be resumed as follows:

1. Agents, GroupChat, and GroupChatManager are created in line with the original group chat
2. A set of messages is available (mandatory)
3. run resume_chat on the manager object, passing in the set of messages as well as any other parameters (that will then be passed through to ConversableAgent's initiate_chat.

The resume function does the following:

1. If messages is passed in as a JSON string, converts to a List[Dict] messages object
2. Sends provided messages around to each agent
3. Remove any provided termination string from the last message (optional)
4. Checks if the last message in the provided messages will invoke termination, warning the user if so
5. Writes out a message noting the resumption, how many messages have been provided, and which was the last agent
6. Calls ConversableAgent's initiate_chat on the last speaker to kick-off the resuming conversation.
7. Note that the GroupChat's max_round property is still utilised and does not include the original messages (e.g. if max_round is 3 it will resume for 3 more rounds)

The changes include:

• Updated autogen/agentchat/groupchat.py
• Updated test/agentchat/test_groupchat.py
• New documentation under topics/groupchat - User Guide > GroupChat > Resuming GroupChat

Why does it use ConversableAgent's initiate_chat instead of the manager's run_chat?
initiate_chat will call the manager's run_chat as part of the manager's attached reply functions. Additionally, initiate_chat provides the ability to set silent and supports summarisation.

Serialising state
The only state required is the messages. This in straightforward to serialise and persist. New functions on GroupChatManager, messages_to_string and messages_from_string will assist with the serialisation and deserialisation, respectively.

Validating the group chat
As we only use messages as input, it can provide basic validation that the name in each message aligns with the group chat's agents. This doesn't guarantee that the original group chat setup is the same as the one used for resuming.

Similarly, the developer will need to create the group chat objects themselves, rather than having it restored from state.

Future work can be done to increase the state to include all the group chat objects.

Testing in notebooks
I've run the resume functionality using the groupchat notebooks. I've saved these here.

Automated testing the group chat
I have not included testing the group chat resume in full in the test cases. I'd like to test this but I'm not sure how to test it effectively using LLMs/OpenAI - any help here is appreciated.

Notes:

• As group chats vary considerably, it would be beneficial for others to test this in their own group chats. My testing has included the documentation notebooks and my debating scenario. This includes text-based chats including python code execution. I have not tested in non-text scenarios.

Related issue number

This PR has been created from the need identified in issue #2359 and is associated with the discussion #2301.

Aligned with Roadmap #2358.

Checks

• I've included any doc changes needed for https://microsoft.github.io/autogen/. See https://microsoft.github.io/autogen/docs/Contribute#documentation to build and test documentation locally.
• I've added tests (if relevant) corresponding to the changes introduced in this PR.
• I've made sure all auto checks have passed.

[marklysze]

Okay, added further tests and added to the notebook with termination and resume examples.

Please note that resume needs to start from the manager to maintain the conversation as a group chat, otherwise if you start from another agent it looks to be just an agent-to-agent chat.

Note here

[aswny]

@ekzhu @marklysze first of all, thank you so much for this PR! I have a query.

In common groupchat scenario of multiple agents, the chat may terminate due to any agent's termination message and the termination condition/keyword of each agent may be different.

Think of an example like this:

 math_solver = autogen.AssistantAgent(
    ..,
    system_message="Answer only if user query is related to mathematics, otherwise say 'TERMINATE'",
    is_termination_msg="TERMINATE",
  )

  critic = autogen.AssistantAgent(
    ...,
    system_message="Think step-by-step and analyze the solution. If everything looks good, say 'APPROVED'",
    is_termination_msg="APPROVED",
  )

Now, when resuming the chat, the GroupChatManager.resume method does allow a remove_termination_string but the user isn't sure which termination string caused termination.

In this particular case, I think maintaining a map of termination strings for each agent would work; it can then be mapped to termination_strings_map[last_agent].

But, do you think allowing the remove_termination_string to be a function, similar to is_termination_msg in AssistantAgent, would help resolve such complicated scenarios better?

[marklysze]

Hey @aswny, thanks for your thoughts and identifying the use-case.

I see a couple of options:

1. Remove the termination messages from the messages state JSON
2. Update the resume function to allow the parameter to take in a function, a list of termination strings, or the agent-termination mapping

I think the function idea provides the most flexibility, though the list of termination strings may also work for the majority of use-cases.

Creating a new PR would be a good idea to move this along.