Struggling with predefined functions in group chat.

[nubgamerz]

I want to have LLM's access to predefined functions.
I've written those functions, and it works when there's no group chat.

llm_config = { "functions": [ { "name": "write_to_file", "description": "Use this function to write content to a file", "parameters": { "type": "object", "properties": { "filename": { "type": "string", "description": "The filename to write to", }, "content": { "type": "string", "description": "The content to write", } }, "required": ["filename", "content"], }, }, { "name": "read_from_file", "description": "Use this function to read the content of a file", "parameters": { "type": "object", "properties": { "filename": { "type": "string", "description": "The filename to read from", } }, "required": ["filename"], }, }, { "name": "read_pdf", "description": "Use this function to read the content of a pdf file", "parameters": { "type": "object", "properties": { "filename": { "type": "string", "description": "The filename to read from", } }, "required": ["filename"], }, }, { "name": "create_directory", "description": "Use this function to create a directory", "parameters": { "type": "object", "properties": { "directory_path": { "type": "string", "description": "The directory path to create", } }, "required": ["directory_path"], }, }, ], "config_list": config_list, "seed": 42, "request_timeout": 120 }

user_proxy = UserProxyAgent( name="user_proxy", system_message="A human that will provide the necessary information to the assistant", function_map={ "write_to_file": write_to_file, "read_from_file": read_from_file, "read_pdf": read_pdf, "create_directory": create_directory, }, code_execution_config={"work_dir": "fileread"})

assistant = AssistantAgent( name="assistant", system_message="""You are an assistant, you must blah blah""", llm_config=llm_config )

When initiating this, it works flawlessly. Functions are used properly.

But if I add another llm and a group chat

architect = AssistantAgent( name="architect", system_message="""You are a blah blah, and will get info from assistant etc etc""", llm_config=llm_config, )

groupchat = GroupChat( agents=[user_proxy, assistant, architect], messages=[],)

manager = GroupChatManager(groupchat=groupchat, llm_config=llm_config)

If I now initiate the chat to the manager, the assistant agent says the functions cannot be found
`assistant (to chat_manager):

***** Response from calling function "read_from_file" *****
Error: Function read_from_file not found.
***********************************************************`

Am I missing something here? Do I need to define functions somehow in the group chat? Because if I remove the manager and the group chat, it works fine.

[sonichi]

Could you try adding to the system_message for the UserProxyAgent: "Execute suggested function calls." or something similar to let the group chat manager know this agent needs to be used to execute the function call?

[nubgamerz]

Hey, thanks, tried that and still no luck

    name="user_proxy",
    system_message="A human that will provide the necessary information to the group chat manager. Execute suggested function calls.",
    function_map={
        "write_to_file": write_to_file,
        "read_from_file": read_from_file,
        "read_pdf": read_pdf,
        "create_directory": create_directory,
    },
    human_input_mode="ALWAYS",
    code_execution_config={"work_dir": "fileread"}) ```
    

[LittleLittleCloud]

Why predefined function call doesn't work in group chat

The predefined function doesn't work in group chat because in autogen, a function_call is not immediately triggered after it's being proposed. Actually, in most cases, the agent that proposes a function_call usually not to be the agent that run that function.

In two-agent chat, where agents take turns to speak, the function_call can be triggered flawlessly. Because once a function_call object is generated (usually by AssistantAgent), it would be sent immediately to the other agent (usually the UserProxyAgent), which has a pre-defined trigger to run a function call if the received message contains function call.

While in group chat, where a dynamic chat flow applies, the original trigger way for function_call doesn't work anymore. Because GroupChatManager doesn't have information on which agent should execute that function_call when a function_call object is received. And if it selects the wrong agent that doesn't have the right function executor, you will see the error message XXX function not found

Work-around: How to fix it.

@ilaffey2 (I hope I @ the right person) has an excellent solution where you can use to override the GroupChat in your code. Basically, the solution is 1) having a UserProxyAgent to hold the entire function_map which contains all function executors for agents in the group and 2) In select_speaker step, return that UserProxyAgent as the next speaker if the latest message is a function_call

from autogen import GroupChat, ConversableAgent, UserProxyAgent
from dataclasses import dataclass


@dataclass
class ExecutorGroupchat(GroupChat):
    dedicated_executor: UserProxyAgent = None

    def select_speaker(
        self, last_speaker: ConversableAgent, selector: ConversableAgent
    ):
        """Select the next speaker."""

        try:
            message = self.messages[-1]
            if "function_call" in message:
                return self.dedicated_executor
        except Exception as e:
            print(e)
            pass

        selector.update_system_message(self.select_speaker_msg())
        final, name = selector.generate_oai_reply(
            self.messages
            + [
                {
                    "role": "system",
                    "content": f"Read the above conversation. Then select the next role from {self.agent_names} to play. Only return the role.",
                }
            ]
        )
        if not final:
            # i = self._random.randint(0, len(self._agent_names) - 1)  # randomly pick an id
            return self.next_agent(last_speaker)
        try:
            return self.agent_by_name(name)
        except ValueError:
            return self.next_agent(last_speaker)

[nubgamerz]

Hey, yeah I tried this solution as it was recommended on Discord.

But there seems to be a bug.

groupchat = ExecutorGroupchat(
    agents=[user_proxy, assistant, architect], messages=[],max_round=20)

assistant (to chat_manager):

***** Suggested function Call: read_from_file *****
Arguments:
{
"filename": "instructions.txt"
}

---

AttributeError: 'NoneType' object has no attribute 'generate_reply

Though this could be just me not using the class correctly.

[LittleLittleCloud]

Can you share the complete code? From the code snippet you share,, it seems that you didn't pass the dedicated_executor to ExecutorGroupchat?

groupchat = ExecutorGroupchat(
    agents=[user_proxy, assistant, architect], messages=[],max_round=20)

[nubgamerz]

Sure, but I will need to leave out some prompts as it's work related.

from autogen import config_list_from_json, AssistantAgent, UserProxyAgent, GroupChat, GroupChatManager, Agent, ConversableAgent
import fitz
import os
from dataclasses import dataclass

@dataclass
class ExecutorGroupchat(GroupChat):
    dedicated_executor: UserProxyAgent = None

    def select_speaker(
        self, last_speaker: ConversableAgent, selector: ConversableAgent
    ):
        """Select the next speaker."""

        try:
            message = self.messages[-1]
            if "function_call" in message:
                return self.dedicated_executor
        except Exception as e:
            print(e)
            pass

        selector.update_system_message(self.select_speaker_msg())
        final, name = selector.generate_oai_reply(
            self.messages
            + [
                {
                    "role": "system",
                    "content": f"Read the above conversation. Then select the next role from {self.agent_names} to play. Only return the role.",
                }
            ]
        )
        if not final:
            # i = self._random.randint(0, len(self._agent_names) - 1)  # randomly pick an id
            return self.next_agent(last_speaker)
        try:
            return self.agent_by_name(name)
        except ValueError:
            return self.next_agent(last_speaker)

def read_pdf(filename):
    try:
        with fitz.open(filename) as pdf_document:
            pdf_text = ''
            for page_num in range(pdf_document.page_count):
                page = pdf_document[page_num]
                pdf_text += page.get_text()
            return pdf_text
    except FileNotFoundError:
        print(f'Error: {filename} not found.')
        return None
    except Exception as e:
        print(f'Error: Unable to read {filename}. {str(e)}')
        return None

def write_to_file(filename, content):
    try:
        with open(filename, 'w') as file:
            file.write(content)
        print(f'Content successfully written to {filename}.')
    except IOError:
        print(f'Error: Unable to write to {filename}.')

def read_from_file(filename):
    try:
        with open(filename, 'r') as file:
            content = file.read()
            return content
    except FileNotFoundError:
        print(f'Error: {filename} not found.')
        return None
    except IOError:
        print(f'Error: Unable to read {filename}.')
        return None
    
def create_directory(directory_path):
    try:
        os.makedirs(directory_path)
        print(f'Directory created: {directory_path}')
    except FileExistsError:
        print(f'Directory already exists: {directory_path}')
    except Exception as e:
        print(f'Error: Unable to create directory {directory_path}. {str(e)}')

config_list = config_list_from_json(env_or_file="OAI_CONFIG_LIST")

llm_config = {
    "functions": [
        {
            "name": "write_to_file",
            "description": "Use this function to write content to a file",
            "parameters": {
                "type": "object",
                "properties": {
                    "filename": {
                        "type": "string",
                        "description": "The filename to write to",
                    },
                    "content": {
                        "type": "string",
                        "description": "The content to write",
                    }
                },
                "required": ["filename", "content"],
            },
        },
        {
            "name": "read_from_file",
            "description": "Use this function to read the content of a file",
            "parameters": {
                "type": "object",
                "properties": {
                    "filename": {
                        "type": "string",
                        "description": "The filename to read from",
                    }
                },
                "required": ["filename"],
            },
        },
        {
            "name": "read_pdf",
                "description": "Use this function to read the content of a pdf file",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "filename": {
                            "type": "string",
                            "description": "The filename to read from",
                        }
                    },
                "required": ["filename"],
            },
        },
        {
            "name": "create_directory",
            "description": "Use this function to create a directory",
            "parameters": {
                "type": "object",
                "properties": {
                    "directory_path": {
                        "type": "string",
                        "description": "The directory path to create",
                    }
                },
                "required": ["directory_path"],
            },
        },
    ],
    "config_list": config_list,
    "seed": 45,
    "request_timeout": 120
}

user_proxy = UserProxyAgent(
    name="user_proxy",
    system_message="A human that will provide the necessary information to the group chat manager. Execute suggested function calls.",
    function_map={
        "write_to_file": write_to_file,
        "read_from_file": read_from_file,
        "read_pdf": read_pdf,
        "create_directory": create_directory,
    },
    human_input_mode="ALWAYS",
    code_execution_config={"work_dir": "fileread"})

assistant = AssistantAgent(
    name="assistant",
    system_message="""You are an assistant, you must blah blah""",
    llm_config=llm_config
)
architect = AssistantAgent(
    name="azure_architect",
    system_message="""You are a blah blah""",
    llm_config=llm_config,
)

groupchat = ExecutorGroupchat(
    agents=[user_proxy, assistant, architect], messages=[],max_round=20)

manager = GroupChatManager(groupchat=groupchat, llm_config=llm_config)

user_proxy.initiate_chat(
    manager,
    message="""do the thing."""
    )

If you have any suggestions on what I'm missing, please let me know, this is all relatively new for me.

[LittleLittleCloud]

Try passing the dedicated_executor to groupchat (which is user_proxy in this case)

from

groupchat = ExecutorGroupchat( agents=[user_proxy, assistant, architect], messages=[],max_round=20)

to

groupchat = ExecutorGroupchat( agents=[user_proxy, assistant, architect], messages=[],max_round=20, dedicated_executor = user_proxy)

[nubgamerz]

YES! This is it.

Thank you so much. I hope this gets pulled into autogen.

[LittleLittleCloud]

Cool, glad to see you are unblocked!

[nubgamerz]

quick question, with this dedicated executor, will this mess up the chat manager to hand off tasks to other agents?

In my tests, it ends after the first agent completes it's job, but it should be going back to the chat manager, so the next task is completed by the second agent.

[LittleLittleCloud]

If you have chat log to share that would be helpful. Otherwise I would need to run your code tmr to take a closer look

[nubgamerz]

You know what, not to worry, this was me being hasty without fully testing.
It was a prompting issue. Once I clarified the prompts, it completed as expected.

This is truly next gen, using this function call + generation method, I can add functionality to our GPT4 knowledge base chat bot that's already live.

Really appreciate the help from all.

If we can get that executor class into the main autogen package, I think this can really help simplify certain scenarios.

If I can follow up with a small, slightly off topic question... how can we record the logs to JSON without redirecting stdout?

I saw the documentation mentioned, but it's not super clear. Could you provide an example of where I can record the agent conversion logs to an output JSON file?

[LittleLittleCloud]

I actually not quite familiar on how logging works in autogen as well. Maybe @sonichi knows more. If you just want to save the chat message from a group chat,,, here's how I do it.

with open(output_txt, "w", encoding='utf-8') as f:
            for message in groupchat.messages:
                # write a seperator
                f.write("-" * 20 + "\n")
                f.write(f'''###
{message["name"]}
###''' + "\n")
                f.write(message["content"] + "\n")
                f.write("-" * 20 + "\n")
    except Exception as e:
        raise e

[sonichi]

@nubgamerz do you want to save the log after the chat finishes or on the fly? After finishing, you can use @LittleLittleCloud 's solution to dump the messages groupchat.messages into a json file using json.dump.

[xianzhisheng]

Hope this issue will be fixed soon and updated in the next release

[sonichi]

@LittleLittleCloud @nubgamerz do you plan to make a PR to add the solution to the library? The new agents can be added to contrib/

[LittleLittleCloud]

The ExecutorGroupchat can't process two function_call with the same name but a different implementation. It's a good work-around but I'll be hesitant to bring it to the library. A more elaborate design needs to be considered here

[sonichi]

That's why I suggest putting it under contrib/. Also we should document that limitation clearly.
If you have a better design, please feel free to suggest that and an ETA.
Who's the original author of ExecutorGroupchat ?

[LittleLittleCloud]

Should be @ilaffey2?

[sonichi]

@ilaffey2 are you interested in making a PR?
@LittleLittleCloud do you have objection against it?

[nubgamerz]

@LittleLittleCloud @nubgamerz do you plan to make a PR to add the solution to the library? The new agents can be added to contrib/

This is not my work, so it would be unproper for me to create a PR for this.

[cccc11231]

@nubgamerz a quick question. Are you using local LLM? If yes, can you suggest which model is compatible with Autogen and predefined function feature? I am using CodeLlama, but it cannot recognize predefined functions.