[ENH] accessing the run steps with GPTAssistantAgent

[YannDubs]

The current GPTAssistantAgent only returns the output from the last step when you call _invoke_assistant. But in many cases I would like to be able to see what were the intermediate calls that were made. You can get that information using self.openai_client.beta.beta.threads.runs.steps.list(thread_id=..., run_id=..., order="asc") as shown here. But I don't think you can currently get the run_id of the last call to get those steps. For more interpretability and when developing, it would be nice to either:

1. save the latest run_id to be able to get the steps
   and/or
2. have an option in _invoke_assistant to return all the steps

E.g. this i useful when using web search or code_interpreter to know what were the arguments and how many calls were there.

Ps: thanks for the great library 💯

[gagb]

@YannDubs checkout this PR #665 Feel free to review it and suggest improvements. Would like to get this merged before thanksgiving

[YannDubs]

Thanks @gagb, But I don't quite see how the PR addresses the issue. You still can't access the run.id after invoking the assistant, right?

[gagb]

It doesn't yet, but perhaps _get_run_response and _process_messages can be modified to log steps (or add them to the message itself). Or did I misunderstand your question?

autogen/autogen/agentchat/contrib/gpt_assistant_agent.py

Line 151 in 7ae462d

response = self._get_run_response(assistant_thread, run)

autogen/autogen/agentchat/contrib/gpt_assistant_agent.py

Line 158 in 7ae462d

def _process_messages(self, assistant_thread, run):

[sonichi]

@IANTHEREAL FYI

[IANTHEREAL]

@YannDubs We have implemented a feature that logs more (the parameters and results) on function calls, which should fulfill the requirements for tracing function calls. I am also considering to extend this tracing capability to other tools, such as the code interpreter, in a subsequent phase.

[YannDubs]

Thanks, @IANTHEREAL #717 seems useful but still only concerns the function execution, right? It won't show all the steps in the assistant if you use the code interpreter, right?

I might be wrong, but I was under the impression that OpenAI controls those substeps and you can only access them using client.beta.threads.runs.steps

[IANTHEREAL]

@YannDubs Yes. Currently, I have resolved the issue with function call tracing, which is controlled by the external program itself (not openai api). To tracing openai built-in tools like code interpreter, it is necessary to utilize the client.beta.threads.runs.steps api, which I will work on it in my free time

[thinkall]

Closing this issue due to inactivity. If you have further questions, please open a new issue or join the discussion in AutoGen Discord server: https://discord.com/invite/Yb5gwGVkE5