Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

langgraph[patch]: remove stringify of tool msg contnt #1810

Merged
merged 6 commits into from
Sep 23, 2024
Merged
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 25 additions & 0 deletions libs/langgraph/langgraph/prebuilt/tool_node.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
from __future__ import annotations

Check notice on line 1 in libs/langgraph/langgraph/prebuilt/tool_node.py

View workflow job for this annotation

GitHub Actions / benchmark

Benchmark results

......................................... fanout_to_subgraph_10x: Mean +- std dev: 59.2 ms +- 1.2 ms ......................................... WARNING: the benchmark result may be unstable * the standard deviation (6.90 ms) is 12% of the mean (56.7 ms) Try to rerun the benchmark with more runs, values and/or loops. Run 'python -m pyperf system tune' command to reduce the system jitter. Use pyperf stats, pyperf dump and pyperf hist to analyze results. Use --quiet option to hide these warnings. fanout_to_subgraph_10x_sync: Mean +- std dev: 56.7 ms +- 6.9 ms ......................................... fanout_to_subgraph_10x_checkpoint: Mean +- std dev: 76.7 ms +- 1.3 ms ......................................... fanout_to_subgraph_10x_checkpoint_sync: Mean +- std dev: 81.8 ms +- 0.9 ms ......................................... fanout_to_subgraph_100x: Mean +- std dev: 554 ms +- 9 ms ......................................... fanout_to_subgraph_100x_sync: Mean +- std dev: 499 ms +- 5 ms ......................................... fanout_to_subgraph_100x_checkpoint: Mean +- std dev: 771 ms +- 24 ms ......................................... fanout_to_subgraph_100x_checkpoint_sync: Mean +- std dev: 789 ms +- 6 ms ......................................... react_agent_10x: Mean +- std dev: 41.4 ms +- 3.4 ms ......................................... react_agent_10x_sync: Mean +- std dev: 29.6 ms +- 0.2 ms ......................................... react_agent_10x_checkpoint: Mean +- std dev: 52.7 ms +- 1.3 ms ......................................... react_agent_10x_checkpoint_sync: Mean +- std dev: 42.9 ms +- 3.5 ms ......................................... react_agent_100x: Mean +- std dev: 411 ms +- 7 ms ......................................... react_agent_100x_sync: Mean +- std dev: 330 ms +- 3 ms ......................................... react_agent_100x_checkpoint: Mean +- std dev: 926 ms +- 13 ms ......................................... react_agent_100x_checkpoint_sync: Mean +- std dev: 824 ms +- 11 ms ......................................... wide_state_25x300: Mean +- std dev: 20.3 ms +- 0.2 ms ......................................... wide_state_25x300_sync: Mean +- std dev: 12.7 ms +- 0.1 ms ......................................... wide_state_25x300_checkpoint: Mean +- std dev: 238 ms +- 7 ms ......................................... wide_state_25x300_checkpoint_sync: Mean +- std dev: 237 ms +- 14 ms ......................................... wide_state_15x600: Mean +- std dev: 23.4 ms +- 0.3 ms ......................................... wide_state_15x600_sync: Mean +- std dev: 14.7 ms +- 0.1 ms ......................................... wide_state_15x600_checkpoint: Mean +- std dev: 416 ms +- 14 ms ......................................... wide_state_15x600_checkpoint_sync: Mean +- std dev: 417 ms +- 17 ms ......................................... wide_state_9x1200: Mean +- std dev: 23.5 ms +- 0.3 ms ......................................... wide_state_9x1200_sync: Mean +- std dev: 14.7 ms +- 0.1 ms ......................................... wide_state_9x1200_checkpoint: Mean +- std dev: 268 ms +- 8 ms ......................................... wide_state_9x1200_checkpoint_sync: Mean +- std dev: 269 ms +- 15 ms

Check notice on line 1 in libs/langgraph/langgraph/prebuilt/tool_node.py

View workflow job for this annotation

GitHub Actions / benchmark

Comparison against main

+-----------------------------------------+---------+-----------------------+ | Benchmark | main | changes | +=========================================+=========+=======================+ | fanout_to_subgraph_100x_checkpoint | 795 ms | 771 ms: 1.03x faster | +-----------------------------------------+---------+-----------------------+ | fanout_to_subgraph_100x | 568 ms | 554 ms: 1.02x faster | +-----------------------------------------+---------+-----------------------+ | react_agent_100x_checkpoint | 941 ms | 926 ms: 1.02x faster | +-----------------------------------------+---------+-----------------------+ | react_agent_100x_checkpoint_sync | 835 ms | 824 ms: 1.01x faster | +-----------------------------------------+---------+-----------------------+ | wide_state_15x600 | 23.7 ms | 23.4 ms: 1.01x faster | +-----------------------------------------+---------+-----------------------+ | fanout_to_subgraph_10x_checkpoint | 77.6 ms | 76.7 ms: 1.01x faster | +-----------------------------------------+---------+-----------------------+ | wide_state_9x1200 | 23.7 ms | 23.5 ms: 1.01x faster | +-----------------------------------------+---------+-----------------------+ | react_agent_10x_sync | 30.0 ms | 29.6 ms: 1.01x faster | +-----------------------------------------+---------+-----------------------+ | wide_state_15x600_sync | 14.9 ms | 14.7 ms: 1.01x faster | +-----------------------------------------+---------+-----------------------+ | fanout_to_subgraph_100x_sync | 504 ms | 499 ms: 1.01x faster | +-----------------------------------------+---------+-----------------------+ | react_agent_10x_checkpoint | 53.3 ms | 52.7 ms: 1.01x faster | +-----------------------------------------+---------+-----------------------+ | react_agent_100x | 415 ms | 411 ms: 1.01x faster | +-----------------------------------------+---------+-----------------------+ | fanout_to_subgraph_10x | 59.7 ms | 59.2 ms: 1.01x faster | +-----------------------------------------+---------+-----------------------+ | react_agent_100x_sync | 333 ms | 330 ms: 1.01x faster | +-----------------------------------------+---------+-----------------------+ | wide_state_9x1200_sync | 14.8 ms | 14.7 ms: 1.01x faster | +-----------------------------------------+---------+-----------------------+ | fanout_to_subgraph_100x_checkpoint_sync | 795 ms | 789 ms: 1.01x faster | +-----------------------------------------+---------+-----------------------+ | wide_state_25x300_sync | 12.8 ms | 12.7 ms: 1.00x faster | +-----------------------------------------+---------+-----------------------+ | wide_state_25x300 | 20.4 ms | 20.3 ms: 1.00x faster | +-----------------------------------------+---------+-----------------------+ | fanout_to_subgraph_10x_checkpoint_sync | 82.0 ms | 81.8 ms: 1.00x faster | +-----------------------------------------+---------+-----------------------+ | Geometric mean | (ref) | 1.01x faster | +-----------------------------------------+---------+-----------------------+ Benchmark hidden because not significant (9): fanout_to_subgraph_10x_sync, react_agent_10x_checkpoint_sync, react_agent_10x, wide_state_15x600_checkpoint_sync, wide_state_25x300_checkpoint, wide_state_25x300_checkpoint_sync, wide_state_9x1200_checkpoint, wide_state_9x1200_checkpoint_sync, wide_state_15x600_checkpoint

import asyncio
import json
Expand Down Expand Up @@ -43,6 +43,27 @@
TOOL_CALL_ERROR_TEMPLATE = "Error: {error}\n Please fix your mistakes."


def msg_content_output(output: Any) -> str | List[dict]:
recognized_content_block_types = ("image", "image_url", "text", "json")
if isinstance(output, str):
return output
elif isinstance(output, list) and all(
[
baskaryan marked this conversation as resolved.
Show resolved Hide resolved
isinstance(x, dict) and x.get("type") in recognized_content_block_types
for x in output
]
):
return output
# Technically a list of strings is also valid message content but it's not currently
# well tested that all chat models support this. And for backwards compatibility
# we want to make sure we don't break any existing ToolNode usage.
else:
try:
return json.dumps(output, ensure_ascii=False)
except Exception:
return str(output)


class ToolNode(RunnableCallable):
"""A node that runs the tools called in the last AIMessage.

Expand Down Expand Up @@ -128,6 +149,8 @@
tool_message: ToolMessage = self.tools_by_name[call["name"]].invoke(
input, config
)
# TODO: handle this properly in core
tool_message.content = msg_content_output(tool_message.content)
return tool_message
except Exception as e:
if not self.handle_tool_errors:
Expand All @@ -143,6 +166,8 @@
tool_message: ToolMessage = await self.tools_by_name[call["name"]].ainvoke(
input, config
)
# TODO: handle this properly in core
tool_message.content = msg_content_output(tool_message.content)
return tool_message
except Exception as e:
if not self.handle_tool_errors:
Expand Down
Loading