Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add one more unit test for exception propagation #1757

Merged
merged 1 commit into from
Sep 18, 2024
Merged

Add one more unit test for exception propagation #1757

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions libs/langgraph/tests/test_pregel_async.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import asyncio

Check notice on line 1 in libs/langgraph/tests/test_pregel_async.py

View workflow job for this annotation

GitHub Actions / benchmark

Benchmark results 

.........................................
WARNING: the benchmark result may be unstable
* the standard deviation (7.61 ms) is 12% of the mean (62.2 ms)

Try to rerun the benchmark with more runs, values and/or loops.
Run 'python -m pyperf system tune' command to reduce the system jitter.
Use pyperf stats, pyperf dump and pyperf hist to analyze results.
Use --quiet option to hide these warnings.

fanout_to_subgraph_10x: Mean +- std dev: 62.2 ms +- 7.6 ms
.........................................
WARNING: the benchmark result may be unstable
* the standard deviation (7.65 ms) is 14% of the mean (55.4 ms)

Try to rerun the benchmark with more runs, values and/or loops.
Run 'python -m pyperf system tune' command to reduce the system jitter.
Use pyperf stats, pyperf dump and pyperf hist to analyze results.
Use --quiet option to hide these warnings.

fanout_to_subgraph_10x_sync: Mean +- std dev: 55.4 ms +- 7.6 ms
.........................................
fanout_to_subgraph_10x_checkpoint: Mean +- std dev: 76.0 ms +- 1.3 ms
.........................................
fanout_to_subgraph_10x_checkpoint_sync: Mean +- std dev: 80.6 ms +- 1.0 ms
.........................................
fanout_to_subgraph_100x: Mean +- std dev: 580 ms +- 15 ms
.........................................
fanout_to_subgraph_100x_sync: Mean +- std dev: 494 ms +- 6 ms
.........................................
fanout_to_subgraph_100x_checkpoint: Mean +- std dev: 826 ms +- 32 ms
.........................................
fanout_to_subgraph_100x_checkpoint_sync: Mean +- std dev: 771 ms +- 10 ms
.........................................
react_agent_10x: Mean +- std dev: 38.5 ms +- 0.7 ms
.........................................
react_agent_10x_sync: Mean +- std dev: 30.0 ms +- 0.4 ms
.........................................
react_agent_10x_checkpoint: Mean +- std dev: 52.1 ms +- 1.3 ms
.........................................
react_agent_10x_checkpoint_sync: Mean +- std dev: 39.2 ms +- 0.7 ms
.........................................
react_agent_100x: Mean +- std dev: 415 ms +- 16 ms
.........................................
react_agent_100x_sync: Mean +- std dev: 341 ms +- 14 ms
.........................................
react_agent_100x_checkpoint: Mean +- std dev: 989 ms +- 21 ms
.........................................
react_agent_100x_checkpoint_sync: Mean +- std dev: 923 ms +- 30 ms
.........................................
wide_state_25x300: Mean +- std dev: 20.2 ms +- 0.2 ms
.........................................
wide_state_25x300_sync: Mean +- std dev: 12.5 ms +- 0.1 ms
.........................................
wide_state_25x300_checkpoint: Mean +- std dev: 245 ms +- 7 ms
.........................................
wide_state_25x300_checkpoint_sync: Mean +- std dev: 236 ms +- 9 ms
.........................................
wide_state_15x600: Mean +- std dev: 23.1 ms +- 0.3 ms
.........................................
wide_state_15x600_sync: Mean +- std dev: 14.3 ms +- 0.2 ms
.........................................
wide_state_15x600_checkpoint: Mean +- std dev: 418 ms +- 12 ms
.........................................
wide_state_15x600_checkpoint_sync: Mean +- std dev: 415 ms +- 12 ms
.........................................
wide_state_9x1200: Mean +- std dev: 23.4 ms +- 0.4 ms
.........................................
wide_state_9x1200_sync: Mean +- std dev: 14.5 ms +- 0.3 ms
.........................................
wide_state_9x1200_checkpoint: Mean +- std dev: 274 ms +- 8 ms
.........................................
wide_state_9x1200_checkpoint_sync: Mean +- std dev: 265 ms +- 9 ms

Check notice on line 1 in libs/langgraph/tests/test_pregel_async.py

View workflow job for this annotation

GitHub Actions / benchmark

Comparison against main 

+-----------------------------------------+---------+-----------------------+
| Benchmark                               | main    | changes               |
+=========================================+=========+=======================+
| react_agent_100x_checkpoint             | 981 ms  | 989 ms: 1.01x slower  |
+-----------------------------------------+---------+-----------------------+
| wide_state_9x1200_checkpoint_sync       | 262 ms  | 265 ms: 1.01x slower  |
+-----------------------------------------+---------+-----------------------+
| wide_state_25x300_checkpoint_sync       | 233 ms  | 236 ms: 1.01x slower  |
+-----------------------------------------+---------+-----------------------+
| react_agent_100x                        | 411 ms  | 415 ms: 1.01x slower  |
+-----------------------------------------+---------+-----------------------+
| wide_state_25x300_checkpoint            | 242 ms  | 245 ms: 1.01x slower  |
+-----------------------------------------+---------+-----------------------+
| wide_state_15x600                       | 22.8 ms | 23.1 ms: 1.01x slower |
+-----------------------------------------+---------+-----------------------+
| react_agent_10x_checkpoint_sync         | 38.6 ms | 39.2 ms: 1.02x slower |
+-----------------------------------------+---------+-----------------------+
| wide_state_9x1200_checkpoint            | 270 ms  | 274 ms: 1.02x slower  |
+-----------------------------------------+---------+-----------------------+
| react_agent_10x                         | 37.9 ms | 38.5 ms: 1.02x slower |
+-----------------------------------------+---------+-----------------------+
| react_agent_100x_sync                   | 335 ms  | 341 ms: 1.02x slower  |
+-----------------------------------------+---------+-----------------------+
| wide_state_15x600_checkpoint_sync       | 408 ms  | 415 ms: 1.02x slower  |
+-----------------------------------------+---------+-----------------------+
| react_agent_10x_sync                    | 29.4 ms | 30.0 ms: 1.02x slower |
+-----------------------------------------+---------+-----------------------+
| fanout_to_subgraph_10x_checkpoint       | 74.2 ms | 76.0 ms: 1.02x slower |
+-----------------------------------------+---------+-----------------------+
| wide_state_9x1200                       | 22.8 ms | 23.4 ms: 1.03x slower |
+-----------------------------------------+---------+-----------------------+
| wide_state_25x300                       | 19.6 ms | 20.2 ms: 1.03x slower |
+-----------------------------------------+---------+-----------------------+
| react_agent_10x_checkpoint              | 50.7 ms | 52.1 ms: 1.03x slower |
+-----------------------------------------+---------+-----------------------+
| wide_state_15x600_sync                  | 13.7 ms | 14.3 ms: 1.04x slower |
+-----------------------------------------+---------+-----------------------+
| fanout_to_subgraph_10x_checkpoint_sync  | 77.2 ms | 80.6 ms: 1.04x slower |
+-----------------------------------------+---------+-----------------------+
| fanout_to_subgraph_100x_sync            | 473 ms  | 494 ms: 1.04x slower  |
+-----------------------------------------+---------+-----------------------+
| fanout_to_subgraph_100x_checkpoint_sync | 738 ms  | 771 ms: 1.04x slower  |
+-----------------------------------------+---------+-----------------------+
| react_agent_100x_checkpoint_sync        | 883 ms  | 923 ms: 1.04x slower  |
+-----------------------------------------+---------+-----------------------+
| wide_state_9x1200_sync                  | 13.7 ms | 14.5 ms: 1.05x slower |
+-----------------------------------------+---------+-----------------------+
| wide_state_25x300_sync                  | 11.9 ms | 12.5 ms: 1.05x slower |
+-----------------------------------------+---------+-----------------------+
| fanout_to_subgraph_100x                 | 543 ms  | 580 ms: 1.07x slower  |
+-----------------------------------------+---------+-----------------------+
| fanout_to_subgraph_10x_sync             | 50.7 ms | 55.4 ms: 1.09x slower |
+---------------------------------------
import operator
import re
import sys
Expand Down Expand Up @@ -223,6 +223,25 @@
assert inner_task_cancelled


async def test_node_cancellation_on_other_node_exception_two() -> None:
async def awhile(input: Any) -> None:
await asyncio.sleep(1)

async def iambad(input: Any) -> None:
raise ValueError("I am bad")

builder = Graph()
builder.add_node("agent", awhile)
builder.add_node("bad", iambad)
builder.set_conditional_entry_point(lambda _: ["agent", "bad"], then=END)

graph = builder.compile()

with pytest.raises(ValueError, match="I am bad"):
# This will raise ValueError, not CancelledError
await graph.ainvoke(1)


@pytest.mark.parametrize("checkpointer_name", ALL_CHECKPOINTERS_ASYNC)
async def test_dynamic_interrupt(checkpointer_name: str) -> None:
class State(TypedDict):
Expand Down
Loading