Added basic integration with LangGraph

[SoumilRathi]

🔍 Review Summary

Purpose:
Introduce a basic integration with Langgraph to enhance system capabilities for handling complex agent interactions and improve debugging and monitoring processes.

Key Changes:

• New Feature: Introduced LanggraphCallbackHandler and AsyncLanggraphCallbackHandler classes to manage events for Langgraph agents. Includes methods for handling LLM, chain, tool, retriever, and agent actions, as well as error handling and logging.
• Test: Added test cases for LanggraphCallbackHandler using a simulated agent workflow. Includes tools for Wikipedia search and datetime retrieval, and a supervisor agent to manage task execution.

Impact:
Enhances the system's ability to handle complex agent interactions and improves debugging and monitoring processes.

Original Description

No existing description found

[areibman]

Super cool, thank for raising @SoumilRathi ! Will take a look

[the-praxs]

@SoumilRathi thanks for the PR! I see that its working but I noticed 2 LLMEvents for the same conversation as denoted by the ChatViewer -

• First LLMEvent -
  This one is showing the model as ChatOpenAI and the human in the User role. It's better to have this an ActionEvent.

Second LLMEvent -
This one's the accurate one and looks good.

This is the code I used to test (adapted from Langfuse):

from typing import Annotated
import agentops
from dotenv import load_dotenv
import os

from langchain_openai import ChatOpenAI
from langchain_core.messages import HumanMessage
from typing_extensions import TypedDict
 
from langgraph.graph import StateGraph
from langgraph.graph.message import add_messages

from agentops.partners.langgraph_callback_handler import LanggraphCallbackHandler
 
load_dotenv()
agentops.init(api_key=os.getenv("AGENTOPS_API_KEY"), default_tags=["langgraph-test-v1"])
 
class State(TypedDict):
    # Messages have the type "list". The `add_messages` function in the annotation defines how this state key should be updated
    # (in this case, it appends messages to the list, rather than overwriting them)
    messages: Annotated[list, add_messages]
 
graph_builder = StateGraph(State)
 
llm = ChatOpenAI(model = "gpt-4o", temperature = 0.2)
 
# The chatbot node function takes the current State as input and returns an updated messages list. This is the basic pattern for all LangGraph node functions.
def chatbot(state: State):
    return {"messages": [llm.invoke(state["messages"])]}
 
# Add a "chatbot" node. Nodes represent units of work. They are typically regular python functions.
graph_builder.add_node("chatbot", chatbot)
 
# Add an entry point. This tells our graph where to start its work each time we run it.
graph_builder.set_entry_point("chatbot")
 
# Set a finish point. This instructs the graph "any time this node is run, you can exit."
graph_builder.set_finish_point("chatbot")
 
# To be able to run our graph, call "compile()" on the graph builder. This creates a "CompiledGraph" we can use invoke on our state.
graph = graph_builder.compile()
 
# Initialize AgentOps CallbackHandler for Langchain (tracing)
handler = LanggraphCallbackHandler()
 
for s in graph.stream({"messages": [HumanMessage(content = "What is Langfuse?")]},
                      config={"callbacks": [handler]}):
    print(s)

I would request you to resolve this issue and also create an example notebook having different examples (say the code like above and a more complex agent architecture) to check if everything's working.

[the-praxs]

This works but its better to make these changes to ensure consistency.

[the-praxs]

Add default_tags here.

[the-praxs]

Add default_tags here.

[the-praxs]

These should have default values for safety if the attributes are absent.

[the-praxs]

logs or details not params.

[the-praxs]

logs or details not params.

[the-praxs]

logs or details not params.

[the-praxs]

logs or details not params.

[the-praxs]

This function should be defined outside the class to work with both the LanggraphCallbackHandler and AsyncLanggraphCallbackHandler.

[the-praxs]

This function should be defined outside the class to work with both the LanggraphCallbackHandler and AsyncLanggraphCallbackHandler.

[the-praxs]

These should have default values for safety if the attribute is absent.

[the-praxs]

@SoumilRathi I have tested the _test_langgraph_handler.py and the same issues with LLMEvent are present here -

• First event -

• Second event -

Please resolve the inconsistencies in the ChatML format for the prompt and completions to ensure everything is looking correct -