Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: fix bug triggered by using ada embeddings #1915

Merged
merged 4 commits into from
Oct 22, 2024
Merged

Conversation

cpacker
Copy link
Collaborator

@cpacker cpacker commented Oct 22, 2024

To reproduce bug:

  1. Open ADE, create agent w/ gpt-4o-mini backend + text-embeddings-ada-002 embeddings backend
  2. Ask agent to add "banana" to your archival memory
  3. Error should trigger server side (on the client, the message will just stall after the inner thoughts)
image
Task exception was never retrieved
future: <Task finished name='Task-25' coro=<to_thread() done, defined at /Users/loaner/miniconda3/lib/python3.10/asyncio/threads.py:12> exception=1 validation error for FunctionCallDelta
function_call_id
  Field required [type=missing, input_value={'name': 'archival_memory...ert', 'arguments': None}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.9/v/missing>
Traceback (most recent call last):
  File "/Users/loaner/miniconda3/lib/python3.10/asyncio/threads.py", line 25, in to_thread
    return await loop.run_in_executor(None, func_call)
  File "/Users/loaner/miniconda3/lib/python3.10/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/Users/loaner/dev/MemGPT-fresh/letta/server/server.py", line 737, in send_messages
    return self._step(user_id=user_id, agent_id=agent_id, input_messages=message_objects)
  File "/Users/loaner/dev/MemGPT-fresh/letta/server/server.py", line 430, in _step
    usage_stats = letta_agent.step(
  File "/Users/loaner/dev/MemGPT-fresh/letta/agent.py", line 749, in step
    step_response = self.inner_step(
  File "/Users/loaner/dev/MemGPT-fresh/letta/agent.py", line 958, in inner_step
    raise e
  File "/Users/loaner/dev/MemGPT-fresh/letta/agent.py", line 874, in inner_step
    response = self._get_ai_reply(
  File "/Users/loaner/dev/MemGPT-fresh/letta/agent.py", line 501, in _get_ai_reply
    raise e
  File "/Users/loaner/dev/MemGPT-fresh/letta/agent.py", line 472, in _get_ai_reply
    response = create(
  File "/Users/loaner/dev/MemGPT-fresh/letta/llm_api/llm_api_tools.py", line 97, in wrapper
    raise e
  File "/Users/loaner/dev/MemGPT-fresh/letta/llm_api/llm_api_tools.py", line 66, in wrapper
    return func(*args, **kwargs)
  File "/Users/loaner/dev/MemGPT-fresh/letta/llm_api/llm_api_tools.py", line 149, in create
    response = openai_chat_completions_process_stream(
  File "/Users/loaner/dev/MemGPT-fresh/letta/llm_api/openai.py", line 348, in openai_chat_completions_process_stream
    raise e
  File "/Users/loaner/dev/MemGPT-fresh/letta/llm_api/openai.py", line 262, in openai_chat_completions_process_stream
    stream_interface.process_chunk(
  File "/Users/loaner/dev/MemGPT-fresh/letta/server/rest_api/interface.py", line 773, in process_chunk
    processed_chunk = self._process_chunk_to_letta_style(chunk=chunk, message_id=message_id, message_date=message_date)
  File "/Users/loaner/dev/MemGPT-fresh/letta/server/rest_api/interface.py", line 529, in _process_chunk_to_letta_style
    function_call=FunctionCallDelta(name=self.function_name_buffer, arguments=None),
  File "/Users/loaner/Library/Caches/pypoetry/virtualenvs/letta-JSsUGnlY-py3.10/lib/python3.10/site-packages/pydantic/main.py", line 212, in __init__
    validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
pydantic_core._pydantic_core.ValidationError: 1 validation error for FunctionCallDelta
function_call_id
  Field required [type=missing, input_value={'name': 'archival_memory...ert', 'arguments': None}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.9/v/missing

@cpacker
Copy link
Collaborator Author

cpacker commented Oct 22, 2024

New error:

image
INFO:     ::1:53229 - "POST /v1/agents/agent-6fa72914-3c0a-48b0-9139-98870173ed5c/messages HTTP/1.1" 200 OK
/Users/loaner/dev/MemGPT-fresh/letta/server/rest_api/interface.py:705: UserWarning: Couldn't find delta in chunk: id='chatcmpl-AKwyaJFyixv7hgt9VmgHLESPQiOB6' choices=[ChunkChoice(finish_reason=None, index=0, delta=MessageDelta(content=None, tool_calls=None, function_call=None), logprobs=None)] created=datetime.datetime(2024, 10, 22, 0, 23, 48, tzinfo=TzInfo(UTC)) model='gpt-4o-mini-2024-07-18' system_fingerprint='fp_482c22a7bc' object='chat.completion.chunk'
  warnings.warn(f"Couldn't find delta in chunk: {chunk}")
Parsing ChatCompletion stream failed with error:
list index out of range
Letta.letta.server.server - ERROR - Error in server._step: list index out of range
Traceback (most recent call last):
  File "/Users/loaner/dev/MemGPT-fresh/letta/server/server.py", line 430, in _step
    usage_stats = letta_agent.step(
  File "/Users/loaner/dev/MemGPT-fresh/letta/agent.py", line 749, in step
    step_response = self.inner_step(
  File "/Users/loaner/dev/MemGPT-fresh/letta/agent.py", line 958, in inner_step
    raise e
  File "/Users/loaner/dev/MemGPT-fresh/letta/agent.py", line 874, in inner_step
    response = self._get_ai_reply(
  File "/Users/loaner/dev/MemGPT-fresh/letta/agent.py", line 501, in _get_ai_reply
    raise e
  File "/Users/loaner/dev/MemGPT-fresh/letta/agent.py", line 472, in _get_ai_reply
    response = create(
  File "/Users/loaner/dev/MemGPT-fresh/letta/llm_api/llm_api_tools.py", line 97, in wrapper
    raise e
  File "/Users/loaner/dev/MemGPT-fresh/letta/llm_api/llm_api_tools.py", line 66, in wrapper
    return func(*args, **kwargs)
  File "/Users/loaner/dev/MemGPT-fresh/letta/llm_api/llm_api_tools.py", line 149, in create
    response = openai_chat_completions_process_stream(
  File "/Users/loaner/dev/MemGPT-fresh/letta/llm_api/openai.py", line 348, in openai_chat_completions_process_stream
    raise e
  File "/Users/loaner/dev/MemGPT-fresh/letta/llm_api/openai.py", line 321, in openai_chat_completions_process_stream
    accum_message.tool_calls[tool_call_delta.index].id = tool_call_delta.id
IndexError: list index out of range
None
Task exception was never retrieved
future: <Task finished name='Task-73' coro=<to_thread() done, defined at /Users/loaner/miniconda3/lib/python3.10/asyncio/threads.py:12> exception=IndexError('list index out of range')>
Traceback (most recent call last):
  File "/Users/loaner/miniconda3/lib/python3.10/asyncio/threads.py", line 25, in to_thread
    return await loop.run_in_executor(None, func_call)
  File "/Users/loaner/miniconda3/lib/python3.10/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/Users/loaner/dev/MemGPT-fresh/letta/server/server.py", line 737, in send_messages
    return self._step(user_id=user_id, agent_id=agent_id, input_messages=message_objects)
  File "/Users/loaner/dev/MemGPT-fresh/letta/server/server.py", line 430, in _step
    usage_stats = letta_agent.step(
  File "/Users/loaner/dev/MemGPT-fresh/letta/agent.py", line 749, in step
    step_response = self.inner_step(
  File "/Users/loaner/dev/MemGPT-fresh/letta/agent.py", line 958, in inner_step
    raise e
  File "/Users/loaner/dev/MemGPT-fresh/letta/agent.py", line 874, in inner_step
    response = self._get_ai_reply(
  File "/Users/loaner/dev/MemGPT-fresh/letta/agent.py", line 501, in _get_ai_reply
    raise e
  File "/Users/loaner/dev/MemGPT-fresh/letta/agent.py", line 472, in _get_ai_reply
    response = create(
  File "/Users/loaner/dev/MemGPT-fresh/letta/llm_api/llm_api_tools.py", line 97, in wrapper
    raise e
  File "/Users/loaner/dev/MemGPT-fresh/letta/llm_api/llm_api_tools.py", line 66, in wrapper
    return func(*args, **kwargs)
  File "/Users/loaner/dev/MemGPT-fresh/letta/llm_api/llm_api_tools.py", line 149, in create
    response = openai_chat_completions_process_stream(
  File "/Users/loaner/dev/MemGPT-fresh/letta/llm_api/openai.py", line 348, in openai_chat_completions_process_stream
    raise e
  File "/Users/loaner/dev/MemGPT-fresh/letta/llm_api/openai.py", line 321, in openai_chat_completions_process_stream
    accum_message.tool_calls[tool_call_delta.index].id = tool_call_delta.id
IndexError: list index out of range

@cpacker
Copy link
Collaborator Author

cpacker commented Oct 22, 2024

Outstanding bug where trailing " is dropped:
image

@cpacker cpacker merged commit 588261e into main Oct 22, 2024
15 checks passed
@cpacker cpacker deleted the fix-embeddings-bug branch October 22, 2024 23:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant