Improvements to JSON handling for local LLMs

[cpacker]

"Closes" the double-JSON error mentioned in #177

• Add a double-json string parse fallback for common JSON failure mode (double object)
• Make "no wrapper specified" only pop up once
• Allow retries on failed JSON decoding errors

Move to separate PR:

- [ ] Add/remove heartbeats manually
- [ ] Move web UI to openai extension setup? See @TheOnlyWiseJEDI's comments on discord
- [ ] Print the model that's running in the backend (with GET for lm studio)

---

Example fixing the double-JSON problem

Example bad LLM output (from #177):

{
  "function": "core_memory_append",
  "params": {
    "name": "human",
    "content": "Horst, 42 years old, from Germany."
  }
}
{
  "function": "send_message",
  "params": {
    "message": "Got it! Your age and nationality are now saved in my memory."
  }
}

MemGPT output (without patch):

memgpt.errors.LocalLLMError: Failed to parse JSON from local LLM response - error: Failed to decode JSON from LLM output:
{
      "function": "core_memory_append",
      "params": {
        "name": "human",
        "content": "Horst, 42 years old, from Germany."
      }
    }
    {
      "function": "send_message",
      "params": {
        "message": "Got it! Your age and nationality are now saved in my memory."
      }
    }

MemGPT output (with patch):

💭 Bootup sequence complete. Persona activated. Testing messaging functionality.
🧑 {'type': 'login', 'last_login': 'Never (first login)', 'time': '2023-11-02 11:59:28 PM PDT-0700'}
Hit enter to begin (will request first MemGPT message)

Warning: no wrapper specified for local LLM, using the default wrapper (you can remove this warning by specifying the wrapper with --model)
💭 None
⚡🧠 [function] updating memory with core_memory_append


`/dumpraw`

{'role': 'function', 'name': 'send_message', 'content': '{"status": "OK", "message": null, "time": "2023-11-02 11:59:28 PM PDT-0700"}'}
{'role': 'user', 'content': '{"type": "login", "last_login": "Never (first login)", "time": "2023-11-02 11:59:28 PM PDT-0700"}'}
{'role': 'assistant', 'content': None, 'function_call': {'name': 'core_memory_append', 'arguments': '{"name": "human", "content": "Horst, 42 years old, from Germany."}'}}
{'role': 'function', 'name': 'core_memory_append', 'content': '{"status": "OK", "message": null, "time": "2023-11-02 11:59:29 PM PDT-0700"}'}

Example fixing the sep token (eg <im_sep>, FUNC RET, ...) after JSON problem

Example bad LLM output:

{
  "function": "send_message",
  "params": {
    "message": "Welcome to our platform, Chad!"
  }
}<|im_end|>
<|im_start|>assistant

MemGPT output (without patch):

memgpt.errors.LocalLLMError: Failed to parse JSON from local LLM response - error: Failed to decode JSON from LLM output:
{
  "function": "send_message",
  "params": {
    "message": "Welcome to our platform, Chad!"
  }
}<|im_end|>
<|im_start|>assistant

MemGPT output (with patch):

💭 Bootup sequence complete. Persona activated. Testing messaging functionality.
🧑 {'type': 'login', 'last_login': 'Never (first login)', 'time': '2023-11-03 12:02:56 AM PDT-0700'}
Hit enter to begin (will request first MemGPT message)

Warning: no wrapper specified for local LLM, using the default wrapper (you can remove this warning by specifying the wrapper with --model)
💭 None
🤖 Welcome to our platform, Chad!

\dumpraw

{'role': 'function', 'name': 'send_message', 'content': '{"status": "OK", "message": null, "time": "2023-11-03 12:02:56 AM PDT-0700"}'}
{'role': 'user', 'content': '{"type": "login", "last_login": "Never (first login)", "time": "2023-11-03 12:02:56 AM PDT-0700"}'}
{'role': 'assistant', 'content': None, 'function_call': {'name': 'send_message', 'arguments': '{"message": "Welcome to our platform, Chad!"}'}}
{'role': 'function', 'name': 'send_message', 'content': '{"status": "OK", "message": null, "time": "2023-11-03 12:02:56 AM PDT-0700"}'}

[cpacker]

Add this in a separate PR to address #245

[cpacker]

This just counts opening vs closing brackets to try and pull the first JSON object out of a potential "run-on JSON object"

[cpacker]

Actually this should be added to the other wrappers too

[vivi]

🤌

[vivi]

LGTM 📜