ChatML/ChatMessage Prompt Templates

[tuan3w]

Describe the feature or potential improvement

Hi langfuse team,

Chat llm models of OpenAI support role-based messages. From our observation as well as from the community, using role-based prompts are often better. So it would be nice if langfuse can support this type of template in prompt management.
Thanks.

Additional information

No response

[marcklingen]

Do you mean ChatML?

Example

messages=[
      {"role": "system", "content": "You are a helpful assistant."},
      {"role": "user", "content": "Who won the world series in 2020?"},
      {"role": "assistant", "content": "The Los Angeles Dodgers won the World Series in 2020."},
      {"role": "user", "content": "Where was it played?"}
 ]

Would you want to edit a single message or an array of messages via Langfuse prompt management?

[tuan3w]

@marcklingen
Yes, that's what I meant.

[marcklingen]

awesome, thanks for confirming. currently you can probably get to a workaround using the prompt.config in langfuse but this is something we want to support with langfuse prompt management more natively in the future

[lmyslinski]

I'm really surprised this isn't higher up. Single message prompts are de-facto obsolete by now, as every LLM library uses the chat message schema (system/user/assistant) as their defaults. OpenAI already refers to this schema as "Legacy": https://platform.openai.com/docs/guides/text-generation/completions-api

+1

[marcklingen]

We have added this a while back in April, marking this as Done

https://langfuse.com/changelog/2024-04-09-prompt-management-chatml

Thanks again for adding this here!