Seeking Best Practice Advice: Displaying Matplotlib Chart in Playground Chat via Agent Toolkit

[NTTLuke]

Hello everyone,

I'm working on a specific scenario and would love to get your insights on the best practices for this implementation.

I've developed a toolkit using Matplotlib to create line charts, where I perform complex calculations to produce the final image. This toolkit, which is part of my agent's functionality, works well.
It generates the image as expected, which I can either save directly or convert to a base64 string.

The toolkit integrates smoothly within the agent's workflow, and the image creation process is functioning as intended. My current challenge is displaying the generated image directly in the Playground chat. I tried setting up a FastAPI endpoint to serve the images, and I've also tried prompting the agent with specific instructions to display the image. However, the image URL in the response has always the default "https://www.phidata.app/..." prefix and I'm not able to show the image in the chat.

Does anyone have experience with displaying images in the Playground or a best practice approach for achieving this?

Thanks in advance.

[ashpreetbedi]

@NTTLuke this is an amazing recommendation! we're going to be discussing internally today