Supply images with OpenAI API text generation

[SinanAkkoyun]

Hello! Is it possible to supply text with special tokens that get tokenized? I need to give a model tool calling functionality but as SGLang doesn't support it I'd like to parse them manually and pass them into system prompt
Thanks!

[merrymercy]

Yes, it is possible.

https://sgl-project.github.io/references/sampling_params.html#multi-modal
and
https://sgl-project.github.io/backend/openai_api_vision.html