[Bug] Fix the Image Input of Batch Generation

[OBJECT907]

Motivation

When using the OpenAI API for batch generation, all prompts receive the same image, specifically the last image in the batch. Upon investigation, I discovered a bug in openai_api/adapter.py:

• Issue in v1_chat_generate_request (line 824): The image_data_list gathers all images for the batch. However, on line 938, the variable adapted_request = GenerateReqInput(...) incorrectly uses image_data (a temporary variable inside the loop) as an argument, which leads to all prompts receiving the last image in the batch.

• Additionally, in srt/managers/io_struct.py, line 136, the self.image_data will duplicate num times, preventing any warning or indexing error but causing incorrect behavior.

Modifications

• Replaced image_data with image_data_list in v1_chat_generate_request to ensure each prompt receives the correct image from the batch.
• Removed the duplication of images.
• Tested the fix with examples/runtime/openai_batch_chat.py and it now works as expected.

Checklist

• Format your code according to the Contributor Guide.
• Add unit tests as outlined in the Contributor Guide.
• Update documentation as needed, including docstrings or example tutorials.

[merrymercy]

Good catch! Can you add a test case under https://github.com/sgl-project/sglang/blob/main/test/srt/test_vision_openai_server.py?

[OBJECT907]

Good catch! Can you add a test case under https://github.com/sgl-project/sglang/blob/main/test/srt/test_vision_openai_server.py?

Can I add a .jsonl file in this directory? The batch request needs to upload an input file, like in https://github.com/sgl-project/sglang/blob/main/examples/runtime/openai_batch_chat.py

[merrymercy]

You can use code to write that to a temporary json file

[merrymercy]

Let me merge this first as it is a bug fix. In the meanwhile, a unit test will be great for us to prevent this from happening again in the future. Could you send a new follow-up PR?