Skip to content

Commit

Permalink
[Doc] Update mmb_eval_gradio
Browse files Browse the repository at this point in the history
  • Loading branch information
kennymckormick committed Nov 23, 2024
1 parent 289bb86 commit ee7dfb7
Showing 1 changed file with 0 additions and 1 deletion.
1 change: 0 additions & 1 deletion scripts/mmb_eval_gradio.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@
HEADER = """
# Welcome to MMBench👏👏
We are delighted that you are willing to submit the evaluation results to the MMBench official website! The evaluation service currently can handle submissions of MMBench, MMBench-CN, and CCBench. We use `gpt-3.5-turbo-0125` to help answer matching. Evaluation Codes in VLMEvalKit: https://github.com/open-compass/VLMEvalKit. Please adopt / follow the implementation of VLMEvalKit to generate the submission files.
Moreover, this is a temporary solution, which **does not support ChatGPT-based answer extraction**. So you need to make sure values in the `prediction` field of your submission files are single characters in A, B, C, D. Other free-form answers can not be recognized by the evaluation script and will be marked as **WRONG**.
The evaluation script is available at https://github.com/open-compass/VLMEvalKit/tree/main/scripts/mmb_eval_gradio.py
Please contact `opencompass@pjlab.org.cn` for any inquirys about this script.
Expand Down

0 comments on commit ee7dfb7

Please sign in to comment.