This is the official repository for paper:First Multi-Dimensional Evaluation of Flowchart Comprehension for Multimodal Large Language Models arxiv link
FlowCE is built upon 500 real-world flowcharts, ensuring an ample diversity in each chart and across five dimensions in real flowchart scenarios, including reasoning, information extraction, localization recognition, summarization, and logical verification.
The full datasets can be obtained from the following address: https://github.com/360AILAB-NLP/FlowCE/tree/main/flowce_benchmark
For your attention: The image data used in this work is solely for scientific research purposes, and only the source link address for each image is made available. if you want to get the images, just download these images from the 5 different task image urls file given.
The evaluation code will be published soon..