README.md

You can evaluate ChartVLM model on ChartX benchmark via the following steps:

```
cd eval
```

## Download benchmark ChartX
Put ChartX under the path `eval/ChartX` 

## Inference using ChartVLM 
Use ChartVLM model to perform the inference process:

```
python infer_ChartX.py \
--model_path ${PATH_TO_CHARTVLM_MODEL}$ \
--benchmark_json_path ${PATH_TO_JSON_FILE_IN_OFIICIAL_CHARTX_FOLDER}$ 
```
The output json file will be saved under `eval/infer_result`.

## Evaluation on chart-related tasks
We adopt Structuring Chart-oriented Representation Metric (SCRM) for Structrual Extraction (SE) task, GPT-acc metric for Question-Answering task, and GPT-score metric for description task, summarization task and redrawing code task.

`${PATH_TO_INFERENCE_RESULT_JSON_FILE}$` indicates the inference results saved in `eval/infer_result`.

*For Structrual Extraction (SE):
```
python eval_SE_ChartX.py \ 
--infer_result_dir ${PATH_TO_INFERENCE_RESULT_JSON_FILE}$ 
```

*For Question Answering:
```
python eval_qa_ChartX.py \
--infer_result_dir ${PATH_TO_INFERENCE_RESULT_JSON_FILE}$ \ 
--your_openai_key ${YOUR_OPENAI_KEY}$
```

*For Description:
```
python eval_des_ChartX.py \
--infer_result_dir ${PATH_TO_INFERENCE_RESULT_JSON_FILE}$ \ 
--your_openai_key ${YOUR_OPENAI_KEY}$
```

*For Summarization:
```
python eval_sum_ChartX.py \
--infer_result_dir ${PATH_TO_INFERENCE_RESULT_JSON_FILE}$ \ 
--your_openai_key ${YOUR_OPENAI_KEY}$
```

*For Redrawing code:
```
python eval_redraw_code_ChartX.py \
--infer_result_dir ${PATH_TO_INFERENCE_RESULT_JSON_FILE}$ \ 
--your_openai_key ${YOUR_OPENAI_KEY}$
```
Note that all the evaluation result `log` file will be saved under `eval/eval_result`