Question about caption evaluation results

[TTXiann]

i use the pretrain weight "scanrefer_scst_vote2cap_detr_pp_XYZ_RGB_NORMAL.pth" and get the follow result:

INFO: iou@0.5 matched proposals: [1537 / 2068],
[BLEU-1] Mean: 0.6676, Max: 1.0000, Min: 0.0000
[BLEU-2] Mean: 0.5745, Max: 1.0000, Min: 0.0000
[BLEU-3] Mean: 0.4757, Max: 1.0000, Min: 0.0000
[BLEU-4] Mean: 0.3895, Max: 1.0000, Min: 0.0000
[CIDEr] Mean: 0.7525, Max: 6.3784, Min: 0.0000
[ROUGE-L] Mean: 0.5467, Max: 1.0000, Min: 0.1015
[METEOR] Mean: 0.2631, Max: 1.0000, Min: 0.0448

This result is not consistent with the paper, is this normal?

[ch3cook-fdu]

The result varies mainly because of: 1. The randomness in data pre-processing (point down sampling), and 2. Different PyTorch versions.
Thus it is common that the performance may vary a little.

It is encouraged to execute all the training sessions.

[TTXiann]

The result varies mainly because of: 1. The randomness in data pre-processing (point down sampling), and 2. Different PyTorch versions. Thus it is common that the performance may vary a little.

It is encouraged to execute all the training sessions.

OK, I re-run the data preprocessing and got the following results：

INFO: iou@0.5 matched proposals: [1575 / 2068],
[BLEU-1] Mean: 0.6838, Max: 1.0000, Min: 0.0000
[BLEU-2] Mean: 0.5880, Max: 1.0000, Min: 0.0000
[BLEU-3] Mean: 0.4855, Max: 1.0000, Min: 0.0000
[BLEU-4] Mean: 0.3970, Max: 1.0000, Min: 0.0000
[CIDEr] Mean: 0.7662, Max: 6.7565, Min: 0.0000
[ROUGE-L] Mean: 0.5550, Max: 1.0000, Min: 0.1015
[METEOR] Mean: 0.2660, Max: 1.0000, Min: 0.0448

This proves that randomness does exist. Thank you for your reply!