IndexError when running evaluation

[YourSaDady]

Hi. I tried to follow the README example, and when I came to the Evaluation step, an IndexError occurred when I ran python evaluate.py --model openlm-research/open_llama_3b --domain ID --result ParaRel_openllama_3b:

I think the error is due to that in evaluate.py, the SURE and UNSURE are lists of encodings but not their ids:
SURE.append(tokenizer("sure")[0]) UNSURE.append(tokenizer("unsure")[0])

[shizhediao]

Hi,
Thank you very much for your comment!
We updated our code at #6. Please pull the latest code and test it again.
Thanks!