Add mean reciprocal rank (MRR) computation in KGEModel

[songsong0425]

Greetings, thank you for confirming this request.
I tried to add mean reciprocal rank (MRR) in the torch_geometric/nn/kge/base.py and examples/kge_fb15k_237.py.
(Please see the related discussion in here)
If there are any problems, please forgive me and freely edit the code.
Thanks!

[songsong0425]

Are there any additional variables for the evaluation metrics? I couldn't understand why I faced this error after add mrr to base.py and kge_fb15k_237.py:

File "/home/songyeon/PyG_contribution/pytorch_geometric/examples/kge_fb15k_237.py", line 83, in <module>
    rank, mrr, hits = test(val_data)
ValueError: not enough values to unpack (expected 3, got 2)

[rusty1s]

Are there any additional variables for the evaluation metrics? I couldn't understand why I faced this error after add mrr to base.py and kge_fb15k_237.py:

Looks like you didn't install the package with your corresponding changes?

[songsong0425]

Thank you for editing! At that time, that error was evoked in the local debugging process. Now I confirmed the changes and know how it works. Again, thank you for your support :)

[songsong0425]

@rusty1s Greetings, I'd like to ask a question about this job. When I downloaded pyg-nightly 2.4.0.dev20231126 and ran my code, it returned the error I reported on Oct 31. I'll be glad if you check the error.

# Reference: kge_fb15k_237.py 
@torch.no_grad()
def test(data):
    model_opt = model.to(device)
    model_opt.load_state_dict(torch.load('checkpoint.pt', map_location=device))
    model_opt.eval()

    return model_opt.test(
        head_index=data.edge_type_index[0],
        rel_type=data.edge_type,
        tail_index=data.edge_type_index[1],
        batch_size=20000,
        k=10
    )

num_epochs = 10
train(num_epochs)

rank, mrr, hits_at_10 = test(test_data)
print(f'Test Mean Rank: {rank:.2f}, Test MRR: {mrr:.4f}, Test Hits@10: {hits_at_10:.4f}')

Regards

[rusty1s]

Which error are you seeing? What does

import torch_geometric
print(torch_geometric.__version__)

return?

[songsong0425]

The error is as below:

Traceback (most recent call last):
  File "/scratch/r902a02/KGEModels/ComplEx/kge_comparison_A100_latest.py", line 139, in <module>
    rank, mrr, hits_at_10 = test(test_data)
ValueError: not enough values to unpack (expected 3, got 2)

and print(torch_geometric.__version__) returned 2.4.0.

[rusty1s]

It looks like you have both PyG and pyg-nightly installed. Can you run

pip uninstall torch-geometric

repeatedly, and then install via pip install pyg-nightly. print(torch_geometric.__version__) should actually return the exact nightly version.

[songsong0425]

Sorry for the late reply. I fixed it. Thank you so much!