You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I believe the current configuration will never use gdeval by default for ndcg@k. But when the component is used, it produces different ndcg@k values than pytrec_eval. For instance, on a bm25 run on the COIVID dataset for ndcg@10, we get:
I believe the current configuration will never use gdeval by default for ndcg@k. But when the component is used, it produces different ndcg@k values than pytrec_eval. For instance, on a bm25 run on the COIVID dataset for ndcg@10, we get:
This seems to be because gdeval uses a variant of ndcg that places higher importance on relevant documents.
This should be parameterized. Something like
ndcg_exprel@k
?The text was updated successfully, but these errors were encountered: