CronQuestions[1] is the largest known Temporal KGQA dataset, clearly stratified into buckets of structural complexity. CronQuestions expands the only known previous dataset by a factor of 340×. This dataset consists of two parts: a KG with temporal annotations, and a set of natural language questions requiring temporal reasoning. Temporal KG was prepared by taking all facts with temporal annotations from the WikiData subset. The QA dataset was generated with a set of templates for temporal reasoning. These were made using the five most frequent relations from our WikiData subset, namely member of sports team, position held, award received, spouse, and employer.
| Model / System | Year | Hits@1 | Hits@10 | Reported by |
|---|---|---|---|---|
| TempoQR-Hard | 2021 | 0.918 | 0.978 | Mavromatis et. al. |
| TempoQR-Soft | 2021 | 0.799 | 0.957 | Mavromatis et. al. |
| TMA | 2021 | 0.784 | 0.943 | Liu et. al. |
| AE-TQ | 2022 | 0.769 | - | Long et. al. |
| EntityQR | 2021 | 0.745 | 0.944 | Mavromatis et. al. |
| CronKGQA | 2021 | 0.647 | 0.884 | Mavromatis et. al. |
| EaE | 2021 | 0.288 | 0.678 | Mavromatis et. al. |
| EmbedKGQA | 2021 | 0.288 | 0.672 | Mavromatis et. al. |
| T-EaE-add | 2021 | 0.278 | 0.663 | Saxena et. al. |
| BERT | 2021 | 0.243 | 0.620 | Mavromatis et. al. |
| RoBERTa | 2021 | 0.225 | 0.585 | Mavromatis et. al. |
| BERT | 2021 | 0.071 | 0.213 | Saxena et. al. |
| RoBERTa | 2021 | 0.07 | 0.202 | Saxena et. al. |
| KnowBERT | 2021 | 0.07 | 0.201 | Saxena et. al. |
| T5-3B | 2021 | 0.081 | - | Saxena et. al. |
[1] Saxena, Apoorv, Soumen Chakrabarti, and Partha Talukdar. Question Answering Over Temporal Knowledge Graphs. arXiv preprint arXiv:2106.01515 (2021).