Evaluation

Named Entity Recognition Performance

The model file that comes with MITIE gets an F1 score of 88.10 on the CoNLL 2003 NER task. This is measured on the eng.testb test dataset. The model was trained on the train/val data from the CoNLL 2003 task.

These results are comparable to other state-of-the-art NER systems like the Stanford NER tool.

Named Entity Recognition Speed

MITIE's NER implementation is designed for bulk data processing at high speeds. We measured this by running part of the English Gigaword corpus through MITIE and measuring the total processing time. On a 2.40GHz Intel Xeon (E5-2665, single-threaded), MITIE is able to process 53,600 words per second. This is significantly faster than most research NER systems available via open source licenses.