Skip to content

Latest commit

 

History

History
107 lines (80 loc) · 5.61 KB

README.md

File metadata and controls

107 lines (80 loc) · 5.61 KB
Raw

This Repository contains the code which implements the approach described in the following Arxiv Preprint: https://arxiv.org/abs/1610.09756 which is published in ICON-16 conference (http://aclweb.org/anthology/W/W16/W16-63.pdf).

Contributors

  • shreenibhar (Shreenivas Bharadwaj)
  • monikkinom
  • vinayakathavale

ner-lstm

Requirements

Named Entity Recognition using multi-layered bidirectional LSTMs and task adapted word embeddings

Named Entity Recognition is a classification problem of identifying the names of people,organisations,etc (different classes) in a text corpus. Previous approaches to the problems have involved the usage of hand crafted language specific features, CRF and HMM based models, gazetteers, etc. Growing interest in deep learning has led to application of deep neural networks to the existing problems like that of NER. We have implemented a 2 layer bidirectional LSTM network using tensorflow to classify the named entities for CoNNL 2003 NER Shared Task. Classification on the NER Hindi dataset of icon-2013 proceedings was also done. The process and code usage are given below. All codes use argparse for intuitive usage.

Generating the embedding model

Sentences are used as inputs for the recurrent neural network. Representation of words in the sentence is via the form of embeddings. Hence the features for the recurrent neural network are sentences a.k.a sequence of words a.k.a sequence of embeddings. Each unique word should have certain number of features, these are called embeddings or also vectors. These are the input features to the neural architecture we are using.

English

You need a corpus comprising of text separated by only spaces if you are training a model. First train the model or load from an existing one from the files given in embeddings. wordvec_model.py - creates a model of word2vec, 2 ways to create the model either by supplying a corpus to train or restore from word2vec gensim bin file. glove_model.py - creates a model of glove, 2 ways to create the model either by supplying a corpus to train or restore from a glove vector.txt file. Copy corpus in Glove-1.2 directory and run the code from embeddings folder and give name of corpus as param. rnnvec_model.py - creates a model of LSTM, only way is by supplying a corpus.

Hindi

Follow the same steps as english but first convert corpus to english type using hindi_util.py. ######Comparison We have done a comparison between 111 dimension embedding models by training all of them on a small 100mb corpus and evaluating on the conll ner dataset.

Model Test_a Test_b
Word2Vec 88.33 83.40
Glove 89.62 83.10
RnnVec 81.07 75.20

Preparing the inputs

Now we have the embedding model, we have to use that to convert our sentences of words to sentences of embeddings. First use resize_input.py to resize your data set to a max sentence length. Use the trained embedding model along with get_conll_embeddings.py or get_icon_embeddings.py for conll and icon respectively to get the pickled input data ready to be fed and train the recurrent neural network. Note that we are adding 11 extra features here to the embeddings themselves which include pos, chunk and capital features of the word.

Deep Neural Network

We have used Google's Tensorflow to implement a bidirectional multilayered rnn cell (LSTM). The hyper parameters are present at the top in main.py. Tweaking the parameters can yield a variety of results which are worth noting. We have used a softmax layer as the last layer of the network to produce the final classification outputs. We tried working with different optimizers and we found that AdamOptimzer produced the best results. The function to calculate the F1 Scores, Prediction Accuracy and Recall is also included in model.py. We have also included the ability to save/restore an existing model using tensorflow's saver functions. The path of the generated pickle file from above needs to be set in input.py. Use the model.py to run the deep neural network which will start running and optimizing the F1 scores.

Final Results

Dataset Model Embedding size Test_a Test_b
CONLL Glove 311 93.99 90.32
CONLL Word2Vec 311 93.5 89.4
ICON Glove 311 78.6 77.48

CONLL samples

A sample result produced by conll eval script is presented here.

Word2Vec 311 dimensions
  • Test_a

processed 49644 tokens with 8211 phrases; found: 8080 phrases; correct: 7619. Accuracy = 98.54%

Class Precision Recall FB1 Numbers
NER 94.29 92.79 93.54 8080
LOC 94.86 94.25 94.56 2023
MISC 91.99 85.09 88.40 1123
ORG 91.98 89.76 90.86 2020
PER 96.40 97.16 96.78 2914
  • Test_b

processed 45151 tokens with 7719 phrases; found: 7740 phrases; correct: 6911. Accuracy = 97.49%

Class Precision Recall FB1 Numbers
NER 89.29 89.53 89.41 7740
LOC 89.67 90.87 90.27 1898
MISC 75.80 75.80 75.80 905
ORG 88.03 87.27 87.65 2415
PER 95.04 95.69 95.37 2522

Papers on WordVectors

https://papers.nips.cc/paper/5021-distributed-representations-of-words-and-phrases-and-their-compositionality.pdf http://www-personal.umich.edu/~ronxin/pdf/w2vexp.pdf Good tutorial on WordVectors: https://www.tensorflow.org/versions/r0.9/tutorials/word2vec/index.html