Tokenize a word in sentence using SPACY

This program determines the TEXT, LEMMA, POS, TAG, DEP, SHAPE, ALPHA, STOP of a word in a sentence using SPACY.

TEXT - the actual text used from the input LEMMA - base form of the word/text POS - the simple part-of-speech tag. TAG - the detailed part-of-speech tag. DEP - Syntactic dependency, i.e. the relation between tokens. SHAPE - the word shape – capitalization (X,x), punctuation, digits (d) ALPHA - Is the token an alpha character? STOP - Is the token part of a stop list, i.e. the most common words of the language?

Usage:

$ python tokenize_word.py