Use NLTK Tokeniser

[meshde]

EruditeX/Helpers/utils.py

Line 206 in 4acd8e0

for word in sentence.split():

This line uses the python string function split() for tokenization. Using the nltk tokenizer would prove better for this purpose.