Version 3.1.0
- Added
-o
switch totokenize
command to return original token text, enabling the tokenizer to run as a sentence splitter only.
-o
switch to tokenize
command to return original token text, enabling the tokenizer to run as a sentence splitter only.