Refactor Tokenizer

[Argmaster]

Currently tokenizer and token representation classes does not support dumping to some easy to compare format which could be used for testing. Additionally there is not simple way to add native implementation of tokenizer to provide optional speedup of tokenization process. Currently the idea would be to extract token filling logic from token classes and make them as simple containers as possible, hence allowing construction from C/C++/Rust implementation of tokenizer.