[{%= name %}][author-www-url] [![npmjs.com][npmjs-img]][npmjs-url] [![The MIT License][license-img]][license-url] [![npm downloads][downloads-img]][downloads-url]
{%= description %} 🍋
[![travis build status][travis-img]][travis-url] [![coverage status][coveralls-img]][coveralls-url] [![dependency status][david-img]][david-url]
This is not finished yet, but go to examples directory and look deeply there. 🎉
We have few initial examples:
- advanced - making the lexer to not be "on per character basis", by overwriting the
.tokenize
method using plugin. - semver - tokenize semver string or alike
- simple - tokenize simple string (for example:
a > (b + 2)
) - csv - tokenize CSV string, and partially parsing
- more upcoming... going to port the PostCSS Tokenizer
And finally, after all, benchmarking.
npm i {%= name %} --save
For more use-cases see the tests
const {%= varname %} = require('{%= name %}')
const prevNext = require('{%= name %}-prev-next')
limon
.use(prevNext())
.use(function (app) {
return function (ch, i, input) {
// console.log('prev is:', this.prev())
// console.log('next is:', this.next())
if (/\s/.test(ch)) {
this.tokens.push(['whitespace', ch, i])
return
}
if (/\W/.test(ch)) {
this.tokens.push(['symbol', ch, i])
return
}
if (/\d/.test(ch)) {
this.tokens.push(['digit', ch, i])
return
}
this.tokens.push(['letter', ch, i])
}
})
var tokens = limon.tokenize('a > (b + 2)')
console.log(tokens)
// =>
// [ [ 'letter', 'a', 0 ],
// [ 'whitespace', ' ', 1 ],
// [ 'symbol', '>', 2 ],
// [ 'whitespace', ' ', 3 ],
// [ 'symbol', '(', 4 ],
// [ 'letter', 'b', 5 ],
// [ 'whitespace', ' ', 6 ],
// [ 'symbol', '+', 7 ],
// [ 'whitespace', ' ', 8 ],
// [ 'digit', '2', 9 ],
// [ 'symbol', ')', 10 ] ]
{%= apidocs('index.js') %}
{% if (verb.related && verb.related.list && verb.related.list.length) { %}
{%= related(verb.related.list, {words: 12}) %} {% } %}
Pull requests and stars are always welcome. For bugs and feature requests, [please create an issue](https://github.com/{%= repository %}/issues/new).
But before doing anything, please read the CONTRIBUTING.md guidelines.
Charlike Make Reagent [![new message to charlike][new-message-img]][new-message-url]
[![{%= author.username %}.tk][author-www-img]][author-www-url] [![keybase {%= author.username %}][keybase-img]][keybase-url] [![{%= author.username %} npm][author-npm-img]][author-npm-url] [![{%= author.username %} twitter][author-twitter-img]][author-twitter-url] [![{%= author.username %} github][author-github-img]][author-github-url]
{%= reflinks(verb.reflinks) %}
[npmjs-url]: https://www.npmjs.com/package/{%= name %} [npmjs-img]: https://img.shields.io/npm/v/{%= name %}.svg?label={%= name %}
[license-url]: https://github.com/{%= repository %}/blob/master/LICENSE [license-img]: https://img.shields.io/npm/l/{%= name %}.svg
[downloads-url]: https://www.npmjs.com/package/{%= name %} [downloads-img]: https://img.shields.io/npm/dm/{%= name %}.svg
[travis-url]: https://travis-ci.org/{%= repository %} [travis-img]: https://img.shields.io/travis/{%= repository %}/master.svg
[coveralls-url]: https://coveralls.io/r/{%= repository %} [coveralls-img]: https://img.shields.io/coveralls/{%= repository %}.svg
[david-url]: https://david-dm.org/{%= repository %} [david-img]: https://img.shields.io/david/{%= repository %}.svg
[author-www-url]: http://www.{%= author.username.toLowerCase() %}.tk [author-www-img]: https://img.shields.io/badge/www-{%= author.username.toLowerCase() %}.tk-fe7d37.svg
[keybase-url]: https://keybase.io/{%= author.username.toLowerCase() %} [keybase-img]: https://img.shields.io/badge/keybase-{%= author.username.toLowerCase() %}-8a7967.svg
[author-npm-url]: https://www.npmjs.com/~{%= author.username.toLowerCase() %} [author-npm-img]: https://img.shields.io/badge/npm-~{%= author.username.toLowerCase() %}-cb3837.svg
[author-twitter-url]: https://twitter.com/{%= author.username %} [author-twitter-img]: https://img.shields.io/badge/twitter-@{%= author.username %}-55acee.svg
[author-github-url]: https://github.com/{%= author.username %} [author-github-img]: https://img.shields.io/badge/github-@{%= author.username %}-4183c4.svg
[new-message-url]: https://github.com/{%= author.username %}/ama [new-message-img]: https://img.shields.io/badge/ask%20me-anything-green.svg