[tokenizer] don't reset comment state in case of long endings

[AndreasMadsen]

Found the bug when scraping http://www.guardian.co.uk/world/2013/jun/06/florida-tropical-storm-andrea

See testcase below:

var Parser = require('htmlparser2').Parser;

var stream = new Parser({
  onopentag: function (tagname, attr) {
    console.log('open: ' + tagname, attr);
  },

  oncomment: function (value) {
    console.log('open comment: ' + value);
  },

  ontext: function (text) {
    console.log('text: ' + text);
  },

  oncommentend: function () {
    console.log('close comment');
  },

  onclosetag: function (tagname) {
    console.log('close: ' + tagname);
  }
});

stream.write('<meta id="before">');
stream.write('<!-- text --->');
stream.write('<meta id="after">');
stream.end();

Outputs this on master:

open: meta { id: 'before' }
close: meta
open comment:  text ---><meta id="after">
close comment

with this fix:

open: meta { id: 'before' }
close: meta
open comment:  text -
close comment
open: meta { id: 'after' }
close: meta

[AndreasMadsen]

That was quick :)

[fb55]

Good catch, thanks!

There should be a similar issue with CDATA, where ]]]> is ignored. The AFTER_CDATA_2 state needs a similar check (for ]). It would be great if you could fix that too (or at least contribute a test case).