Test inline @link in Lexer

[andrewnicols]

This is a followup to #261. Whilst the parsing of the tags themselves is correct in phpstan, Rector uses the PHPStan Lexer to tokenize the values. It seems that the tokenisation bundles the closing curly brace into the previous text content unless there is whitespace between them.

This test demonstrates that issue.

This issue is also present for opening/closing angle brackets, and parens. It seems that the OTHER token is too greedy.

It may be that this is the intended behaviour of the Lexer but I haven't been able to find any documentation or information to confirm either way.

rectorphp/rector#8977 has additional info on the issue and how it is encountered.

[ondrejmirtes]

A little personal rant here: I appreciate what the Rector guys do, but they always implement hacky workarounds for bugs or missing features in PHPStan and phpdoc-parser. They are the heaviest users of PHPStan's internals but they rarely open any issues or pull requests.

If they properly reported their needs, I could address them here and they could use the parser in a clean way, without the need to descend into the lexer.

As an example, slevomat/coding-standard does this right. When I implemented the format-preserving printer, slevomat/coding-standard adopted in saved literally thousands of lines of code in their repository: slevomat/coding-standard#1577

If Rector collaborated more and used PHPStan in a proper way without workarounds, it'd be easier for everyone.

[andrewnicols]

Sorry @ondrejmirtes, I can only agree. I'm just a frustrated user trying to work out why my rules aren't being applied properly and trying to take the time to not only report a bug but try and find a fix at the same time. I've only started using Rector in the past few days.

Do you think that this is an expected behaviour in the Lexer then?

[ondrejmirtes]

I'd say the specific representation in the lexer doesn't matter as long as the parser consumes it and produces proper AST.