Skip to content

Commit

Permalink
Tokenizers/PHP: bug fix - empty block comment
Browse files Browse the repository at this point in the history
This commit fixes an edge case tokenizer bug, where a - completely empty, not even whitespace - _block comment_, would be tokenized as a docblock.

Without this commit, the `/**/` code snippet was tokenized as:
```
  8 | L07 | C  1 | CC 0 | ( 0) | T_DOC_COMMENT_OPEN_TAG     | [  4]: /**/
  9 | L07 | C  5 | CC 0 | ( 0) | T_DOC_COMMENT_CLOSE_TAG    | [  0]:
```

With the fix applied, it will be tokenized as:
```
  8 | L07 | C  1 | CC 0 | ( 0) | T_COMMENT                  | [  4]: /**/
```
  • Loading branch information
jrfnl committed May 18, 2024
1 parent d96c7d6 commit c54cb10
Show file tree
Hide file tree
Showing 3 changed files with 24 additions and 1 deletion.
2 changes: 1 addition & 1 deletion src/Tokenizers/PHP.php
Original file line number Diff line number Diff line change
Expand Up @@ -786,7 +786,7 @@ protected function tokenize($string)

if ($tokenIsArray === true
&& ($token[0] === T_DOC_COMMENT
|| ($token[0] === T_COMMENT && strpos($token[1], '/**') === 0))
|| ($token[0] === T_COMMENT && strpos($token[1], '/**') === 0 && $token[1] !== '/**/'))
) {
$commentTokens = $commentTokenizer->tokenizeString($token[1], $this->eolChar, $newStackPtr);
foreach ($commentTokens as $commentToken) {
Expand Down
3 changes: 3 additions & 0 deletions tests/Core/Tokenizer/Comment/SingleLineDocBlockTest.inc
Original file line number Diff line number Diff line change
@@ -1,5 +1,8 @@
<?php

/* testEmptyBlockCommentNoWhiteSpace */
/**/

/* testEmptyDocblockWithWhiteSpace */
/** */

Expand Down
20 changes: 20 additions & 0 deletions tests/Core/Tokenizer/Comment/SingleLineDocBlockTest.php
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,26 @@ public static function dataDocblockOpenerCloser()
}//end dataDocblockOpenerCloser()


/**
* Verify an empty block comment is tokenized as T_COMMENT, not as a docblock.
*
* @covers PHP_CodeSniffer\Tokenizers\PHP::tokenize
*
* @return void
*/
public function testEmptyBlockCommentNoWhiteSpace()
{
$expectedSequence = [
[T_COMMENT => '/**/'],
];

$target = $this->getTargetToken('/* '.__FUNCTION__.' */', [T_COMMENT, T_DOC_COMMENT_OPEN_TAG]);

$this->checkTokenSequence($target, $expectedSequence);

}//end testEmptyBlockCommentNoWhiteSpace()


/**
* Verify tokenization of an empty, single line DocBlock.
*
Expand Down

0 comments on commit c54cb10

Please sign in to comment.