TestCase doesn't report test methods correctly when using phpdbg

[smuuf]

Version: 2.5.2, WSL (Ubuntu), but reproduced also in non-WSL Ubuntu.

Bug Description

tl;dr

When using phpdbg Nette Tester doesn't see the only test method defined in anonymous class which extends \Tester\TestCase class.

Slightly longer tl;dr

phpdbg and php interpreters seem to differ in how they behave when echoing a string containing a NULL byte. This causes the code ...

tester/src/Framework/TestCase.php

Lines 214 to 218 in 7fd3b98

	Environment::$checkAssertions = false;
	header('Content-Type: text/plain');
	echo "\n";
	echo 'TestCase:' . static::class . "\n";
	echo 'Method:' . implode("\nMethod:", $methods) . "\n";

... to result in bad (mangled) output when using phpdbg, which then causes Nette Tester to think there are no test case methods, if there is only one, when using an anonymous class extending a \Tester\TestCase class.

Explanation

1. phpdbg and php interpreters differ in how they behave when echoing a string containing a NULL byte.
   • Who knows why. I was unable to pinpoint the exact place in https://github.com/php/php-src which would explain this.
2. When using anonymous classes, PHP builds its name like so:
   • https://github.com/php/php-src/blob/d545b1d64350cac9cbf27859ad44d3ba32f6b736/Zend/zend_compile.c#L8266-L8284
   • ... which means the class name string contains a \0 NULL byte.
3. When this anonymous class's name is echoed, php somehow deals with (ignores?) the NULL byte, while phpdbg seems to stop the echo at that point.
   • How to reproduce? Create a file nullbyteclassname.php with this content:

   <?php
   
   $x = new class {
     function printMe() {
       echo "-START\n";
       echo "123" . static::class . "456\n";
       echo "-END\n";
     }
   };
   
   $x->printMe();

   • Then execute it both ways and see the results:
     • php

     [smuuf@smuuf-hp]$ php8.2 nullbyteclassname.php
     -START
     123class@anonymous/mnt/d/produkce/coding/php/php-nette-tester/nullbyteclassname.php:3$0456
     -END

     • phpdbg

     [smuuf@smuuf-hp]$ phpdbg8.2 -qrrb -S cli nullbyteclassname.php
     -START
     123class@anonymous-END

   • ⚠️ What phpdbg echoes is not a complete class name. And the 456\n is also trimmed.
4. In Nette Tester, the method TestCase::sendMethodList() reports a list of test methods using this code:

   echo "\n";
   echo 'TestCase:' . static::class . "\n";
   echo 'Method:' . implode("\nMethod:", $methods) . "\n";

   ... but due to the reasons stated above ...
   1. ❌ ... when using phpdbg, this results in mangled output, like so:

        NOTICE THIS -------------------++
                                       ||
                                       VV
      TestCase:Tester\TestCase@anonymousMethod:testMe
      Dependency:/mnt/d/produkce/coding/php/php-nette-tester/anonclasstestcase.phpt
      Dependency:/mnt/d/produkce/coding/php/php-nette-tester/src/Framework/TestCase.php
      

      ... which is incorrect (not expected) and information about the only test method is mixed on the line starting with TestCase: - thus the information about the only test method is lost. ❗
   2. ✅ ... on the other way php does it like so:

      TestCase:Tester\TestCase@anonymous/mnt/d/produkce/coding/php/php-nette-tester/anonclasstestcase.phpt:8$0
      Method:testMe
      Dependency:/mnt/d/produkce/coding/php/php-nette-tester/anonclasstestcase.phpt
      Dependency:/mnt/d/produkce/coding/php/php-nette-tester/src/Framework/TestCase.php
      

      ... which is correct (expected).

Steps To Reproduce

1. Clone nette/tester repo and install Composer dependencies.
2. Create a file anonclasstestcase.phpt in the repo's root dir:

   <?php
   
   require __DIR__ . '/vendor/autoload.php';
   
   use Tester\TestCase;
   use Tester\Assert;
   
   (new class extends TestCase {
       public function testMe(): void {
           Assert::true(true);
       }
   })->run();

3. Execute these and look at the results:
   • php

     php8.2 -d memory_limit=4096M -d register_argc_argv=on anonclasstestcase.phpt --method=nette-tester-list-methods

     Result:

     TestCase:Tester\TestCase@anonymous/mnt/d/produkce/coding/php/php-nette-tester/anonclasstestcase.phpt:8$0
     Method:testMe
     Dependency:/mnt/d/produkce/coding/php/php-nette-tester/anonclasstestcase.phpt
     Dependency:/mnt/d/produkce/coding/php/php-nette-tester/src/Framework/TestCase.php
     

   • phpdbg

     phpdbg8.2 -qrrb -d memory_limit=4096M -d register_argc_argv=on anonclasstestcase.phpt --method=nette-tester-list-methods

     Result:

     TestCase:Tester\TestCase@anonymousMethod:testMe
     Dependency:/mnt/d/produkce/coding/php/php-nette-tester/anonclasstestcase.phpt
     Dependency:/mnt/d/produkce/coding/php/php-nette-tester/src/Framework/TestCase.php
     

Expected Behavior

Even when using phpdbg Nette Tester should find the only test case method defined inside anonymous class which extends \Tester\TestCase class.

Possible Solution

It seems that preprocessing the result of static::class by removing "\0" from it solves the problem.

---

Possibly related:

• escapeshellarg(): Input string contains NULL byte with anonymous classes paratestphp/paratest#202
• Consider stripping null byte in frame name brendangregg/FlameGraph#280
• https://www.php.net/manual/en/language.oop5.anonymous.php#121021

[smuuf]

It looks like the underlying cause is in fact a bug in phpdbg, which I reported: php/php-src#14553 (comment)