Tests: allow for running on PHPUnit 10/11

Until now, the tests would run on PHPUnit 9 for PHP 7.3 and higher.

PHPUnit 10 (PHP 8.1+) was released in February 2023, PHPUnit 11 (PHP 8.2+) in February 2024 and the update for this test suite is quite painless, so let's allow for running the tests on PHPUnit 10 and 11.

This commit actions this.

Things to be aware of:

Handling of PHPUnit deprecations

Prior to PHPUnit 10.5.32 and PHPUnit 11.3.3, if the `failOnDeprecation` attribute was set (previously: `convertDeprecationsToExceptions`), tests runs would not only fail on PHP deprecation notices, but also on PHPUnit native deprecation notices when running on PHPUnit 10/11.

This undesireable behaviour has now (finally) been reverted, which makes updating test suites for compatibility with PHPUnit 10/11 a lot more straight-forward.

This is the reason that the minimum PHPUnit 10/11 versions in the `composer.json` file are set to such specific versions.

As part of this change, PHPUnit 10.5.32 and 11.3.3 introduce two new options for both the configuration file and as CLI arguments: `displayDetailsOnPhpunitDeprecations` and `failOnPhpunitDeprecation`, both of which will default to `false`.

These options have been added to the PHPUnit configuration for PHPUnit 10/11 to safeguard them against changes in the default value.

Test configuration file changes

The configuration file specification has undergone changes in PHPUnit 9.3 and 10.0.

Most notably:
* In PHPUnit 9.3, the manner of specifying the code coverage configuration has changed.
* In PHPUnit 10.0, a significant number of attributes of the `<phpunit>` element were removed or renamed.

While the `--migrate-configuration` command can upgrade a configuration for the changes in the format made in PHPUnit 9.3, some of the changes in the configuration format in PHPUnit 10 don't have one-on-one replacements and/or are not taken into account.

With that in mind, I deem it more appropriate to have a dedicated PHPUnit configuration file for PHPUnit 10+ to ensure the test run will behave as intended.

To that end:
* The `phpunit.xml.dist` file has been renamed to `phpunitlte9.xml.dist` (for PHPUnit <= 9) and a new `phpunit.xml.dist` file has been added for PHPUnit 10+.
* The renamed file is set to be excluded from release archives via `.gitattributes` and a potential local overload file for it is set to be ignored by default via the `.gitignore` file.
* Additional scripts have been added to the `composer.json` file to run the tests either with the new or the old configuration file.
* The GH Actions `test` workflow has been updated to select the correct configuration file based on the PHPUnit version on which the tests are being run.

Other command/config changes

Also note that the `--coverage-cache <dir>` CLI flag was deprecated in PHPUnit 10 and removed in PHPUnit 11. A generic `cacheDirectory` attribute is the recommended replacement (though a CLI option is also available).
The new attribute has been added to the PHPUnit 10+ config file, while for PHPUnit 9.3 - 9.5, the flag is still added to the command in CI (as adding the PHPUnit 9.3 equivalent to the config file would make the config incompatible with PHPUnit < 9.3).

Composer will not install PHPUnit 11

Psalm 5 has a dependency on PHP-Parser 4, while PHPUnit 11 requires PHP-Parser 5, so Psalm currently effectively blocks the installation of PHPUnit 11 via Composer.
Running the tests on PHPUnit 11 will work fine though (tested with a PHAR file).

Ref:
* https://phpunit.de/announcements/phpunit-11.html
* https://phpunit.de/announcements/phpunit-10.html

.gitattributes

@@ -5,13 +5,14 @@
 # https://www.reddit.com/r/PHP/comments/2jzp6k/i_dont_need_your_tests_in_my_production
 # https://blog.madewithlove.be/post/gitattributes/
 #
-/.gitattributes    export-ignore
-/.gitignore        export-ignore
-/phpcs.xml.dist    export-ignore
-/phpstan.neon.dist export-ignore
-/phpunit.xml.dist  export-ignore
-/.github           export-ignore
-/Tests/            export-ignore
+/.gitattributes       export-ignore
+/.gitignore           export-ignore
+/phpcs.xml.dist       export-ignore
+/phpstan.neon.dist    export-ignore
+/phpunit.xml.dist     export-ignore
+/phpunitlte9.xml.dist export-ignore
+/.github              export-ignore
+/Tests/               export-ignore
 
 #
 # Auto detect text files and perform LF normalization

.github/workflows/test.yml

@@ -111,8 +111,21 @@ jobs:
           composer-options: --ignore-platform-req=php
           custom-cache-suffix: $(date -u "+%Y-%m")
 
+      - name: Grab PHPUnit version
+        id: phpunit_version
+        run: echo "VERSION=$(vendor/bin/phpunit --version | grep --only-matching --max-count=1 --extended-regexp '\b[0-9]+\.[0-9]+')" >> $GITHUB_OUTPUT
+
+      - name: Determine PHPUnit config file to use
+        id: phpunit_config
+        run: |
+          if [ "${{ startsWith( steps.phpunit_version.outputs.VERSION, '10.' ) || startsWith( steps.phpunit_version.outputs.VERSION, '11.' ) }}" == "true" ]; then
+            echo 'FILE=phpunit.xml.dist' >> $GITHUB_OUTPUT
+          else
+            echo 'FILE=phpunitlte9.xml.dist' >> $GITHUB_OUTPUT
+          fi
+
       - name: Run the unit tests
-        run: composer test
+        run: vendor/bin/phpunit --no-coverage -c ${{ steps.phpunit_config.outputs.FILE }}
 
   #### CODE COVERAGE STAGE ####
   # N.B.: Coverage is only checked on the lowest and highest stable PHP versions
@@ -178,21 +191,30 @@ jobs:
       - name: "DEBUG: Show grabbed version"
         run: echo ${{ steps.phpunit_version.outputs.VERSION }}
 
+      - name: Determine PHPUnit config file to use
+        id: phpunit_config
+        run: |
+          if [ "${{ startsWith( steps.phpunit_version.outputs.VERSION, '9.' ) && steps.phpunit_version.outputs.VERSION >= '9.3' }}" == "true" ]; then
+            echo 'FILE=phpunitlte9.xml.dist' >> $GITHUB_OUTPUT
+            echo 'EXTRA_ARGS=--coverage-cache ./build/phpunit-cache' >> $GITHUB_OUTPUT
+          elif [ "${{ startsWith( steps.phpunit_version.outputs.VERSION, '10.' ) || startsWith( steps.phpunit_version.outputs.VERSION, '11.' ) }}" == "true" ]; then
+            echo 'FILE=phpunit.xml.dist' >> $GITHUB_OUTPUT
+            echo 'EXTRA_ARGS=' >> $GITHUB_OUTPUT
+          else
+            echo 'FILE=phpunitlte9.xml.dist' >> $GITHUB_OUTPUT
+            echo 'EXTRA_ARGS=' >> $GITHUB_OUTPUT
+          fi
+
       # PHPUnit 9.3 started using PHP-Parser for code coverage which can cause interference.
       # As of PHPUnit 9.3.4, a cache warming option is available.
       # Using that option prevents issues with PHP-Parser backfilling PHP tokens when PHPCS does not (yet),
       # which would otherwise cause tests to fail on tokens being available when they shouldn't be.
       - name: "Warm the PHPUnit cache (PHPUnit 9.3+)"
         if: ${{ steps.phpunit_version.outputs.VERSION >= '9.3' }}
-        run: vendor/bin/phpunit --coverage-cache ./build/phpunit-cache --warm-coverage-cache
-
-      - name: "Run the unit tests with code coverage (PHPUnit < 9.3)"
-        if: ${{ steps.phpunit_version.outputs.VERSION < '9.3' }}
-        run: composer coverage
+        run: vendor/bin/phpunit -c ${{ steps.phpunit_config.outputs.FILE }} ${{ steps.phpunit_config.outputs.EXTRA_ARGS }} --warm-coverage-cache
 
-      - name: "Run the unit tests with code coverage (PHPUnit 9.3+)"
-        if: ${{ steps.phpunit_version.outputs.VERSION >= '9.3' }}
-        run: composer coverage -- --coverage-cache ./build/phpunit-cache
+      - name: "Run the unit tests with code coverage"
+        run: vendor/bin/phpunit -c ${{ steps.phpunit_config.outputs.FILE }} ${{ steps.phpunit_config.outputs.EXTRA_ARGS }}
 
       - name: Upload coverage results to Coveralls
         if: ${{ success() }}

.gitignore

@@ -2,6 +2,7 @@
 /vendor/
 /build/
 phpunit.xml
+phpunitlte9.xml
 .phpunit.result.cache
 .phpcs.xml
 phpcs.xml

composer.json

@@ -41,6 +41,8 @@
     "scripts": {
         "test": "./vendor/bin/phpunit --no-coverage",
         "coverage": "./vendor/bin/phpunit",
+        "test-lte9": "./vendor/bin/phpunit -c phpunitlte9.xml.dist --no-coverage",
+        "coverage-lte9": "./vendor/bin/phpunit -c phpunitlte9.xml.dist",
         "lint": "./vendor/bin/phpcs",
         "fix": "./vendor/bin/phpcbf",
         "phpstan": "./vendor/bin/phpstan analyse",
@@ -52,7 +54,7 @@
         "squizlabs/php_codesniffer": "^3.5.6"
     },
     "require-dev": {
-        "phpunit/phpunit": "^4.8.36 || ^5.7.21 || ^6.5 || ^7.0 || ^8.0 || ^9.0",
+        "phpunit/phpunit": "^4.8.36 || ^5.7.21 || ^6.5 || ^7.0 || ^8.0 || ^9.0 || ^10.5.32 || ^11.3.3",
         "sirbrillig/phpcs-import-detection": "^1.1",
         "phpcsstandards/phpcsdevcs": "^1.1",
         "phpstan/phpstan": "^1.7",

phpunit.xml.dist

@@ -1,27 +1,37 @@
 <?xml version="1.0" encoding="UTF-8"?>
 <phpunit
 	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
-	xsi:noNamespaceSchemaLocation="http://schema.phpunit.de/9.0/phpunit.xsd"
-	bootstrap="Tests/bootstrap.php"
-	convertErrorsToExceptions="true"
-	convertWarningsToExceptions="true"
-	convertNoticesToExceptions="true"
-	convertDeprecationsToExceptions="true"
+	xsi:noNamespaceSchemaLocation="http://schema.phpunit.de/10.5/phpunit.xsd"
+		bootstrap="Tests/bootstrap.php"
+		cacheDirectory="build/.phpunit-cache"
+		displayDetailsOnTestsThatTriggerErrors="true"
+		displayDetailsOnTestsThatTriggerWarnings="true"
+		displayDetailsOnTestsThatTriggerNotices="true"
+		displayDetailsOnTestsThatTriggerDeprecations="true"
+		displayDetailsOnIncompleteTests="true"
+		displayDetailsOnSkippedTests="true"
+		displayDetailsOnPhpunitDeprecations="false"
+		failOnWarning="true"
+		failOnNotice="true"
+		failOnDeprecation="true"
+		failOnPhpunitDeprecation="false"
 	>
 	<testsuites>
 		<testsuite name="VariableAnalysis">
 			<directory>Tests</directory>
 		</testsuite>
 	</testsuites>
 
-	<filter>
-		<whitelist addUncoveredFilesFromWhitelist="true" processUncoveredFilesFromWhitelist="false">
+	<source>
+		<include>
 			<directory>./VariableAnalysis/</directory>
-		</whitelist>
-	</filter>
+		</include>
+	</source>
 
-	<logging>
-		<log type="coverage-text" target="php://stdout" showOnlySummary="true"/>
-		<log type="coverage-clover" target="build/logs/clover.xml"/>
-	</logging>
+	<coverage includeUncoveredFiles="true" ignoreDeprecatedCodeUnits="true">
+		<report>
+			<clover outputFile="build/logs/clover.xml"/>
+			<text outputFile="php://stdout" showOnlySummary="true"/>
+		</report>
+	</coverage>
 </phpunit>

phpunitlte9.xml.dist

@@ -0,0 +1,27 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<phpunit
+	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+	xsi:noNamespaceSchemaLocation="http://schema.phpunit.de/9.0/phpunit.xsd"
+	bootstrap="Tests/bootstrap.php"
+	convertErrorsToExceptions="true"
+	convertWarningsToExceptions="true"
+	convertNoticesToExceptions="true"
+	convertDeprecationsToExceptions="true"
+	>
+	<testsuites>
+		<testsuite name="VariableAnalysis">
+			<directory>Tests</directory>
+		</testsuite>
+	</testsuites>
+
+	<filter>
+		<whitelist addUncoveredFilesFromWhitelist="true" processUncoveredFilesFromWhitelist="false">
+			<directory>./VariableAnalysis/</directory>
+		</whitelist>
+	</filter>
+
+	<logging>
+		<log type="coverage-text" target="php://stdout" showOnlySummary="true"/>
+		<log type="coverage-clover" target="build/logs/clover.xml"/>
+	</logging>
+</phpunit>