Skip to content

Commit

Permalink
Tests: allow for running on PHPUnit 10/11
Browse files Browse the repository at this point in the history 
Until now, the tests would run on PHPUnit 9 for PHP 7.3 and higher.

PHPUnit 10 (PHP 8.1+) was released in February 2023, PHPUnit 11 (PHP 8.2+) in February 2024 and the update for this test suite is quite painless, so let's allow for running the tests on PHPUnit 10 and 11.

This commit actions this.

Things to be aware of:

Handling of PHPUnit deprecations

Prior to PHPUnit 10.5.32 and PHPUnit 11.3.3, if the `failOnDeprecation` attribute was set (previously: `convertDeprecationsToExceptions`), tests runs would not only fail on PHP deprecation notices, but also on PHPUnit native deprecation notices when running on PHPUnit 10/11.

This undesireable behaviour has now (finally) been reverted, which makes updating test suites for compatibility with PHPUnit 10/11 a lot more straight-forward.

This is the reason that the minimum PHPUnit 10/11 versions in the `composer.json` file are set to such specific versions.

As part of this change, PHPUnit 10.5.32 and 11.3.3 introduce two new options for both the configuration file and as CLI arguments: `displayDetailsOnPhpunitDeprecations` and `failOnPhpunitDeprecation`, both of which will default to `false`.

These options have been added to the PHPUnit configuration for PHPUnit 10/11 to safeguard them against changes in the default value.

Test configuration file changes

The configuration file specification has undergone changes in PHPUnit 9.3 and 10.0.

Most notably:
* In PHPUnit 9.3, the manner of specifying the code coverage configuration has changed.
* In PHPUnit 10.0, a significant number of attributes of the `<phpunit>` element were removed or renamed.

While the `--migrate-configuration` command can upgrade a configuration for the changes in the format made in PHPUnit 9.3, some of the changes in the configuration format in PHPUnit 10 don't have one-on-one replacements and/or are not taken into account.

With that in mind, I deem it more appropriate to have a dedicated PHPUnit configuration file for PHPUnit 10+ to ensure the test run will behave as intended.

To that end:
* The `phpunit.xml.dist` file has been renamed to `phpunitlte9.xml.dist` (for PHPUnit <= 9) and a new `phpunit.xml.dist` file has been added for PHPUnit 10+.
* The renamed file is set to be excluded from release archives via `.gitattributes` and a potential local overload file for it is set to be ignored by default via the `.gitignore` file.
* Additional scripts have been added to the `composer.json` file to run the tests either with the new or the old configuration file.
* The GH Actions `test` workflow has been updated to select the correct configuration file based on the PHPUnit version on which the tests are being run.

Other command/config changes

Also note that the `--coverage-cache <dir>` CLI flag was deprecated in PHPUnit 10 and removed in PHPUnit 11. A generic `cacheDirectory` attribute is the recommended replacement (though a CLI option is also available).
The new attribute has been added to the PHPUnit 10+ config file, while for PHPUnit 9.3 - 9.5, the flag is still added to the command in CI (as adding the PHPUnit 9.3 equivalent to the config file would make the config incompatible with PHPUnit < 9.3).

Ref:
* https://phpunit.de/announcements/phpunit-11.html
* https://phpunit.de/announcements/phpunit-10.html
  • Loading branch information
@jrfnl
jrfnl committed Sep 8, 2024
1 parent 9211ff6 commit f1cfb66
Show file tree
Hide file tree
Showing 6 changed files with 94 additions and 31 deletions.
15 changes: 8 additions & 7 deletions .gitattributes
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,14 @@
# https://www.reddit.com/r/PHP/comments/2jzp6k/i_dont_need_your_tests_in_my_production
# https://blog.madewithlove.be/post/gitattributes/
#
/.gitattributes export-ignore
/.gitignore export-ignore
/phpcs.xml.dist export-ignore
/phpstan.neon.dist export-ignore
/phpunit.xml.dist export-ignore
/.github export-ignore
/Tests/ export-ignore
/.gitattributes export-ignore
/.gitignore export-ignore
/phpcs.xml.dist export-ignore
/phpstan.neon.dist export-ignore
/phpunit.xml.dist export-ignore
/phpunitlte9.xml.dist export-ignore
/.github export-ignore
/Tests/ export-ignore

#
# Auto detect text files and perform LF normalization
Expand Down
40 changes: 31 additions & 9 deletions .github/workflows/test.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -111,8 +111,21 @@ jobs:
composer-options: --ignore-platform-req=php
custom-cache-suffix: $(date -u "+%Y-%m")

- name: Grab PHPUnit version
id: phpunit_version
run: echo "VERSION=$(vendor/bin/phpunit --version | grep --only-matching --max-count=1 --extended-regexp '\b[0-9]+\.[0-9]+')" >> $GITHUB_OUTPUT

- name: Determine PHPUnit config file to use
id: phpunit_config
run: |
if [ "${{ startsWith( steps.phpunit_version.outputs.VERSION, '10.' ) || startsWith( steps.phpunit_version.outputs.VERSION, '11.' ) }}" == "true" ]; then
echo 'FILE=phpunit.xml.dist' >> $GITHUB_OUTPUT
else
echo 'FILE=phpunitlte9.xml.dist' >> $GITHUB_OUTPUT
fi
- name: Run the unit tests
run: composer test
run: vendor/bin/phpunit --no-coverage -c ${{ steps.phpunit_config.outputs.FILE }}

#### CODE COVERAGE STAGE ####
# N.B.: Coverage is only checked on the lowest and highest stable PHP versions
Expand Down Expand Up @@ -178,21 +191,30 @@ jobs:
- name: "DEBUG: Show grabbed version"
run: echo ${{ steps.phpunit_version.outputs.VERSION }}

- name: Determine PHPUnit config file to use
id: phpunit_config
run: |
if [ "${{ startsWith( steps.phpunit_version.outputs.VERSION, '9.' ) && steps.phpunit_version.outputs.VERSION >= '9.3' }}" == "true" ]; then
echo 'FILE=phpunitlte9.xml.dist' >> $GITHUB_OUTPUT
echo 'EXTRA_ARGS=--coverage-cache ./build/phpunit-cache' >> $GITHUB_OUTPUT
elif [ "${{ startsWith( steps.phpunit_version.outputs.VERSION, '10.' ) || startsWith( steps.phpunit_version.outputs.VERSION, '11.' ) }}" == "true" ]; then
echo 'FILE=phpunit.xml.dist' >> $GITHUB_OUTPUT
echo 'EXTRA_ARGS=' >> $GITHUB_OUTPUT
else
echo 'FILE=phpunitlte9.xml.dist' >> $GITHUB_OUTPUT
echo 'EXTRA_ARGS=' >> $GITHUB_OUTPUT
fi
# PHPUnit 9.3 started using PHP-Parser for code coverage which can cause interference.
# As of PHPUnit 9.3.4, a cache warming option is available.
# Using that option prevents issues with PHP-Parser backfilling PHP tokens when PHPCS does not (yet),
# which would otherwise cause tests to fail on tokens being available when they shouldn't be.
- name: "Warm the PHPUnit cache (PHPUnit 9.3+)"
if: ${{ steps.phpunit_version.outputs.VERSION >= '9.3' }}
run: vendor/bin/phpunit --coverage-cache ./build/phpunit-cache --warm-coverage-cache

- name: "Run the unit tests with code coverage (PHPUnit < 9.3)"
if: ${{ steps.phpunit_version.outputs.VERSION < '9.3' }}
run: composer coverage
run: vendor/bin/phpunit -c ${{ steps.phpunit_config.outputs.FILE }} ${{ steps.phpunit_config.outputs.EXTRA_ARGS }} --warm-coverage-cache

- name: "Run the unit tests with code coverage (PHPUnit 9.3+)"
if: ${{ steps.phpunit_version.outputs.VERSION >= '9.3' }}
run: composer coverage -- --coverage-cache ./build/phpunit-cache
- name: "Run the unit tests with code coverage"
run: vendor/bin/phpunit -c ${{ steps.phpunit_config.outputs.FILE }} ${{ steps.phpunit_config.outputs.EXTRA_ARGS }}

- name: Upload coverage results to Coveralls
if: ${{ success() }}
Expand Down
1 change: 1 addition & 0 deletions .gitignore
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
/vendor/
/build/
phpunit.xml
phpunitlte9.xml
.phpunit.result.cache
.phpcs.xml
phpcs.xml
Expand Down
4 changes: 3 additions & 1 deletion composer.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,8 @@
"scripts": {
"test": "./vendor/bin/phpunit --no-coverage",
"coverage": "./vendor/bin/phpunit",
"test-lte9": "./vendor/bin/phpunit -c phpunitlte9.xml.dist --no-coverage",
"coverage-lte9": "./vendor/bin/phpunit -c phpunitlte9.xml.dist",
"lint": "./vendor/bin/phpcs",
"fix": "./vendor/bin/phpcbf",
"phpstan": "./vendor/bin/phpstan analyse",
Expand All @@ -52,7 +54,7 @@
"squizlabs/php_codesniffer": "^3.5.6"
},
"require-dev": {
"phpunit/phpunit": "^4.8.36 || ^5.7.21 || ^6.5 || ^7.0 || ^8.0 || ^9.0",
"phpunit/phpunit": "^4.8.36 || ^5.7.21 || ^6.5 || ^7.0 || ^8.0 || ^9.0 || ^10.5.32 || ^11.3.3",
"sirbrillig/phpcs-import-detection": "^1.1",
"phpcsstandards/phpcsdevcs": "^1.1",
"phpstan/phpstan": "^1.7",
Expand Down
38 changes: 24 additions & 14 deletions phpunit.xml.dist
View file
Original file line number Diff line number Diff line change
@@ -1,27 +1,37 @@
<?xml version="1.0" encoding="UTF-8"?>
<phpunit
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:noNamespaceSchemaLocation="http://schema.phpunit.de/9.0/phpunit.xsd"
bootstrap="Tests/bootstrap.php"
convertErrorsToExceptions="true"
convertWarningsToExceptions="true"
convertNoticesToExceptions="true"
convertDeprecationsToExceptions="true"
xsi:noNamespaceSchemaLocation="http://schema.phpunit.de/10.5/phpunit.xsd"
bootstrap="Tests/bootstrap.php"
cacheDirectory="build/.phpunit-cache"
displayDetailsOnTestsThatTriggerErrors="true"
displayDetailsOnTestsThatTriggerWarnings="true"
displayDetailsOnTestsThatTriggerNotices="true"
displayDetailsOnTestsThatTriggerDeprecations="true"
displayDetailsOnIncompleteTests="true"
displayDetailsOnSkippedTests="true"
displayDetailsOnPhpunitDeprecations="false"
failOnWarning="true"
failOnNotice="true"
failOnDeprecation="true"
failOnPhpunitDeprecation="false"
>
<testsuites>
<testsuite name="VariableAnalysis">
<directory>Tests</directory>
</testsuite>
</testsuites>

<filter>
<whitelist addUncoveredFilesFromWhitelist="true" processUncoveredFilesFromWhitelist="false">
<source>
<include>
<directory>./VariableAnalysis/</directory>
</whitelist>
</filter>
</include>
</source>

<logging>
<log type="coverage-text" target="php://stdout" showOnlySummary="true"/>
<log type="coverage-clover" target="build/logs/clover.xml"/>
</logging>
<coverage includeUncoveredFiles="true" ignoreDeprecatedCodeUnits="true">
<report>
<clover outputFile="build/logs/clover.xml"/>
<text outputFile="php://stdout" showOnlySummary="true"/>
</report>
</coverage>
</phpunit>
27 changes: 27 additions & 0 deletions phpunitlte9.xml.dist
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
<?xml version="1.0" encoding="UTF-8"?>
<phpunit
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:noNamespaceSchemaLocation="http://schema.phpunit.de/9.0/phpunit.xsd"
bootstrap="Tests/bootstrap.php"
convertErrorsToExceptions="true"
convertWarningsToExceptions="true"
convertNoticesToExceptions="true"
convertDeprecationsToExceptions="true"
>
<testsuites>
<testsuite name="VariableAnalysis">
<directory>Tests</directory>
</testsuite>
</testsuites>

<filter>
<whitelist addUncoveredFilesFromWhitelist="true" processUncoveredFilesFromWhitelist="false">
<directory>./VariableAnalysis/</directory>
</whitelist>
</filter>

<logging>
<log type="coverage-text" target="php://stdout" showOnlySummary="true"/>
<log type="coverage-clover" target="build/logs/clover.xml"/>
</logging>
</phpunit>

0 comments on commit f1cfb66

Please sign in to comment.