This repository has been archived by the owner on Mar 26, 2021. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 30
Expand tests to modern WordPress versions and PHP versions #46
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Based on research in https://github.com/npr/pmp-php-sdk/issues/36, Travis CI runs tests using the following PHPUnit versions: - PHP 5.6: PHPUnit 5.7.23 - PHP 7: PHPUnit 6.4.3 - PHP 7.1: PHPUnit 6.4.3 - PHP 7.2: PHPUnit 7.0.2 This commit reduces the number of tests that are run by more than two thirds. Before, we were running 4 different PHP versions and 11 diffferent WordPress versions. Many of those combinations were incompatible, for different reasons: 1. There are PHP warnings about deprecated PHP-4-style constructors. 2. Requires PHPUnit 5 because PHPUnit 6 support was not included until WordPress 4.7. 3. Unless there's been an update since [this trac thread](https://core.trac.wordpress.org/ticket/39822), WordPress tests still assume PHPUnit 6, and Travis runs PHP 7.2 tests with PHPUnit 7 | phpunit | 5.7 | 6.4 | 6.4 | 7.0 | | | | php | 5.6 | 7.0 | 7.1 | 7.2 | | | | WP | | | | | wp release year | then-current php | |---------|-----|-----|-----|-----|-----------------|------------------| | 3.8 | | 1 | 1 | 1 | 2013 | | | 3.9 | | 1 | 1 | 1 | 2014 | 5.5 | | 4.0 | | 1 | 1 | 1 | 2014 | 5.5 | | 4.1 | | 1 | 1 | 1 | 2014 | 5.5 | | 4.2 | | 2 | 2 | 2 | 2015 | 5.6 | | 4.3 | | 2 | 2 | 2 | 2015 | 5.6 | | 4.4 | | 2 | 2 | 2 | 2015 | 7.0 | | 4.5 | | 2 | 2 | 2 | 2016 | 7.0 | | 4.6 | | 2 | 2 | 2 | 2016 | 7.0 | | 4.7 | | | | 3 | 2016 | 7.1 | | 4.8 | | | | 3 | 2017 | 7.1 | | 4.9 | | | | 3 | 2017 | 7.2 | Accordingly, this commit removes PHP 7.2 from tests, and replaces the separate lists of PHP and WordPress versions with one list of selected PHP and WordPress versions.
Updated version of the table in 09d72a8: Based on research in npr/pmp-php-sdk#36, Travis CI runs tests using the following PHPUnit versions: - PHP 5.6: PHPUnit 5.7.23 - PHP 7: PHPUnit 6.4.3 - PHP 7.1: PHPUnit 6.4.3 - PHP 7.2: PHPUnit 7.0.2 Reasons that given versions of WordPress have tests incompatible with travis-ci.org: 1. There are PHP warnings about deprecated PHP-4-style constructors. 2. Requires PHPUnit 5 because PHPUnit 6 support was not included until WordPress 4.7. 3. Unless there's been an update since [this trac thread](https://core.trac.wordpress.org/ticket/39822), WordPress tests still assume PHPUnit 6, and Travis runs PHP 7.2 tests with PHPUnit 7 | phpunit | 5.7 | 6.4 | 6.4 | 7.0 | | | | php | 5.6 | 7.0 | 7.1 | 7.2 | | | | WP | | | | | wp release year | then-current php | |---------|-----|-----|-----|-----|-----------------|------------------| | 3.8 | | 1 | 1 | 1 | 2013 | | | 3.9 | | 1 | 1 | 1 | 2014 | 5.5 | | 4.0 | | 1 | 1 | 1 | 2014 | 5.5 | | 4.1 | | 1 | 1 | 1 | 2014 | 5.5 | | 4.2 | | 2 | 2 | 2 | 2015 | 5.6 | | 4.3 | | 2 | 2 | 2 | 2015 | 5.6 | | 4.4 | | 2 | 2 | 2 | 2015 | 7.0 | | 4.5 | | 2 | 2 | 2 | 2016 | 7.0 | | 4.6 | | 2 | 2 | 2 | 2016 | 7.0 | | 4.7 | | 2 | 2 | 2 | 2016 | 7.1 | | 4.8 | | | | 3 | 2017 | 7.1 | | 4.9 | | | | 3 | 2017 | 7.2 |
benlk
changed the title
Improve test coverage in all WordPress versions
Expand tests to modern WordPress versions and PHP versions
Mar 20, 2018
3 tasks
jgarlow
approved these changes
Apr 12, 2018
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good. Thanks for the thorough explanation.
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Changes
In
.travis.yml
, and thus in the automated tests that run on Travis CI:Why
This PR expands the versions tested because we want to make sure that the plugin is compatible with recent PHP and WordPress versions. The simple way to do that is to enumerate a list of WordPress versions and a list of PHP versions, and then run the tests at each intersection. Not only is this approach inefficient, it causes problems when old versions of WordPress include a test suite that assumes an old version of PHPUnit, which is not available when Travis CI only offers one version of PHPUnit per PHP version.
Based on research in npr/pmp-php-sdk#36,
Travis CI runs tests using the following PHPUnit versions:
Reasons that given versions of WordPress have tests incompatible with travis-ci.org: