Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow to configure parallel processing #92

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .github/workflows/php-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,10 @@ jobs:
- {name: "Rector module tests", run: sh tests/autoload-patch.sh && vendor/bin/phpunit --testsuit Rector}
- {name: "Sniffs module tests", run: vendor/bin/phpunit --testsuit Sniffs}
- {name: "composer audit", run: composer audit}
env:
EONX_EASY_QUALITY_JOB_SIZE: 20
EONX_EASY_QUALITY_MAX_NUMBER_OF_PROCESS: 32
EONX_EASY_QUALITY_TIMEOUT_SECONDS: 120
name: ${{ matrix.actions.name}} (${{ matrix.php }})
steps:
- uses: actions/checkout@v4
Expand Down
3 changes: 3 additions & 0 deletions config/phpstan/eonx.neon
Original file line number Diff line number Diff line change
@@ -1,3 +1,6 @@
includes:
- phpstan_parallel_settings.php

services:
-
class: EonX\EasyQuality\PHPStan\ThrowExceptionMessageRule
Expand Down
13 changes: 13 additions & 0 deletions config/phpstan/phpstan_parallel_settings.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
<?php
declare(strict_types=1);

use EonX\EasyQuality\Helper\ParallelSettingsHelper;

$config = [];
$config['parameters']['parallel'] = [
'jobSize' => ParallelSettingsHelper::getJobSize(),
'maximumNumberOfProcesses' => ParallelSettingsHelper::getMaxNumberOfProcess(),
'processTimeout' => (float)ParallelSettingsHelper::getTimeoutSeconds(),
];

return $config;
7 changes: 6 additions & 1 deletion quality/ecs.php
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
<?php
declare(strict_types=1);

use EonX\EasyQuality\Helper\ParallelSettingsHelper;
use EonX\EasyQuality\Sniffs\Arrays\AlphabeticallySortedArrayKeysSniff;
use EonX\EasyQuality\Sniffs\Attributes\SortedApiResourceOperationKeysSniff;
use EonX\EasyQuality\Sniffs\Classes\AvoidPublicPropertiesSniff;
Expand Down Expand Up @@ -30,7 +31,11 @@
__DIR__ . '/ecs.php',
__DIR__ . '/rector.php',
])
->withParallel(300, 32, 20)
->withParallel(
ParallelSettingsHelper::getTimeoutSeconds(),
ParallelSettingsHelper::getMaxNumberOfProcess(),
ParallelSettingsHelper::getJobSize()
)
->withSets([EasyQualitySetList::ECS])
->withSkip([
'tests/*/Fixture/*',
Expand Down
1 change: 1 addition & 0 deletions quality/phpstan.neon
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
includes:
- %currentWorkingDirectory%/vendor/jangregor/phpstan-prophecy/extension.neon
- %currentWorkingDirectory%/config/phpstan/eonx.neon

parameters:
bootstrapFiles:
Expand Down
7 changes: 6 additions & 1 deletion quality/rector.php
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
<?php
declare(strict_types=1);

use EonX\EasyQuality\Helper\ParallelSettingsHelper;
use EonX\EasyQuality\Rector\AddSeeAnnotationRector;
use EonX\EasyQuality\Rector\SingleLineCommentRector;
use EonX\EasyQuality\ValueObject\EasyQualitySetList;
Expand All @@ -16,7 +17,11 @@
__DIR__ . '/ecs.php',
__DIR__ . '/rector.php',
])
->withParallel(300, 32, 20)
->withParallel(
ParallelSettingsHelper::getTimeoutSeconds(),
ParallelSettingsHelper::getMaxNumberOfProcess(),
ParallelSettingsHelper::getJobSize()
)
->withImportNames(importDocBlockNames: false)
->withPhpSets(php81: true)
->withSets([
Expand Down
20 changes: 14 additions & 6 deletions readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,12 @@ This package is a way to centralise reusable classes used for coding standards a

1. Create a `quality` directory in your project root.
2. Go to the `quality` directory and run `composer require eonx-com/easy-quality`.
3. Add `quality/vendor` to Git ignore (either in a `.gitignore` file inside the `quality` directory or in you project's root `.gitignore` file).
4. Update your project's `composer.json` file by adding a post-install script (this will automate an installation of `eonx-com/easy-quality` on all the local machines):
3. Set env variables to speed up local execution:
- `EONX_EASY_QUALITY_JOB_SIZE` - number of files to process in parallel (default: 2)
- `EONX_EASY_QUALITY_MAX_NUMBER_OF_PROCESS` - maximum number of parallel processes (default: 4)
- `EONX_EASY_QUALITY_TIMEOUT_SECONDS` - timeout in seconds for each process (default: 120)
4. Add `quality/vendor` to Git ignore (either in a `.gitignore` file inside the `quality` directory or in you project's root `.gitignore` file).
5. Update your project's `composer.json` file by adding a post-install script (this will automate an installation of `eonx-com/easy-quality` on all the local machines):

```json
{
Expand All @@ -22,7 +26,7 @@ This package is a way to centralise reusable classes used for coding standards a
}
```

5. Update your project's `composer.json` file by adding the following scripts. Here we use `veewee/composer-run-parallel` (install it as a **dev** dependency) for the `check-all` script to run multiple commands in parallel. Feel free to modiy these
6. Update your project's `composer.json` file by adding the following scripts. Here we use `veewee/composer-run-parallel` (install it as a **dev** dependency) for the `check-all` script to run multiple commands in parallel. Feel free to modiy these
commands as you wish.

```json
Expand All @@ -38,9 +42,9 @@ This package is a way to centralise reusable classes used for coding standards a
}
```

6. Make sure you have config files for ECS, Rector, PHP Mess Detector, and PHPStan in the project source code root.
7. Run `composer check-all` from the project source code root to make sure everything is working and fix the found issues.
8. If you want to use the quality tools in CI, here is an example of a GitHub action configuration:
7. Make sure you have config files for ECS, Rector, PHP Mess Detector, and PHPStan in the project source code root.
8. Run `composer check-all` from the project source code root to make sure everything is working and fix the found issues.
9. If you want to use the quality tools in CI, here is an example of a GitHub action configuration:

```yaml
coding-standards:
Expand All @@ -59,6 +63,10 @@ This package is a way to centralise reusable classes used for coding standards a
- {name: security, run: composer check-security}
- {name: yaml-linter, run: './bin/console lint:yaml config src translations --parse-tags'}
- {name: ecs, run: composer check-ecs}
env:
EONX_EASY_QUALITY_JOB_SIZE: 20
EONX_EASY_QUALITY_MAX_NUMBER_OF_PROCESS: 32
EONX_EASY_QUALITY_TIMEOUT_SECONDS: 120

name: ${{ matrix.actions.name}} (${{ matrix.php }})

Expand Down
40 changes: 40 additions & 0 deletions src/Helper/ParallelSettingsHelper.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
<?php
declare(strict_types=1);

namespace EonX\EasyQuality\Helper;

final class ParallelSettingsHelper
{
private const DEFAULT_JOB_SIZE = 2;

private const DEFAULT_MAX_NUMBER_OF_PROCESS = 4;

private const DEFAULT_TIMEOUT_SECONDS = 120;

private const ENV_JOB_SIZE = 'EONX_EASY_QUALITY_JOB_SIZE';

private const ENV_MAX_NUMBER_OF_PROCESS = 'EONX_EASY_QUALITY_MAX_NUMBER_OF_PROCESS';

private const ENV_TIMEOUT_SECONDS = 'EONX_EASY_QUALITY_TIMEOUT_SECONDS';

public static function getJobSize(): int
{
$jobSize = \getenv(self::ENV_JOB_SIZE);

return $jobSize ? (int)$jobSize : self::DEFAULT_JOB_SIZE;
}

public static function getMaxNumberOfProcess(): int
{
$maxNumberOfProcess = \getenv(self::ENV_MAX_NUMBER_OF_PROCESS);

return $maxNumberOfProcess ? (int)$maxNumberOfProcess : self::DEFAULT_MAX_NUMBER_OF_PROCESS;
}

public static function getTimeoutSeconds(): int
{
$timeoutSeconds = \getenv(self::ENV_TIMEOUT_SECONDS);

return $timeoutSeconds ? (int)$timeoutSeconds : self::DEFAULT_TIMEOUT_SECONDS;
}
}