FR: Add parallel execution option and task execution stages

[paslandau]

Q	A
Version	0.14.3
Bug?	no
New feature?	yes
Question?	yes
Documentation?	probably
Related tickets

On bigger code bases and / or long task chains, the grumphp execution can take a considerable amount of time. Most of the performed tasks are probably unrelated and don't affect each other (e.g. large_file and phpstan can run independently). Thus, running those tasks in parallel would greatly improve performance.

On the other hand, there might be tasks that should run in a certain order, e.g. parallel-lint should usually be run before phpcs or phpstan as failing early makes sense in this case. Thus, there should be a way to "group" tasks and define an execution order.

Proposal

Consider the following grumphp.yml

parameters:
  processes: 4
  tasks:
    parallel-lint: 
      stage: 0
    composer: 
      stage: 1
    composer_require_checker:
      stage: 1

processes denotes the number of parallel running tasks (can be omitted or 1 for to keep current behavior / avoid BC)
stage acts a an alternative to the already supported priority option. The value has to be an integer so that

• stages can be run in a predefined order (same logic as priority)
• tasks can be grouped into one stage
• all tasks in the same stage can run in parallel
  -priority is ignored if stages are used

Use cases

• Already mentioned in the description: This will greatly (usually ~linear to the number of cores) improve performance.

Implementation

This is probably a bigger change. Possible solution:

• use a task runner that already implements parallel execution capabilities (e.g. Robos ParallelExec)
  • if that is not desirable / to big of a change, re-implementing https://github.com/consolidation/Robo/blob/954f17bfbdd78b829c0eae88055a0446ba589aed/src/Task/Base/ParallelExec.php does not seem to be an overwhelming task (this would actually have the benefit of sneaking "throttling" in there as well ;))
• if Provide option to run a single/list of tasks from the config #580 is implemented, the problem can be reduced to "find the tasks for the current stage" and "start those tasks as individual grumphp tasks" - although the specifics of how to handle / merge the results have to be hashed out. Not deep enough in the code to judge that :|
• since parallel execution usually adds another layer of complexity, I would introduce that feature explicitly as "beta" to get some real world feedback before "officially" announce it as supported.

Thoughts?

Cheers
Pascal