Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Previously the BuildPlan pipeline didn't execute generators and transformers concurrently. All steps were sequentially executed. Holos was primarily concurrent by executing multiple BuildPlans at once. This patch changes the Build implementation for each BuildPlan to execute a GoRoutine pipeline. One producer fans out to a group of routines each executing the pipeline for one artifact in the build plan. The pipeline has 3 stages: 1: Fan-out to build each Generator concurrently. 2: Fan-in to build each Transformer sequentially. 3: Fan-out again to run each validator concurrently. When the artifact pipelines return, the producer closes the tasks channel causing the worker tasks to return. Note the overall runtime for 8 BuildPlans is roughly equivalent to previously at 160ms with --concurrency=8 on my M3 Max. I expect this to perform better than previously when multiple artifacts are rendered for each BuildPlan.
- Loading branch information